Reddibase turns human-solved discussion threads into answer-extraction datasets and identification models.
The core artifact is a source-neutral Thread: a vague request or problem,
its messages, and optional weak answer signals from the source. Reddit is the
first adapter, but the classifier and identifier work from threads.jsonl, not
Reddit-specific post objects.
Each model ships two pieces:
- Resolved-thread classifier: reads a discussion thread and identifies the message that contains the resolved answer.
- Identification model: embeds vague descriptions and retrieves ranked
matches from confirmed
(description -> answer)pairs.
The pipeline is:
source adapter -> threads.jsonl
threads.jsonl -> resolved-thread classifier
threads.jsonl + classifier + solution extractor -> confirmed_pairs.jsonl
confirmed_pairs.jsonl -> SentenceTransformer + FAISS index
Install dependencies:
pip install -r requirements.txtScrape Reddit/Arctic Shift directly into source-neutral threads:
python -m scripts.scrape_threads tipofmyjoystick \
--output data/tipofmyjoystick/threads.jsonlAudit the thread artifact before training:
python -m scripts.audit_threads tipofmyjoystick \
--threads data/tipofmyjoystick/threads.jsonlTrain the resolved-thread classifier:
python -m scripts.train_classifier tipofmyjoystick \
--threads data/tipofmyjoystick/threads.jsonlExtract confirmed answer pairs:
python -m scripts.build_dataset tipofmyjoystick \
--threads data/tipofmyjoystick/threads.jsonlUpgrade an older pair artifact with canonical answers:
python -m scripts.canonicalize_pairs models/tipofmyjoystick/confirmed_pairs.jsonlTrain the identification model and FAISS index:
python -m scripts.train_embedder tipofmyjoystickEvaluate identifier retrieval before publishing:
python -m scripts.evaluate_identifier tipofmyjoystick --sample-size 5000 --top-k 10The same flow is available as small local workflow notebooks:
notebooks/scrape.ipynb
notebooks/classifier.ipynb
notebooks/identifier.ipynb
notebooks/evaluate.ipynb
Once a model is trained and registered:
GET /models/{model_id}/search?q=your+description&top_k=10
Run the API locally:
uvicorn api.main:app --reloadRun the web UI from web/:
npm install
npm run dev| Model | Source | Community | Pairs |
|---|---|---|---|
| tipofmyjoystick | r/tipofmyjoystick | - |
framework/
schema.py Source-neutral Thread, Message, ConfirmedPair dataclasses
threads.py JSONL read/write helpers for Thread artifacts
scraper.py Reddit/Arctic Shift adapter that streams Thread objects
classifier.py Resolved-thread classifier training + inference
solution.py Canonical answer extraction from selected solution comments
embedder.py Identification model + FAISS index
scripts/
scrape_threads.py Scrape source-neutral threads.jsonl
audit_threads.py Audit weak-answer and message coverage before training
train_classifier.py Train resolved-thread classifier
build_dataset.py Extract confirmed pairs from threads
canonicalize_pairs.py Add canonical answer fields to existing pairs
train_embedder.py Train identifier + build FAISS
evaluate_identifier.py Measure same-answer retrieval quality
models/
tipofmyjoystick/
config.yaml
api/
web/