Distill a repeated Jev classification task into a local model, on the fly — same answers, your hardware.
-
Updated
Oct 17, 2026 - Python
#
model-cascade
Here are 13 public repositories matching this topic...
⚖️ Kadı — a calibrated judge in front of your frontier LLM. Routine decisions answered locally with a conformal-prediction guarantee (≤ α disagreement with the oracle); everything else escalated and learned from. Zero-dependency Python.
python privacy zero-dependency uncertainty-quantification knowledge-distillation online-learning gdpr ai-agents claude conformal-prediction mlops guardrails kvkk distribution-shift llm anthropic adaptive-conformal-inference llm-routing llm-cost-optimization model-cascade
-
Updated
Oct 11, 2026 - Python
Distil an expensive LLM API call on a narrow task into a small local model: capture traffic, curate, LoRA fine-tune with MLX on Apple Silicon, evaluate against the teacher, and serve an OpenAI-compatible cascade that escalates low-confidence requests.
text-classification information-extraction calibration lora knowledge-distillation mlx cost-optimization fine-tuning apple-silicon llm llmops qwen llm-evaluation openai-compatible model-cascade
-
Updated
Sep 30, 2026 - Python
Cost-aware routing across agentic pipelines. Empirical benchmarks of accuracy, cost, and dangerous rate.
-
Updated
May 23, 2026 - Python
Confidence-gated LLM cascades that measure how often the cheap tier is wrong on the cases it answers alone.
-
Updated
Oct 16, 2026 - Python
Studying Minimum Sufficient Inference: when objective execution evidence can stop LLM inference without sacrificing reliability.
inference code-generation ml-systems reliable-ai llm ollama llm-evaluation model-routing llm-cascade model-cascade
-
Updated
Oct 17, 2026 - Python
NSGA-II search framework for CIFAR-10 big/little dynamic inference cascades under embedded memory constraints.
-
Updated
May 28, 2026 - Python
LLM serving router that prices every request in dollars and milliseconds. Semantic cache plus a confidence-gated cheap-to-GPT-4 cascade: 96.9% lower cost than always calling the strong model, within 1.6 accuracy points, median latency 450ms to sub-ms on repeated traffic. FastAPI, SQLite cost ledger, reproducible benchmark.
python latency embeddings cost-optimization finops mlops inference-optimization fastapi vector-search gpt-4 llm llm-serving semantic-cache model-routing model-cascade
-
Updated
Jul 27, 2026 - Python
A cost-aware decision layer for routing text requests: a cheap model first, an LLM only when confidence is low, a person when both are unsure. FastAPI service with a React dashboard, measured for accuracy, cost and latency.
react python docker typescript text-classification scikit-learn routing mlops fastapi llm model-cascade
-
Updated
Oct 7, 2026 - TypeScript
Adaptive Cascade Tuning via Importance Sampling
-
Updated
Sep 22, 2026 - Python
Ask the cheap model first. Pay for the strong one only when the cheap answer fails a check, under a budget, with a cost record for every call.
-
Updated
Aug 23, 2026 - TypeScript
Route each LLM call to the cheapest model that will get it right. Verifier-gated cascades with an offline threshold optimizer — 94% of the strongest model's accuracy at 34% lower cost.
-
Updated
Sep 23, 2026 - Python
Divergence-aware multi-agent routing that cuts LLM inference cost 10-100x. One signal routes queries, keys a multilingual cache, and detects hallucinations at AUC 0.90. CIKM 2026.
nlp multi-agent cost-optimization inference-optimization financial-sentiment-analysis finbert llm llm-serving semantic-cache llm-inference hallucination-detection llm-routing model-cascade cikm2026
-
Updated
Sep 14, 2026 - Python
Add this topic to your repo
To associate your repository with the model-cascade topic, visit your repo's landing page and select "manage topics."