AI and machine learning engineer building agentic systems and inference engines from client side to server side.
- AI Researcher
- Finetuning PyTorch models to faster frameworks
- Distilled decision models for inference or games like mario
- Real-time audio AI (transcription, speaker diarization, TTS, VAD, etc)
- Server-side LLM serving: request routing, disaggregated prefill/decode, and diffusion serving in vLLM, SGLang and NVIDIA Dynamo
- Inference Engineering, On-Device ML Engineering, Server-Side ML Engineering, Performance Engineering, Agentic Engineering, ML Engineering
Projects I work on — at Fluid Inference, which was acquired by Levenza
- FluidAudio — speech AI SDK for iOS and macOS (3.2K+ ⭐)
- Supports dozens of commercial applications
- Parakeet TDT V3 Core ML: over 2,000,000 total downloads, peaked at 500,000 monthly downloads
- FluidUse — Core ML runtimes for decision models in Swift, including Decision 3.0 (
D3Manager) - möbius — model conversion and deployment across accelerators
- 500,000+ monthly model downloads across the Core ML models on FluidInference
Open-source collaborations and apps
- SGLang — fixed the diffusion LoRA merge cache silently reusing stale merged weights after an adapter is rewritten in place, with regression tests (#44204) (38K+ ⭐)
- Supertonic — ported Supertonic 3 to Core ML for Apple Neural Engine inference on iOS and macOS (conversion mobius#64, runtime FluidAudio#617) (13.7K+ ⭐)
- NVIDIA Dynamo — made the disaggregated-serving prefill router fail closed with a retryable 503 while a prefill target is still activating, instead of silently sending requests to decode-only workers (#16736) (8.4K+ ⭐)
- mlx-audio — added Kokoro-82M v1.1 Mandarin support in MLX (#341) (8.1K+ ⭐)
- vLLM Semantic Router (6.3K+ ⭐)
- Member of two workgroups: Router Models & Inference Runtime and Developer Experience & Ecosystem
- Early access to Decision 3.0: ported d3 to Core ML (d3-coreml)
- Wrote the Decision 3.0 on Apple Silicon blog post: d3-lite, nano and mini on Core ML, matching upstream on JevBench and typed-decisions, 2.5–4× faster per request on a Mac (#4905, #4975)
- Fixed external RAG result ranking across query windows (#3750)
- Secured Playground feedback for read users (#3755)
- Hardened Dashboard API authorization with mutation-boundary rechecks (a3174df)
- IREE (4K+ ⭐)
- vLLM Production Stack — fixed router static service discovery to reject mismatched model list configs at startup instead of failing at request time (#1084) (2.6K+ ⭐)
- OpenOats — contributed its horizontal wordmark for marketing materials (2.5K+ ⭐)
- Jobpulse — built an AI-powered job search site for beginners that peaked at 10,000 monthly users
Some Experiment Highlights (Swift runtimes in FluidUse)
- laya — Convai's typed decision model with 99.5% of ops on the Apple Neural Engine, 3.7 ms per decision on an M5 Pro (demo) · 1,323 likes · 111 reposts · 120K views
@jackzampolin |
@JiliJeanlouis |
@ClementDelangue reposts |
@victormustar |
- GLiNER2.5-Decide — Fastino's 340M decision model on Core ML: ~4× faster, ~5× less peak RAM, half the size (demo) · 880 likes · 83 reposts · 69K views
- EmbeddingGemma 2 — text encoder 100% on the Apple Neural Engine; sorts 10,080 posts into topics live at ~620/s on a MacBook, any topic splits into subtopics in under a second (demo) · 693 likes · 51 reposts · 34K views
https://x.com/SeraAndroid/status/2108629662238961694?s=20
- clef-text-0.6B — distilled Cloudflare's 9B clef-flash into a 0.6B decision model that runs 100% on the Apple Neural Engine; 1,000 support tickets triaged in 57 s on an M5 Pro, 48 ms per ticket, within a few points of the 9B on classification (demo) · 494 likes · 45 reposts · 32K views
@art_zucker reposts |
@ClementDelangue reposts |
- Decision 3.0 — d3-lite, nano and mini on Core ML; d3-lite triages 1,000 pull requests into vLLM-SR workgroups live on an M5 Pro at 0.12 s per PR, 8.3 PRs/s (demo) · 432 likes · 42 reposts · 28K views
@ClementDelangue |
@victormustar |
Resume · Twitter · LinkedIn · HuggingFace
