NOVA V2: High-Performance Standalone Rust Inference Pipeline & Shard Routing Substrate. Multi-account SWRR credential pooling, lock-free continuous token buckets, canary circuit breakers, resumable ring buffers, and duplex WebSocket streaming.
rust standalone load-balancer reverse-proxy tokio server-sent-events high-availability rate-limiter token-bucket circuit-breaker inference-pipeline websocket-streaming axum llm ai-gateway inference-gateway openai-compatible swrr
-
Updated
Oct 25, 2026 - Rust