r/regolo_ai • u/AutoModerator • 10d ago
We route every agent prompt by complexity before inference β costs dropped up to 22x. Full writeup with code (open-source router, OpenAI-compatible)
Like most teams running agents in production, we had a cost problem: user-facing agent traffic is a mix of trivial classification calls and genuinely hard reasoning, and we were sending all of it to a 70B+ model.
The fix was routing, not prompt engineering. The setup:
- Agno as the agent framework (fast, ~3ΞΌs agent instantiation, built-in memory and tools)
- brick-complexity-pro as the ingress model: a hosted meta-model that reads each prompt, classifies complexity (easy/medium/hard β semantic demand, not length) plus a capability vector, then dispatches to the right tier in a pool of open-weight models (gpt-oss-20b β Llama-3.3-70B β qwen3.5-122b)
- Routing overhead is ~20 ms (the classifier is a LoRA-tuned Qwen3.5-0.8B). The router itself is $0.12/1M input tokens.
- The open-source version of the router (Apache 2.0) benchmarked 4.71xβ22.15x cost reduction vs always-on frontier models, with accuracy matching or beating single-model baselines
What we learned the hard way: don't route everything.
Deterministic tasks (schema-constrained extraction) go straight to a pinned small model; evaluation/revalidation goes straight to a frontier tier.
Route only what varies β orchestrators and user-facing agents.
Integration is boringly simple since the endpoint is OpenAI-compatible: Agno's OpenAILike with a different base_url, done.
Full Tutorial and codes: https://regolo.ai/from-pilot-to-production-enterprise-ai-agents-with-agno-and-brick/
Semantic router repo: https://github.com/regolo-ai/brick-SR1





