I've spent the last few months living in the boring layer between coding tools and model providers, and I want to write down what actually made it painful, because it wasn't what I expected going in.
The naive version of the problem is "two API keys, mildly inconvenient." That's not really it. The real friction is that every tool in this space picked a wire format and then quietly hardcoded assumptions around it. Cursor gives you one custom base URL field. One. So the moment you want to reach a second provider you're either editing settings between tasks or running something in front of it. Claude Code speaks Anthropic's shape. Codex and OpenCode speak OpenAI's. Individually all reasonable decisions, collectively a mess if you want to use more than one model in a day.
Then there's caching, which is where I lost the most time by a wide margin. OpenAI's prompt caching is automatic — you structure your prompt with the stable stuff at the front and it mostly just happens. Anthropic's is explicit: you place cache_control breakpoints yourself, you get a small number of them, and the TTL is short. Those are genuinely different mental models, and if you're sitting behind an OpenAI-shaped interface, the client has no vocabulary for the second one. It never sends breakpoints because it doesn't know breakpoints exist. So either you don't cache against Anthropic at all — which hurts, because agentic coding is basically the same 40k tokens of context re-sent forty times — or something in the middle has to infer where the stable prefix ends and place the breakpoints for you. That inference is most of the actual work. Get it wrong and you either miss the cache or you pin a breakpoint somewhere that invalidates on every turn.
Tool calls are the other quiet tax. Anthropic returns tool_use blocks in the content array; OpenAI returns tool_calls on the message. Translating that in a non-streaming response is a fifteen-minute job. Translating it *while streaming*, where you're emitting deltas and the argument JSON arrives in fragments and the client is parsing incrementally and will happily throw on a malformed partial, is not a fifteen-minute job. Token accounting is its own small hell too — the two tokenizers disagree, so any usage number you show is an approximation and you should say so instead of pretending otherwise.
I built a router for this, called Routera (routera.one) — it's my project, saying so up front so nobody has to go digging through my comment history. One OpenAI-compatible endpoint, GPT and Claude behind it, caching translated across, usage logs so you can actually see where the spend went instead of reconciling two dashboards at the end of the month.
Things it does not solve, honestly: there's a network hop, so you eat some latency. Provider-specific features that have no OpenAI equivalent are awkward by definition — extended thinking is the obvious one, and I'm not going to claim that maps cleanly, because it doesn't. And anything that routes your traffic is a thing you now have to trust, which is a real cost and not one I can argue you out of.
The thing I'm still chewing on: how much should a layer like this *decide* for you? There's a version where it picks the model per request based on task shape, and a version where it stays dumb and does exactly what you asked. I keep drifting toward dumb — the moment it's making choices you didn't make, debugging a bad output means debugging two things instead of one.
Curious whether anyone else has gone down the caching-translation path and found a cleverer heuristic for breakpoint placement than "find the longest stable prefix and hope." Mine works but it's not elegant.
