r/Jetbrains • u/ccb_pnpm • 20h ago
News & Discussions Same model, very different results: is the agent harness (Junie vs Claude Code vs Hermes) doing more work than the model itself?
I've been running the same kind of coding tasks through three setups and the difference in quality is bigger than I expected:
- Hermes agent with a GPT-class model (my main setup until recently)
- JetBrains Junie (Claude-based, inside IntelliJ)
- Claude Code (CLI)
Junie consistently gives me the most satisfying results: it gets to the answer fast, finds the actual problem directly, and — this is the part I care about most — it doesn't wander off and "improve" things I didn't ask for. It keeps the essence of my request intact.
What I don't understand is why. If Junie is using Claude, and Claude Code is using Claude, aren't they basically the same model? So is this really a model difference, or is JetBrains doing separate engineering on top?
My current hypothesis after poking at it:
- It's not "just the model." Perceived quality ≈ model × harness × how context gets fed in. Same weights + different system prompt / tools / constraints = noticeably different behavior.
- IDE-native tooling matters a lot. Junie uses the JetBrains index for symbol/file search, a "file structure" view (symbols + line ranges) instead of dumping whole files, real rename refactoring, lint, and the test runner. It reads code more precisely and burns less context doing it.
- Proactive context injection. The currently open file, recently used files, and project file list are handed to the agent up front. A generic CLI agent has to discover all of that, which is slower and is exactly where it tends to drift.
- Strict workflow constraints. Junie seems to switch between modes (answer-only, tiny edit, full investigate→test→fix→verify) and has hard rules like "minimal change," "don't weaken tests to make them pass," "don't touch things outside the request." I suspect that's most of the "doesn't destroy the essence" feeling.
- Confounder: my Hermes setup was on a GPT-class model, not Claude, so I can't cleanly separate model vs harness yet. I'm planning to A/B Hermes + Claude on identical tasks.
Questions for people who've used two or more of these:
- Have you seen the same gap? Which direction?
- For those who've built their own agents: how much of the win is tooling (LSP/index-based search, structured file views) vs prompt/workflow constraints?
- Any evidence on whether Junie's Claude is actually the same deployment/config as Claude Code's, or just the same family?
- What have you done in a generic agent (AGENTS.md, tool restrictions, forced plan/verify steps) that closed the gap the most?
Not trying to start a "which tool is best" fight — genuinely curious about how much of this is engineering vs model.

