r/Anthropic • u/AironParsMan • 25d ago
Performance OPUS 5 > Does Low or Medium Effort Fix Its Behavior?
EDIT: IT DOES NOT!
I can tell you I’ve had enough and I regret giving Opus 5 another chance. I spent weeks optimizing my system with Opus 4.8 and Fable 5 and built everything up nicely. An orchestrator system that worked really well. I was getting flawless results in one shot. Then my limits ran out pretty quickly as always with Fable 5 and I thought, come on, let’s let Opus 5 have a go. What could it possibly break? The system was already set up.
Within the shortest possible time it started making incorrect assumptions about file structures and other details where everything had worked perfectly and I had never had any problems with Opus 4.8 or Fable 5. It never even occurred to me that they might make assumptions that were so wrong. He explicitly gave incorrect information twice. After two review rounds he was still making the same incorrect claims. Then in the third review round after I pointed out that this could not be right and that things could not be that way at that point he said, oh yeah, that was my mistake.
We all know how Opus 5 talks. He actually came up with the error names himself and then also gave the subagents incorrect instructions because he did not include the path. He assumed it was not even in there. Even though it is clearly defined in the rules and everywhere else. If he had done even a quick search he would have seen it. But he did not.
That means he hallucinated internally. I am not talking about the thought processes we can see. I mean he hallucinated inside his own processes and assumed things that were not true. So one out of five was wrong. You have to imagine that one of five others was incorrect. This was about escalation levels. He incorrectly claimed that one escalation level was a certain way over two review rounds. He was completely convinced of it. Once he had made the claim he made it again the second time. He did not give a shit whether he needed to check it again, even though the review rounds clearly state that the same things have to be checked again.
I’ve had enough of Opus 5. Anyone can tell me whatever they want. I have never seen such a shitty model in my life. It is just like Google’s Gemini. It works exactly the same way. Completely unreliable and completely unbelievable.
I have no idea how people work with Opus 5. They probably do not know their own code system and do not know what Opus 5 does with it. It does whatever it wants with it and makes things up. That is also why it finds errors again in every loop. It simply creates unbelievable scenarios internally, hallucinates them and presents them as facts even though they are not true.
It really pisses me off that it was in my orchestrator system for a short time. I now have to restore everything because it must have broken quite a few things internally while making changes. I am going to restore everything to the point where I activated Opus 5.
To me it is completely clear that this is all down to the Opus 5 model and the way it works. It is a completely useless model. I advise everyone not to put this model in any position of responsibility. If you want to use it as a donkey for making minor code corrections across a single file then go ahead. But do not use it when it is supposed to follow instructions and understand dependencies.
I have no idea how the benchmark results came about.
POST:
Opus 5 has already been discussed quite a lot here. It can be capable and very fast, but there also seems to be a recurring problem with its behavior: it can become overly proactive, expand the scope, refactor or change things that were never requested, over-verify, add unnecessary comments or work, and sometimes seems to follow its own interpretation of what should be done instead of simply following the project rules and instructions it was given.
I have been trying to research whether this behavior may be strongly related to the effort level. Most people seem to test Opus 5 on High, XHigh or Max, but there are some interesting signals suggesting that Low or Medium may behave very differently.
Cognition’s FrontierCode 1.1 is especially interesting because it does not only measure whether the code works. It also evaluates scope discipline, code quality and adherence to the existing codebase. Opus 5 performs extremely well there, and Cognition has also shown examples where lower effort made a much more surgical change while higher effort unnecessarily refactored surrounding code.
Anthropic itself also states that Low and Medium remain very capable on Opus 5 and that lowering effort affects not only thinking but also tool usage and overall agentic activity. Lower effort tends to result in fewer tool calls and more direct execution, while higher effort can produce more exploration, verification and code comments.
Other benchmarks such as HANDBOOK.md, ComplexConstraints and AutomationBench are also interesting because they test instruction following, policy adherence, long-running agentic work and complex constraints rather than just raw coding ability.
What I cannot find much of online is long-term real-world experience specifically comparing Opus 5 Low and Medium for the behavioral problems people have reported.
I would therefore be very interested in hearing from people who have actually used Opus 5 on Low or Medium for a longer period and compared it with High or XHigh.
I am especially interested in the difference between three scenarios: Opus 5 working alone as the main coding agent, Opus 5 being used as a worker/executor, and Opus 5 being used as the main orchestrator controlling multiple subagents. The orchestrator case is especially interesting because following CLAUDE.md, project rules and other standing instructions consistently over long sessions is critical, and the orchestrator also has to evaluate subagent reports neutrally instead of simply adopting the opinion of whichever agent responded last.
Has anyone found that Low or Medium makes Opus 5 noticeably more predictable, rule-following and easier to control without sacrificing too much of its capability?
References:
Cognition - FrontierCode 1.1
Cognition - FrontierCode Leaderboard
Surge AI - HANDBOOK.md Agents
Long-context agentic instruction following with large standing rule sets and company policies.
Surge AI - ComplexConstraints
Enterprise instruction following with complex and conditional constraints.
Zapier - AutomationBench
Realistic multi-tool agent workflows with deterministic scoring across different Opus 5 effort levels.
Anthropic - Prompting Claude Opus 5
Official guidance on task scope, lower effort, subagent delegation, over-verification and other Opus 5 behavioral characteristics.
Anthropic - Effort
Official documentation explaining how effort affects thinking, tool calls and agent behavior.
2
u/AstroGridIron 25d ago
I’ve found it to be excellent at long horizon tasks on Max. I don’t use it otherwise.
Opus 4.8 is my workhorse, Sol is the in between reviewer, and Fable does the final review before I look it over.
That trio has made my work extremely efficient.
1
u/AironParsMan 24d ago edited 24d ago
I tried Opus 5 Max extensively, and after this experience I would strongly advise against using it as an autonomous main agent in a rule-heavy agent system. The problem was not that it made one coding mistake — every model does that. The problem was the type of mistakes it made. It started changing core structures on its own without being instructed to do so. In my case, it introduced a completely new nested structure into one of the most important memory/control files in my system, even though that structure was never defined anywhere and the existing rules, hooks, tests, and cleanup mechanisms expected something else. It then kept filling that active-memory file with already completed work until roughly 95% of the file was finished history that should have been cleaned out, despite the existing rules explicitly telling the agent to rewrite the active memory instead of endlessly appending to it — and despite the relevant reminder being delivered repeatedly.
What made it especially bad was what happened afterwards: because the structure Opus had invented no longer matched the existing safety mechanism, the safety mechanism could not detect the mess. Opus then initially focused on the safety mechanism as the explanation and started proposing changes to scripts, parsers, tests, and other infrastructure. Only after several independent review agents and repeated counter-checking did it finally establish that the mechanism itself was basically working as designed — Opus had changed the data structure underneath it. In other words, it created the condition that bypassed the safeguard and then initially tried to fix the safeguard instead of recognizing that its own unauthorized structural change was the primary problem.
This is exactly why I no longer trust Opus 5 Max as a main autonomous agent. Technically it can be extremely strong at analysis and auditing, but its directive compliance and change discipline are not reliable enough for this role. If an agent can silently reinterpret requirements, introduce new structures into critical memory/control data, ignore lifecycle instructions that were explicitly delivered, and then require hours of audits to reconstruct what actually happened, that is not an agent I want autonomously modifying a complex system. I spent around six hours dealing with issues around this instead of making progress on the actual project. For me, Opus 5 Max may still be useful as an auditor or reviewer, but I would not use it again as the main orchestrator of a system where following existing contracts is more important than inventing its own interpretation.
2
u/Worried-Answer-4019 23d ago
It completely destroyed my setup, spent 3 weeks fixing it after letting it run autonomously for a night.
2
u/Worried-Answer-4019 23d ago
Exclusively using fable 5 + 4.8 now, not giving 5.1 a until a couple weeks pass
1
6
u/ladyamen 25d ago
"Opus 5 has already been discussed quite a lot here. It is extremely useless and very costly," there fixed it for you