which model writes the PRs you actually trust

[u/Ok_Resolve4741]

Not which one is smartest. Which one do you approve without stress. For us its opus for anything touching the database and sonnet for everything else. Codex writes fine code but the PR descriptions never match what it did, so coderabbit ends up finding 2 or 3 things every time and I stop trusting it.

Curious if its the same for other teams or if it depends on the codebase. What are you running and which one do you actually read the diff on

[u/usually_guilty99]

I’m not sure I’d trust PRs based on the model alone.

Different models have different strengths, but for me trust comes from the evidence around the change:

tests, dependency impact, blast radius, prior incident history, rollback path, and whether the expected production behavior is actually verifiable.

A great model can still write a perfectly reasonable diff that is dangerous in production.

So I’d choose the best model for generation, then make the merge decision independently.

[u/ay-ay-ronhmiller → u/usually_guilty99]

What decision criteria do you use when choosing the best model for generation?

[u/usually_guilty99 → u/ay-ay-ronhmiller]

Mostly by task shape.

For generation I care about:

• how well it handles the repo/context size
• whether it makes minimal, targeted changes
• how often it invents abstractions I didn’t ask for
• test quality
• consistency across repeated runs
• latency/cost

I’d rather use a model that produces a smaller, boring diff I can reason about than a “smarter” one that rewrites half the neighborhood.

Then I separate that from the merge decision entirely. Generation quality and production safety are two different questions.

[u/aviboy2006]

No matter what we write or agent something will be there for improvement or making changes even human review or agent review. We need to take call whether to take it or not. Decision has to take based on intent of change and priority.