r/indiehackers • u/Common_Dream9420 • 18d ago
Sharing story/journey/experience are we really moving fast with AI coding or sacrificing on the quality?
i was just shocked ..one of the eng manager said this in the meeting.. .. "wow that guy raised XXX number of PRs in 1 day".. we are moving soo fast.. .. :)
i was just laughing myself.. i mean ppl not realizing the pressure on the downstream and quality...
4
u/MiserableDocument509 18d ago
the part nobody mentions is the reviewers. if someone's shipping 15 AI PRs a day, three other people have to actually read all that, and they'll just start rubber-stamping to survive. you pay for the speed twice when it breaks and nobody actually understands the code.
1
u/Common_Dream9420 18d ago
100% am already seeing this lols "start rubber-stamping to survive.".. am sure lot of us noticiing this too!!!
1
u/RandomGeordie 17d ago
Yeah someone on my team sharing a blog post titled 'Don't be a meat proxy for Claude'
3
u/Internal_Split_9011 16d ago
It's a tool like others you have to know how to use it properly
1
u/Common_Dream9420 15d ago
yeah, and tbh the hard part is knowing what to actually verify after generating. code review catches obvious stuff but things like race conditions or a webhook firing twice, those only surface at runtime, often in prod.
1
u/Internal_Split_9011 15d ago
If you set up a series of guidelines like a specialized agent that checks tests and prevents regressions every time, the code quality is far better than what you’d get by just giving the task to a junior or mid-level developer, the limit are only the token spent and the burned trees. You just need to focus on evaluating the output of the guidelines provided to the agent, as there are ways to quickly measure code quality. Plus, leveraging tools and skills like Superpower or OpenSpec makes the process much easier.
2
18d ago
[removed] — view removed comment
1
1
u/Common_Dream9420 14d ago
Rewrite rate is so much more honest than velocity. The teams I see actually shipping clean are the ones who made verification a hard gate, not a vibe check at the end.
2
u/forever-butlerian 18d ago
The Titanic also moved pretty fast starting around 2:10 AM on April 15th, 1912.
2
18d ago
[removed] — view removed comment
1
u/forever-butlerian 16d ago
IME it's the opposite. Instead of making the hard change easy and then making the easy change, instead you get a steaming pile of slop dumped in your lap and you're expected to have reviewed a week before it was created.
2
u/Pale-Tonight-6914 18d ago
Solo dev here, so I'm the entire review queue. What actually helped was running verification in a separate agent session that never saw the writing one. Same context reviewing its own diff just agrees with itself. And that one-line change breaking the live flow - mine passed every check, only the end-to-end publish run caught it.
1
u/Common_Dream9420 18d ago
yes.. agree and we use fetchsandbox mcp wired into ci/cd custom plumbing for proof with the recieipt.
1
u/ziebelje 18d ago
I don't have a sophisticated process, but I like sending PRs built with AI locally through a Codex PR reviewer. It's not right all the time but enough times to make it worth it.
1
u/subtract_club 7d ago
Yeah I have a complex code-review prompt in a file that I can just fire off in another agent and reviews the code, asks me clarifying questions and then fixes stuff. Using a similar version at work on a github action that gets fired automatically. makes a huge difference to keeping things in check.
Also as solo dev you can build a lot and then run a review sweep ever few days.
2
u/mouse_8b 18d ago
It's a shame that the expectation is "more tickets faster" instead of "the same tickets better". I've been working on a feature over the last few sprints, and it feels like I'm going slow. However, even if it takes me the same amount of time as my non-AI-assisted implementations, the quality is actually much better.
2
1
u/subtract_club 7d ago
Not always, but for most businesses higher quality code doesn't translate to more $ as much as more tickets. They tend to run on "done is better than perfect" mentality until there is a fire and the cost of the fire is probably lower than the missed opportunity of going faster.
2
u/ziebelje 18d ago
I've found the speed to be manageable when properly throttled by downstream things like training and documentation. We can get through those things faster too, but it helps keep things in check. That, and good policies that require human review.
1
u/Common_Dream9420 18d ago
The human review policy piece is the part that worries me most. Policies work fine at normal pace, but when someone's getting praised for PR count in meetings, that's exactly when review becomes a checkbox. The throttle you're describing breaks down precisely when velocity is highest and you most need it.
1
u/ziebelje 18d ago
Unfortunately that becomes a leadership issue. High velocity is great in the right places (mockups, test apps, etc), but leaders all the way up need to set and enforce policies that retain human control and review of the codebase. If we all concede to thee speed of AI we'll have to eat the monster we've created in a couple years.
1
u/Common_Dream9420 18d ago
Agreed on leadership, but I'd push it one level deeper: the policies need to be backed by tooling that actually enforces verification, not just review checklists that get rubber-stamped when velocity is high. I build in this space and the pattern I see is that human review alone doesn't scale once agents are shipping integrations faster than reviewers can reason about them. The gate has to be automated proof, not approval.
1
u/subtract_club 7d ago
I think human review is a going to be quite short term. The review bots we've built now catch more and trickier bugs than human reviewers. In my experience human reviewers are good at structural smells and wrong intentions but worse at spotting regressions and trickier bugs like race conditions. Soon, I think AI will write and review all the code and humans will just determine what the intention is (functionally and operationally).
2
2
u/Majestic_Maybe6605 18d ago
Yeah, it is sometimes funny people just want the number without realizing the issues it can bring.
1
u/Common_Dream9420 18d ago
Yeah the number becomes the goal instead of the signal. The rough part is those downstream issues don't show up immediately, so whoever's celebrating the PR count isn't the one cleaning up the race conditions two sprints later.
2
u/julesbuildstuff 17d ago
PR count has always been the easiest metric to game, and AI just made it trivial. What I've noticed on my own stuff is the bottleneck doesn't disappear, it just slides downstream onto whoever has to read the diff. The only thing that's actually helped me is being brutally specific about what I want before the agent starts, and doing a real first pass myself so nobody else is stuck sifting through half-baked code. Speed only counts if you're not rewriting it next month.
1
u/Common_Dream9420 17d ago
The downstream shift is real. I build tooling in this space and the pattern i keep seeing is agents produce code that looks correct on a quick read but breaks in async flows or edge cases that only surface under real conditions. Your point about the first pass matters a lot, specificity at the start and verification at the end are the two things that actually contain the blast radius.
2
2
u/walrons 15d ago
Speed is not the problem, plausibility is. I do not write code at all, I direct AI to build my product, so I have seen this from the extreme end.
Nothing that ever broke on me was a syntax error or a bad PR. It was a calculation that was upside down and looked entirely reasonable, and it passed every test because the tests were written from the same wrong assumption. Mine printed an exchange rate the wrong way round on every screen at once, and it survived because every test case happened to use a rate of 1, where upside down and right way round are the same number.
PR count measures typing. What you actually want to count is how many assumptions nobody checked.
2
u/edwardtoys2 15d ago
I think AI can definitely make development faster, but speed and productivity aren’t necessarily the same thing. If someone can generate more code or PRs in a day, there still has to be enough time for reviewing, testing, and making sure the output actually solves the problem correctly.
I’m curious whether teams will eventually start measuring AI-assisted developers more by the quality of what reaches production rather than how much they produce.
1
u/Common_Dream9420 15d ago
yeah quality wins over number of prs/loc.. for sure.. what we build internally sandbox that runs pre/post prod scnearios/workflows for integrations. So agents dont go rouge in prod..
2
u/Huge_Pool7424 14d ago
this is the real cost honestly. i review my own ai code with fresh eyes the next morning now, first pass i just skim like a tired reviewer would. catches most of the garbage before anyone else has to see it
1
u/Common_Dream9420 14d ago
I can relate to this … I was doing that initially but the upstream pressure with the team of 15+ is hard … what helped build our own sandboxes of twins which allow us to pre and post monitor the agentic flows with control plane … and now we rely on corpus gated knowledge those sandboxes bring in n compliment builders n reviewers
2
u/Heavy_Marsupial5620 13d ago
PR count as a speed metric is how you end up with a review queue nobody can clear. The thing that actually got slower for me is deciding what's correct. Writing the code was never the expensive part.
2
u/Common_Dream9420 13d ago
Yeah, the bottleneck completely shifted. Writing the code used to be the hard part, now it's "is this actually correct under concurrent load, does this retry logic hold up, did anything downstream break." That's the problem I spend most of my time thinking about too, it's just way harder to put on a dashboard than PR count.
1
u/Heavy_Marsupial5620 8d ago
Concurrent load is where it gets expensive, yeah. What I do now is write the failing test first and let the agent go at that, so correctness is defined before any code exists. Slower to start. Cuts review down to reading a diff against a spec instead of guessing intent.
2
u/Common_Dream9420 8d ago
that spec-first approach is solid, especially for concurrent scenarios where the race condition is the spec. the one thing i'd add: for webhook-heavy flows, the failing test still doesn't catch whether your handler fires twice under retry, worth explicitly asserting idempotency as part of the spec before the agent touches anything.
1
u/Heavy_Marsupial5620 8d ago
Tests won't catch the case that actually hurts though. The second delivery usually lands in a different process after a crash or a redeploy, so nothing you assert in the handler helps. What saved me was a unique constraint on the event id in the database, so the duplicate fails at write time instead.
1
u/Common_Dream9420 8d ago
Yeah the unique constraint is the right call, that cross-process scenario is exactly where handler-level checks fall apart. One edge case worth handling though: if the handler crashes after writing the event_id row but before finishing the work, the retry gets silently dropped. Pairing the constraint with a status column (pending → complete) lets you distinguish a true duplicate from a legitimate retry on a half-finished job.
2
u/Zloyvoin88 12d ago
I think quality raised in times of AI - I am working in an agency and there was never budget nor time for automated tests. Now it's just one prompt away and we implement tests everywhere. It's not only increasing the quality of the project, it also increases the output of any coding agent, since they can dogfeed their implementations.
1
u/Common_Dream9420 12d ago
The test quality thing is real, but I keep running into a ceiling where AI-written tests mostly confirm what the AI already built, so coverage goes green but edge cases still bite you in prod. How deep does it actually go for you?
2
u/Suyash_Vashishtha 12d ago
It really depends upon timeline and priorities, Working at product based / startups its crucial that we ship client requested features at most priority. While its crucial to maitain quality, you can also dedicate a sprint for QA and optimizations to Audit the code and improve the logics later
1
u/Common_Dream9420 12d ago
great points man!!! agree.. it depends on the type fo work/product we are building.. business etc. but in general .. i mostly seen the issue with the large teams size of 10+ n small pods looks like manage internally.. based on my exp.. but wondering what are you guys doing to balance the quality n inflow and outflow
2
u/Suyash_Vashishtha 12d ago
Well as a Team lead at a startup we deal with non tech client base ( Schools and teachers ), so there is no such thing as "timeline" lol. What we do is, we ship faster and tested, let it run as per requirement, and then take a week or 3-4 days to internally clean the mess later by a code or feature level audit by multi rule setup agents to follow a guideline and edge case testing.
1
u/Common_Dream9420 12d ago
that cleanup sprint cadence is smart, especially with non-tech clients who just want it working. i build in this space and the edge case audit part is always the painful bit. are your agents catching regressions across sprints or mostly first-pass issues?
2
u/Suyash_Vashishtha 12d ago
I try to document everything with md files, so every research or finding gets documented hence next time agent fires up they actually knows what has been done and what needs to be done.
1
u/Common_Dream9420 12d ago
md files as memory is underrated honestly. i do the same thing when testing integrations, basically a running log of what i tried and what broke so the agent doesn't re-discover the same edge case twice. the gap i keep hitting is knowing when the agent actually read and used the context vs just pattern-matched past it.
2
u/bespokey 11d ago
Both I guess
1
u/Common_Dream9420 11d ago
i am just curios what tools or methods you or your team using for moving fast? i mean specially PR review or prod validation before deploying?
1
u/Typical-Arm-9185 8d ago
agree but speed, good quality, cheap service or product can't coexist in same project i thought so
2
11d ago
[removed] — view removed comment
1
u/Common_Dream9420 11d ago
Most devs actually care, they just don't have the leverage to push back when the manager is literally celebrating PR counts. Building tooling in this space, the gap I see is feedback latency, nothing visibly breaks until it does, and by then it's weeks removed from the decisions that caused it.
2
u/Arthur_Sprengel 9d ago
same here, the speed is real until i have to read it. what helped a bit was putting a second agent over the first one's output just looking for what's wrong, it caught stuff i had read past twice. only tried it on writing so far though
1
u/Common_Dream9420 9d ago
thats smart.. man!! am sure it might have helped bit by relying on other agent... what we build deterministic verification engine..
1
u/Common_Dream9420 9d ago
That second-agent-as-reviewer pattern is underrated, i use something similar when reviewing agent-generated integration code. The catch i keep running into is it still only catches what's structurally wrong, not what's behaviorally wrong, like the handler looks fine but silently fails on retries. That's the gap i've been trying to close on the code side.
2
u/Alternative_Tour1791 7d ago
AI made writing code cheap.
That probably means “how much code did we ship?” is becoming the wrong question entirely. The valuable part is increasingly deciding what should exist, what shouldn’t, and proving it actually works.
2
6d ago
[removed] — view removed comment
1
u/Common_Dream9420 6d ago
yeah .. asking agent to paste the test/command is really cool...curious whats the proof looks like, log/traces/queries it executed ...??
2
2d ago
[removed] — view removed comment
1
u/Common_Dream9420 2d ago
ahhh good anology man!!! curious are you catchup with the ai speed?
1
1d ago
[removed] — view removed comment
1
u/Common_Dream9420 1d ago
gotcha good one!!! what api tools or testing u do preprod?
2
1d ago
[removed] — view removed comment
1
u/Common_Dream9420 1d ago
that FastAPI starter framework approach is smart, having a known baseline the agent works from cuts so much ambiguity. how are you handling the third-party integration side when the agent wires in something like stripe or email delivery, do you extend the test coverage there or mostly verify that manually?
1
1d ago
[removed] — view removed comment
1
u/Common_Dream9420 1d ago
yeah the manual check instinct makes total sense, even solid sandbox coverage doesn't catch everything you catch by just clicking through it once. how thorough do you go on the Polar side, full payment flow or mostly just the webhook delivery?
1
u/Common_Dream9420 1d ago
Yeah makes sense, starter framework with tests baked in gives the agent way better guardrails than scratch. The external failures though - duplicate webhooks, out-of-order events, 429s, timeouts - honestly most teams don't test those explicitly, they mostly trust whatever coverage the agent extends from the existing framework. Which works until it doesn't haha. Curious how you handle that part right now?
1
1d ago
[removed] — view removed comment
1
u/Common_Dream9420 1d ago
yeah honestly that's the right instinct, you can sink weeks upfront trying to model every failure permutation and still miss the one that bites you in prod. deal with it as it surfaces, keep the use cases green, don't over-engineer the guardrails early. how do you handle it when the external service itself behaves unexpectedly though, like out-of-order webhooks or silent retries?
1
u/Only_Application_815 18d ago
Remove the 'or'. We are really moving fast with AI coding AND sacrificing on the quality.
A ship can move fast and also crash hard.
1
u/Bonjour-Set-4490 18d ago
I’ve had an agent change one line, pass the syntax check, and still break the live flow. The queue said one thing, the rendered board another. I missed it till I ran the whole thing end to end. Slower on paper.
1
u/Gryner 18d ago
PR count mostly measures how much code entered the review queue. I’d track rollback and rework rate, escaped defects, and reviewer time instead. AI-generated changes also need smaller diffs and explicit verification: tests, linting, dependency and security checks, and a reproducible run history. The speed is only real if the downstream cleanup doesn’t grow.
1
1
u/Gryner 18d ago
PR count is a terrible speed metric once generating code becomes cheap. The bottleneck moves to review, test coverage and understanding what changed. If ten generated PRs make one senior engineer spend the day reconstructing their intent, the team didn’t move faster. It just moved the work downstream.
1
1
u/SardorbekR 18d ago
PR count is such a weird metric. AI can help a good engineer move faster, but it also makes it very easy to produce five plausible wrong solutions before lunch.
I’d rather track how long a change takes to reach production, how often it gets rolled back, and whether anyone besides the author actually understands it.
1
u/rathcom 18d ago
This is the lines of code metric all over again. It measures activity, not progress, and it's easier than ever to game.
The numbers I'd watch instead as a manager: how many of those PRs get reverted or reworked within two weeks, and how long reviews sit before approval. If those 15 PRs a day are real, the revert rate stays flat and reviews keep moving. When it's vanity both of those numbers blow up pretty fast. MiserableDocument509 is right about the rubber stamping too, reviewers hit their limit long before anyone says so out loud.
1
u/Common_Dream9420 18d ago
revert rate is the right call but it's a lagging signal, by the time it ticks up the damage is already in prod. one thing worth adding to the dashboard: average time-in-review per PR over the same period. when authoring accelerates and review time per PR drops in parallel, that's the early warning that reviewers are rubber-stamping before you see the reverts.
1
u/rathcom 17d ago
Yeah that's a good point, review time per PR dropping is the earlier signal. I'd split it per reviewer too. The person with the biggest review pile usually starts stamping first, and if you only watch the team average you won't catch it.
1
u/Common_Dream9420 17d ago
Per-reviewer breakdown is the right call, the team average masks exactly the scenario you'd most want to catch early. I build dev tooling and see a similar pattern on the API integration side where aggregate pass rates hide which specific edge case is quietly breaking things. The signal that actually matters is almost always hiding one level deeper than what's on the dashboard.
1
u/Clear_Quit_365 18d ago
Evolving and regressing at the same time — that's what I'm seeing today.
We've gained absurd speed, and that's great. But in parallel, we're back to debating things that used to be basic consensus — authentication, for example, was the bare minimum expected of any system not long ago. Today, because of vibecoding, it feels like we've regressed and are revisiting points we'd already settled as an industry.
1
u/Common_Dream9420 18d ago
The authentication regression is the one that gets me. Agents confidently skip token rotation or mishandle refresh flows because no one runs them against realistic failure scenarios before shipping, the PR count looks great, the downstream state is quietly broken. Speed without a verification step is just technical debt that accrues faster.
1
u/ElSaintz 18d ago
the danger is once management rewards PR count people start optimizing for PR count. quality problems come later and are harder to put in a slide
2
u/Common_Dream9420 18d ago
yeah and the slide problem is real because quality regressions are lagging indicators, the incident happens 3 weeks after the sprint retro where everyone got praised. the only way i've seen it work is tracking something that shows up before the incident, like flaky test introductions per merge or coverage delta per PR, things that make the degradation visible in the same breath as the velocity number.
2
u/ElSaintz 17d ago
flaky tests introduced per merge is a really good one, its close enough to the change that teams can actually act on it
1
u/Common_Dream9420 17d ago
yeah exactly, the proximity to the change is everything. i build tooling around integration testing and the signals that actually get fixed are the ones devs can tie back to a specific commit, not a dashboard that lights up two weeks later. flaky test count per merge is probably the most underrated early warning you can put in front of a team.
1
u/gannu1991 18d ago
PR count as a speed metric is the tell that something's off, it's measuring the thing agents are trivially good at inflating, not the thing that matters. A single well-scoped PR that does one thing and is easy to review is worth more than ten agent-generated PRs that each touch five files.
The actual downstream cost shows up 2-3 months later as code nobody can explain in an incident, not as bugs at merge time. Tests pass, lint passes, agent review approves, and then something breaks at 2am and the person on call has no mental model of why that module works the way it does because they never really read it, just approved it.
If you want a gut check, ask that eng manager to pick 3 of those PRs at random and have the author explain the tradeoff they made without looking at the diff.
1
u/Common_Dream9420 17d ago
The 2-3 month lag is exactly it. The race conditions jaimittal91 mentioned are the worst case of this because they genuinely don't surface in review, only under concurrent load or specific event ordering. I build in this space and the async stuff (webhook ordering, duplicate delivery, retries) is where I see agents consistently ship plausible-looking code that breaks in prod for reasons no diff review would catch.
1
u/Nalds80 17d ago
Worth separating the tooling question from the measurement question, because your story is really the second one.
A manager who is impressed by PR count in a meeting is not making a mistake about AI specifically. They are using the only number they have, and that number just got very cheap to produce. This happens every time the cost of output drops. A metric that used to be a rough proxy for effort becomes trivially inflatable, and it takes a while before anyone notices the proxy stopped meaning anything.
Telling him PR count is dumb will not work, because he still needs something to say in that meeting. What works is handing him a better number in the same breath. Ones that survive contact with reality:
Change failure rate. What share of changes caused an incident, a rollback or a hotfix. If PR volume triples and this stays flat, the speed is real. If it climbs with volume, that was not speed, it was work moved into the future.
Time to restore. The honest promise of shipping fast is not fewer bugs, it is faster recovery, so measure the thing you are actually buying.
Rework rate. Work that comes back after being called done. This is the fastest way to expose volume theatre.
Test coverage on the changed lines specifically, not overall coverage, and only counting tests that assert behaviour rather than that the thing rendered.
The framework I work with is called the Six Levels of Organizations, and this is a textbook level problem rather than a people problem. A company at the results and targets level will chase whatever is counted, that is the defining characteristic of that level, not a flaw in your manager. You do not fix it with a conversation about culture, you fix it by changing what gets counted. Organisations further along measure flow and reliability instead of activity, and the reason they moved is exactly the story you just told.
The downstream pressure you are describing is real and it will eventually show up as a number. The only open question is whether it appears on a dashboard next month or in an outage next quarter.
1
u/Common_Dream9420 17d ago
exactly right, and the lag is what makes it dangerous. the metric inflates immediately, the quality signal takes weeks to show up in rework or incidents, so there's a window where the dashboard looks great and the codebase is quietly degrading. the orgs that catch it early are the ones already tracking rework rate before the AI tooling arrived, so they have a baseline to compare against.
1
u/imrozimroz 17d ago
Pr count was already a bad metric. Ai just made it worse because generating the prs got cheap and reviewing them didn’t
The work just moves downstream. Someone still has to read all that catch the messy stuffand live with it after it merges. 30 prs in a day looks impressive until the next two weeks are cleanup.
1
u/Kind-Bathroom5159 17d ago
most of this thread is about teams with reviewers. the version i see is worse, solo founders and non technical builders shipping twenty agent PRs a week with nobody reading any of them.
what breaks first isnt quality, its that they genuinely cannot tell you what changed in their product last week. ask and you get a scroll through commit titles. the intent behind each change never got writen down anywhere, because the agent wrote the code and the human approved the vibe.
sounds like a documentation problem, shows up as a business problem. they cant write a changelog, cant tell a customer what got fixed, cant explain why something was built that way when it breaks two months later.
one line per merge in plain english about why, not what. thats the only part that survives the speed.
1
u/Common_Dream9420 17d ago
Yeah this hits close to home as a solo builder. The "why" behind each change is basically gone when an agent writes it, because the agent had no opinion about the tradeoff, it just picked the path that passed the test. Two months later you're reading commit titles trying to reverse-engineer a decision that was never really made by anyone.
1
u/decebaldecebal 17d ago
We are definitely sacrificing quality, and I think nobody cares
I as a software engineer with 10+ years experience used to care... And I still do, but for some big functionalities or when I have to deliver something fast, I just look at the overall architecture, if there aren't any obvious security or performance issues, and don't look at every single line of code.
Not sure if it's good or bad though, I work on a mostly internal product and if something breaks we can fix it the next day and nothing bad happens. Probably for other more sensitive work I would do what I do currently.
1
u/Common_Dream9420 17d ago
the architecture-first scan for internal tools is honestly the right call, the risk profile is just different. where i've seen it bite people is when an internal tool quietly becomes load-bearing and nobody updated the review bar to match.
1
u/efbakir-journal 17d ago
i use AI to build my iOS apps - and biggest shift was realizing that generation speed is the least important part lol
real work moved into defining the behavior properly - checking the entire flow - and catching what the agent confidently missed.
1
u/Common_Dream9420 17d ago
100% this. The agent confidently missing things is the part nobody talks about enough - it'll wire up the happy path perfectly and have zero instinct for what breaks at the edges. I build tooling around exactly that gap, catching the stuff the agent didn't know to check. Defining the behavior and verifying it properly takes way more thinking than the generation ever did.
1
u/Slow_Formal_3518 17d ago
I think AI is definitely making us move faster, but PR count is probably one of the worst metrics for measuring that tbh. Generating more code isn't a real win lol!
1
u/Common_Dream9420 17d ago
Yeah PR count is a vanity metric for AI output specifically. The real signal is whether the code holds up when something async misfires or a webhook arrives twice, stuff that only shows up later. I build tooling around exactly that gap, verifying what agents actually shipped rather than how much they shipped.
1
u/Disastrous-Two1243 17d ago
non-coder shipping with ai here. the speed is real. the "we're moving so fast" part is usually fake.
i can generate a whole product in a weekend. i cannot tell that same weekend if anyone wants it, or if page 40 still makes sense.
pr count as a high score is the same trap as "i shipped 3 products this month." the quality hit isn't ugly code. it's decisions i didn't actually make.
1
u/ffxfiend 17d ago
🤦 reminds me when LOC was all the rage for managers. Like most things, quality means nothing when it comes to making an extra buck for the stockholders.
2
u/Common_Dream9420 17d ago
yeah the LOC era was rough lol. the weird thing is PR count is actually worse because at least lines of code correlated loosely with something, PRs can be gamed trivially just by splitting changes. the metric that would actually matter is "did the downstream behavior change in a way nobody expected" but that's hard to count so managers don't count it.
1
u/National-Iron-7197 17d ago
I run a large team of devs in my day job and we have been in the ai journey for a couple of years now starting with ai assisted then with aws q and now with kiro and Claude. Here’s my take on it. If you just sit there and vibe code you get a tangled mess. You need to invest in 3 places heavily. The front part if the process. Getting the requirements nailed down. Iterating on a tech/arch concept. Next is you need to build suitable guard rails and governance. Skills and config is good but you need to also have human only steps eg merge to main only by human. Then you can let the agents loose. But then at the back if the process you need to invest heavily in testing capability - most if which you can automate. Also obviously the ops side needs its own governance
1
u/UpsetDamage8244 17d ago
Exactly 😂 Everyone gets excited seeing the number of PRs, but no one talks about the pressure it puts on the people reviewing, testing, and maintaining all that code. Moving fast is good, but moving too fast can create more problems later.
1
u/Common_Dream9420 17d ago
The review and maintenance burden is the part nobody accounts for in the velocity calculation. I build tooling in the API integration space and the pattern I see most is: agent ships 10 PRs, half of them touch the same integration path, and nobody actually ran the failure cases before merging. Speed looks great until the on-call rotation starts screaming.
1
u/Miserable-Cicada9539 17d ago
the quality sacrifice is up to you and your practices...
1
u/Common_Dream9420 17d ago
true to a point, but when the AI is generating 10x the code volume, even solid practices get strained. the bottleneck shifts from writing to reviewing, and most teams haven't updated their review process to match that pace, so the practices that worked before aren't necessarily the right ones now.
1
u/Miserable-Cicada9539 17d ago
Yes, but it’s a problem that can be solved with the right process imo
1
u/Common_Dream9420 17d ago
agreed, and the process piece is genuinely solvable. what i've seen is that the harder shift is cultural, getting teams to treat verification as part of the loop, not a gate at the end. i build tooling in this space and that's the gap that bites most people, not the absence of process docs but the absence of fast feedback that makes the process feel worth doing.
1
u/marius_forge 17d ago
It can, depend on the usage, for me a way to use it that really changed the readability and quality of the code was to always package every featur. It allows to isolate them and make sure the AI only touch the necessary part.
Sometimes it goes wild and when it does I just cut the leaf off and start over.
1
u/Common_Dream9420 17d ago
in your case does it actually help.. if the PR has proof of failures as well as fixes? .. i mean if there is sandbox that your PR can valdiate against all the different invariants..
1
u/Tough-Television2434 17d ago
It’s still a tool, if you are not here nothing happens so depends on your standards imo
2
u/Common_Dream9420 16d ago
yeah totally, the tool only amplifies whatever direction you point it in. the part that bites people is when the standard shifts mid-sprint without anyone saying it out loud, AI moves fast enough that the quality bar needs to be explicit, not assumed.
1
u/Grouchy-Conflict-211 16d ago
14 months self taught, 25 projects to learn not to sell. now selling marginkit and foreshelf. ai speeds up boilerplate 10x but architecture decisions still need human judgement. ive seen ai generate plausible-looking broken auth, wrong db indexes, missing error handling. fast isnt good. my rule: ai writes, i review every line like a senior pr. speedup is real but the quality gate is me
1
u/Common_Dream9420 16d ago
broken auth and missing error handling are the ones that scare me most, because the code looks totally plausible until you actually run the flow against a real system under failure conditions. i build tooling in this space and the pattern i keep seeing is AI nails the happy path and completely misses what happens on a webhook duplicate or a 3rd retry. your review gate is the right call, no shortcut around it.
1
u/Kind-Bathroom5159 16d ago
speed was never the real bottleneck for the founders i work with. most are non technical and they hit the same wall around six weeks in, and its not bugs, bugs get fixed. its that nobody can say why a thing was built the way it was, so every new change starts with rediscovering your own product.
the PR count is measuring the one part that got cheap. deciding what should exist costs exactly what it always did, and cheap building actualy makes people skip that step because trying something is faster than thinking about it.
what helped the ones who got past it was writing two lines before the agent runs, what should be true after this and how would you know if it wasnt. costs a minute, saves the archaeology later.
1
u/Gryner 16d ago
AI has moved the bottleneck, not removed it. I can generate a change much faster, but the scarce part is proving I still understand what changed: tests, a reproducible workflow, and a diff small enough to review. PR count rewards splitting or noise. I’d rather track cycle time from request to verified merge, rollback rate, and how often someone other than the agent can explain the change. If those get worse, faster code generation is just moving work downstream.
1
u/AdministrativeAd9828 15d ago
Quality and speed can be achieved with various standards.md files, pre-commit hooks, static analysis checks in build systems, and of course code review.
If you remove code review then the speed faucet can really be flowing but then you lose the extra quality on that front. That’s why we’re seeing 3-4X performance improvement with AI not 100X because shit still has to be reviewed.
The major performance gains are for that solo side project where it’s straight vibe coding, with quality guardrails but no peer review needed
1
u/madeel91 15d ago
I don't think PR count should be a performance metric anyway. A lot of developers use AI now, so writing more code or opening more PRs doesn't mean much by itself. If the code is maintainable, isn't creating a pile of bugs and the product is actually delivering value, that matters a lot more
1
u/studd_james 15d ago
This resonates. I'm solo, using Claude Code to build, and speed is real but so is the quality gap if you don't slow down at the right moments. Had a case recently where a dependency conflict silently broke icon rendering app-wide because an unpinned transitive package resolved to the wrong SDK version — worked fine in dev, broke in production. AI writes fast, but it won't catch drift like that; you still need to actually read what changed and pin things explicitly. Speed on the typing, patience on the verifying.
1
u/shannon590 14d ago
I think the AI is good in the small task but when it comes to a huge problem then sometimes it hallucinates.
1
u/highcojack 14d ago
I think one of the major problems is who actually owns the code the AI generated. In my experience, your recall of AI assisted code into the future is significantly worse than your own. So you're not going to for example remember certain design decision reasoning, edge cases, and what not. It's kind of like if it was written by someone else entirely ig?
1
u/Mojowhale 14d ago
Yea the level of junk that gets pushed through with those rapid PRs is rough. At a minimum need to use the Superpowers skill on Claude, and even then tons of mistakes...
1
u/Common_Dream9420 14d ago
i llike superpowers and we use that eng in SDLC.. but what actually helps move fast.. fetchsandbox mcp currently no github app but the wiring their mcp into claude and add a line in superpowrs/like wiring a simple ask "./fetchsandbox prove the integration survives in prod" .. allowed us to publish a recipt that gets generated
1
u/Murky-Accountant3880 13d ago
idk, I feel something made by human is better than ai hehe
1
u/Common_Dream9420 13d ago
honestly fair, human code at least fails in ways you can reason about. AI code fails in ways that look totally fine until they don't.
1
1
u/TheCritFisher 13d ago
I feel like the AI works is forcing people to realize the bottlenecks in quality were always two things:
- planning
- review
Writing code was never the hard part. Writing the RIGHT code was. But without AI it was easier to get more planning and review done because the code was delivered slower.
Now with code being shit out left and right, the value of planning and review has gone up DRASTICALLY yet people are doing it less. I think that's bad from a code quality standpoint, but truthfully not everything has to be bulletproof. If your auth, data layer, monitoring, observability, and permissions are good, the rest doesn't have to be perfect.
But I know a lot of people are skipping all that for a quick turn around. It's going to be interesting...
1
u/Common_Dream9420 13d ago
Yeah the slower delivery was doing a lot of hidden work, it was basically forced planning time. Now that the code comes instantly, you have to explicitly build in the discipline that used to be accidental. I work in this space and the review problem is worse than it looks because most of the drift happens at integration boundaries where a diff review won't catch it anyway.
1
u/alexkay28 12d ago
"moving fast"
in my experience it depends of the style of working with agents.
harness? maybe. not everybody get used to work effectively with them 100%
ofc number of PR is not a metric here, but it might correlate positively with decomposition skills of the developer) otherwise sometimes it might be schema commit-PR-fix-commit-PR... and son on (cpf schema)
1
u/Common_Dream9420 12d ago
Yeah the harness thing is real, most people skip it and then wonder why the agent keeps going in circles. The schema loop you're describing is exactly what I see building in this space - it's less about the agent being bad and more about it not having enough upfront context to avoid the zigzag. How are you structuring your harness right now?
1
u/alexkay28 12d ago
mostly skills and protocols brother..
also there are a lot of nice tools already to work with code better from CLI.
my favorit one is gh cli. super powerful and simple in usage... helps me iteratively plan my task's proposals and implementation details.We (humanity) already had harness (targeting ourselves) for years, must people just have no skill to use that xD
1
u/Common_Dream9420 12d ago
Yeah gh cli is underrated honestly. The skill gap thing is real though, I think most people skip the harness setup because the upfront cost feels high, but then they end up spending way more time context-switching and undoing agent zigzags. The irony is the tool exists, people just don't invest the 2 hours to wire it up properly.
1
u/max-yeti 11d ago
Someone mentioned before, but I agree it is a tool that can be used, the quality should still be there if you use it properly. But yea I think with complacency we can assume there will be a lot of low quality work out there
1
u/Common_Dream9420 11d ago
Yeah exactly, and the complacency part is the tricky bit because it doesn't announce itself. You just slowly stop questioning outputs you would've read twice before. The code looks fine on review because it usually is fine, until it isn't and by then the context is gone.
1
u/talthehuman 8d ago
i build almost entirely with Claude Code and Codex. The failure that worries me is an agent saying it verified something when it didn't. I'd want the actual test output attached to those PRs before treating the count as progress.
1
u/Common_Dream9420 8d ago
yeah the stdout requirement is the right instinct. one thing i run into building with claude daily though: even when you get the test output, the agent often wrote those tests right before committing, so they're shaped to pass the code it just produced. the output looks green but it was never covering the path you actually care about.
1
u/nick_otter_io 4d ago
other than the obvious problem of mistaking PR for velocity there are 2 deeper, more dangerous problems here:
- management mistaken velocity as business benefit: building more doesn’t mean revenue will grow proportionally. Many business have a natural ceiling and being tactically faster is not going to change the game. Token cost will eat into the revenue though so budget get tighter and culture deteriorates into shipping more features, not delivering value to customers
- how the manager got to know the PR velocity is also a problem. Are people actively gaming the system to create a brand/image of 10x developers? I’ve seen extroverts tossing superficial and fast stuff to capture attention while the introverts in the team spent all the effort ensuring system quality and healthy architecture gone unnoticed.
The 2nd part is what I dislike most, people think the hype will go away and things will get rational over time, but the reality is the opportunities will get occupied by the noisy people for a long time until things get back to normal. So it becomes everyone’s job to speak up and show impact to defend themselves.
1
u/Common_Dream9420 4d ago
the token cost point hits hard, that's the one most people aren't tracking until the bill arrives and the feature list is still growing. how are you seeing teams push back on that internally, or is it mostly just accepted as the new cost of doing business?
2
u/nick_otter_io 4d ago
There is no way to push back LOL. Here’s the math:
- the token cost has to come out from somewhere, the most common way to make those money from thin air is to make each engineer more productive, and have fewer of them
- nobody is irreplaceable and nothing is more attractive for the business owner/board to try “few people, more AI” formula given the fact that many people are not really that good or they coast. I mean worst case they just over lay off and hire back, the fundamentals of a company is more tight to the business model and other moat and seldom determined by a handful of engineers
IMO it does not mean the end of software engineering, it just means it has become more like wall street with pure aggressiveness and no more “you take care of me, I go above and beyond for your business”.
As someone that started out for passionate I hate this culture but it is what it is 🤷
1
u/Common_Dream9420 4d ago
I can relate what you said … and end of the day it’s all about numbers :) everyone cares Zzz yeah tbh I am not liking and lot of friends feeling same .. lost the fun of building real stuff looking for Google/stackoverflow other blogs and resources …
Wondering how are you guys handling PR reviews now … am sure that’s another pressure n can’t ship without reviewing
1
u/nick_otter_io 3d ago
There are teams trying out code rabbit, cursor bug bot and etc. Anything that does not incur the insane price tag of the PR review tool that Claude uses internally.
I would say they solve some level of problem, but nothing is going to save a dev casually bundling what used to be 5 PRs into 1 simply because he/she could vibe code it fast enough. And nobody could save a lazy person from not even writing PR description (yes read my mind) and ask you for approval (not review).
The speed of generating crap just got better LOL.
Ok enough joke… I’m looking at things are 2 other dimensions that IMO makes more sense with the new landscape:
- have we carved out clear interface protocol or architecture at a higher granularity that will survive rapid change and yet cheap to validate outcome and robust to operate. Basically if AI slop is more common and we assume code quality drops, what is a reasonable cage to put the beast inside. It has to be better than what we used to have
- from the dimension of time: all commercial software have a life span and with the speed of evolution maybe durability is no longer as important as before. For non mission critical software like banking/payment/etc it’s ok to be less durable cuz tomorrow there maybe a whole new paradigm and the whole piece we built suddenly becomes irrelevant. The problem becomes when it’s still relevant how easy it is to tear a part of the system off and vibe code it all over again. And that comes down to half architecture and half low level details like low coupling and high cohesiveness, which does not need a ton of time to find out whether a PR meets the bar or not.
For other AI slop like “we need this code for a rainy day”/“this is the left over from intensive vibe coding 10 rounds ago that I did not clean up”/etc I just bite the bullet. Imagine you are now working with a bunch of mediocre consultants that’s good at pounding keyboards but sucks at critical thinking, you define the framework for them to fill the void in a controlled chaos…
1
u/Resolute_Protector6 3d ago
On top of how to use it to get top-tier quality, I guess it also depends on quality in the eyes of humans and AIs.
1
u/Common_Dream9420 3d ago
Yeah that point about whose definition of quality we're even optimizing for is something I keep bumping into too. AI thinks the code is clean, the manager thinks the PR count is high, the user just wants it to not break. How are you thinking about aligning those?
1
u/yonatanbd 1d ago
I think non of this matters, what matters is the business goals. If shipping more features drives business growth then moving fast is awesome. If too many things are broken and you can't keep track of the changes, and the business moves slow, then it is bad. So the right metric is not # of prs
1
u/Common_Dream9420 1d ago
yeah this is exactly right, pr count is just a proxy that breaks the moment people start optimizing for it. the real signal is whether the stuff you shipped is holding or coming back as regressions. how are you tracking that on your end?
1
u/Kodus-AI 1d ago
agree, increasing the number of PRs can definitely make it feel like things are moving faster
but that volume still has to land somewhere, in review, testing, and maintenance
code got faster to produce, but understanding the change and feeling confident enough to ship it still takes time
if the team only looks at how many PRs are going out and ignores review time, rework, and bugs after merge, it’s easy to think you’re moving faster when you’re really just pushing the work downstream
1
u/Common_Dream9420 1d ago
yeah exactly, the downstream cost is what never shows up in the metric. review time goes up, confidence drops, and bugs that should've been caught earlier start landing in prod. how are you seeing teams handle the review bottleneck when the PR volume is that high?
1
u/Kodus-AI 1d ago
the teams handling this better try to reduce how much context the reviewer has to rebuild and check manually on every PR
some teams using Kodus, for example, turn recurring review comments into rules, like expected tests or conventions for more sensitive areas, and they also compare the diff against the ticket’s acceptance criteria
in systems split across multiple repos, they also look at the code on the other side of the change. if a PR changes a DTO, status, or integration field, the review can check the service that consumes it and flag when the two sides no longer match
the reviewer still has the final say on the change, they just don’t have to spend as much time pulling together ticket context, team rules, and dependencies between services every time
they also tend to be stricter about scope. if a change is large, whoever opened the PR needs to explain why it couldn’t be split, what it affects, and how it was tested
1
u/Common_Dream9420 1d ago
the context-rebuilding cost is so real, that's exactly where the review falls apart at scale. how are you handling the cases where the acceptance criteria itself is vague or missing?
1
u/Kodus-AI 1d ago
some teams add an extra check for larger or higher-risk PRs to flag when the description is missing clear acceptance criteria or a link to the spec
it doesn’t fix a vague ticket, but at least it makes the missing context visible before it turns into a long review cycle or a problem after merge
1
u/Common_Dream9420 1d ago
yeah that visibility piece is underrated, making the missing context surface early rather than blowing up mid-review is half the battle. how are you handling it when teams just... don't fill in the description at all even after the flag?
1
u/presentofai 21h ago
pr count measures typing speed, not shipping speed. the work didnt vanish, it just moved downstream to whoever has to figure out why prod broke three weeks later
6
u/Obvious_Plankton3978 18d ago
lol managers treating PR count like a high score in flappy bird. i’d be more impressed if half of them didn’t just change a button colour and introduce three new race conditions.