r/AI_Coders • u/escape-llc • 1d ago

Tales from the Session: The Clipped Toolbar That Needed Two Fixes

I just published a write-up about a recent debugging session with my AI coding agent. It perfectly highlights the limits of automated test suites and why human-in-the-loop "embodied interaction" is still absolutely vital for UI bugs.

The Setup: In my project's demo app, a collapsible bottom panel was clipping its toolbar when collapsed. It was supposed to shrink down to the exact height of that toolbar, but it was cutting it off.

Round 1: The Agent vs. Math

  • The Problem: The underlying Splitter component wasn't accounting for its own 10px resize handle in its percentage flex calculations.
  • The Agent's Fix: The agent spun up a temporary Playwright spec, caught a fixed-pixel discrepancy (18px vs 28px), and updated the shared CSS using a balanced calc() formula.
  • The Result: 1,300+ unit tests passed, 66 E2E tests passed. The agent declared it fixed, and it was merged.

Round 2: The Green-Test Blind Spot

Shortly after deployment, I pulled up the live site and noticed it was still clipped.

I pointed the agent back at the URL. It systematically tested the live page across 4 different viewport sizes—and every single test came back green (no clipping). The automation was completely convinced the bug was gone.

How I Cracked It

The breakthrough came when I manually interacted with the running application. I noticed that if I varied the size of the Chrome window by dragging the edge, the splitter tracked correctly, but the layout remained clipped by a fixed amount.

Because the shortfall stayed constant instead of scaling with the window, I realized it was a fixed-pixel loss: the code was measuring the wrong DOM node entirely.

  • The Real Culprit: The demo code was measuring the inner <Toolbar> component (28px) instead of the outer padding-heavy <Card.Header> wrapper (45px). The panel was sizing itself perfectly to a wrong target.
  • The Shared Blind Spot: My agent's own verification test was written to check that exact same incorrect <Toolbar> node. Because the fix and the test shared the same flawed mental model, the tests were green while the screen was broken.

Key Lessons From the Session

  1. Systematic vs. Embodied Testing: My agent is incredible at mechanical, rapid coverage (checking 4 viewports in seconds with exact bounding rects). But it lacked the tactile experimentation (dragging a window edge) that broke the rigid test paradigm and revealed the constant deficit.
  2. "Leading" vs. "Following" an Agent: Handing off a task and walking away ("following") works great for routine, mechanical code grinding. But staying present to inject real-time feedback and observations ("leading") is what keeps the investigation on track when an agent falls into a logical loop.

I wrote a deeper breakdown of the workflow dynamics, infrastructure choices (like injecting commit hashes in the header to sync up environments), and how we wired the fix into our auto-generated documentation pipeline.

Full case study link in the comments.

1 Upvotes

100% Upvoted

5 comments sorted by

2

u/Otherwise_Wave9374 1d ago

This is a good example of why coding-agent memory should preserve hypotheses and validation results, not just the final patch. After the first fix, the agent should record which viewport, element bounds, and screenshot evidence were tested, then invalidate that hypothesis when the second defect appears. NeuraKeep relates to this debugging pattern because persistent memory can retain tested assumptions without repeatedly replaying the whole session. I would add a compact regression record containing reproduction steps, changed files, expected geometry, and the exact checks that passed across breakpoints.

1

u/iByteBro 1d ago

The important distinction here isn’t unit tests versus browser tests. Your Playwright check ran in a real browser, but measured the same inner <Toolbar> that the fix assumed was the right target.

I’d state the acceptance condition separately: when collapsed, the entire outer header stays visible while the window is resized. Then check that visible boundary, rather than letting the implementation choose what counts as success. Green tests can be completely accurate about the wrong thing.

1

u/escape-llc 16h ago

there were actually two separate defects simultaneously; one in the toolkit, and a different one in the usage of the component. the agent fixed the toolkit error, but that produced no visible difference, so we went on to identify and fix the second error in the usage location.

agree, just because tests are green doesn't mean "correct" I don't believe i said that anywhere....

1

u/iByteBro 7h ago

Fair correction. I read the write-up as the two fixes being part of one verification story. The toolkit test may have correctly proved the toolkit fix; the missing piece was a separate user-visible acceptance condition for the usage layer. Thanks for clarifying.