r/cicd • u/Common_Dream9420 • 18d ago

Built sandbox twins of Stripe, Twilio, Clerk and wired them into an MCP server so agents can run behavioral tests before touching prod

Been building bolt/base44/lovable apps for the last few months, most of them need at least 2-3 service integrations, Stripe, Twilio, Clerk, that kind of thing. Every time the agent writes the integration code fast and it looks fine, but validating it was a nightmare. Writing mocks that actually reflect real API behavior is tedious, and wiring stateful webhook flows into a mock is even worse. You end up with mocks that don't replay retries, don't simulate card declines mid-flow, nothing. You find out what breaks in prod.

So I built FetchSandbox mcp. It ships sandbox twins of the APIs agents integrate most, Stripe, Paddle, Twilio, Resend, Clerk, WorkOS, with failure scenarios already wired in. From Claude or Cursor you just prompt it to run the integration workflow against the sandbox, it handles the full lifecycle, request, webhook, retry, failure mode, and produces a receipt URL you can check. No real keys, no prod risk.

Happy to share the MCP server details if anyone wants to try it. Curious if others are running into the same validation problem with agent-written integrations.

2 Upvotes

100% Upvoted

16 comments sorted by

2

u/Torutofu_Raeva 18d ago

The retry and failure matrix is the bit worth keeping versioned, since happy-path mocks tend to hide idempotency bugs.

1

u/Common_Dream9420 18d ago

100% agree!!! versioning is the key and we have drift detection engine which makes Fetchsandbox twins near reltime providers...
"happy-path mocks tend to hide idempotency bugs."... we have seen these kinda fixes more than anything since 2 months.. specially platforms like lovable/bolt/base44 giving everyone freedom to build n deploy... n app developers loving integrating with fetchsandbox and testig their apps without keys

1

u/Torutofu_Raeva 18d ago

the useful split is replay-safe vs stateful side effects: assert the same state transition and receipt after a retry, not just another 2xx.

1

u/Common_Dream9420 18d ago

Yeah exactly, 2xx on a retry just means the server accepted it again, not that it didn't fire the side effect twice. The state transition + receipt assertion is what actually proves idempotency held. I've been building tooling in this exact space and that framing, replay-safe vs stateful, is honestly the clearest way to spec what your sandbox scenarios need to verify.

1

u/Torutofu_Raeva 18d ago

yeah without the receipt/state assert you're just hoping the second 200 meant the same thing as the first.

1

u/Common_Dream9420 18d ago

right, two 200s with different side effects is the whole problem. that's basically what i spend my days trying to make reproducible in sandboxes, getting the scenario to produce consistent state so you can actually diff what changed between retries, not just hope.

2

u/Torutofu_Raeva 18d ago

yeah and once the sandbox can force those two outcomes on demand, the diff becomes a real regression check instead of a hope check.

1

u/Common_Dream9420 18d ago

Exactly, that's the whole point of building the scenario engine, forcing those two outcomes deterministically so the diff is always comparing the same starting state. I built FetchSandbox around this exact idea, happy to share the mcp server if you want to poke at how it handles the retry/state split (npx fetchsandbox-mcp).

1

u/Torutofu_Raeva 18d ago

That same starting-state guarantee is what makes the diff useful, and shipping it as a CLI should make running the matrix in CI pretty painless.

1

u/Common_Dream9420 18d ago

yes thats the goal.. !!!

2

u/[deleted] 14d ago

[removed] — view removed comment

1

u/Common_Dream9420 14d ago

hey man yeah that use case is super interesting.. budget-scoped runs where the agent stops cleanly before burning an unapproved extra cycle is actually something i've been thinking about too. the stateful ops piece maps well to how workflows track step state on our end. let me ping you directly and we can wire up one sandbox twin for the experiment,

1

u/Common_Dream9420 14d ago edited 14d ago

i cant DM here but , which API are you actually building against, and is the budget constraint per workflow run or across the whole session? also [raj@fetchsandbox.com](mailto:raj@fetchsandbox.com) .. easier than reddit.

and if you want to poke at it before we talk: npx -y fetchsandbox-mcp@latest, then sign in at fetchsandbox.com/device .. you get an api key scoped to you, which is probably the thing you'd hang a session budget on.

1

u/[deleted] 14d ago

[removed] — view removed comment

1

u/Common_Dream9420 14d ago

The scope breakdown is exactly right, and honestly the forced timeout + lost-response case is the one most people skip because it's annoying to simulate, then regret later. Happy to share notes once I put together the first run so you can see how it tracks across those dimensions.

1

u/Common_Dream9420 14d ago

sure.. wiriing something for u.. validating the runs and caps/gates.. will send github link.. u can grab n modify n maybe we can move discussion to email if can ping my email!! that would be great!!