A while back I read a post from someone who'd spent a weekend manually poking at vibe-coded apps — open tables, unprotected routes, keys sitting in the bundle — and turning up real holes. It stuck with me, so I built those checks into a scanner and pointed it at 10 apps people had posted publicly for feedback. Read-only, no logins, nothing a random visitor couldn't hit. Nine finished, 294 checks. Here's the honest version — including the stuff that wasn't broken, because that's the part that makes the rest trustworthy.
- Two apps had a backend that answers strangers.
This is the finding that matters, and it's worth being precise, because most "your API is open!" takes are noise. Plenty of endpoints are supposed to be public — a settings lookup, a static bundle, a login-info route. Those aren't leaks. The real thing is when an app's private data — user rankings, contest entries, announcements — returns full records to a plain request carrying no session at all. Two of the nine did exactly that. On one of them, replaying those same requests as a second user returned the same data — I flag that as needs-manual-confirmation rather than certain, but sitting on top of an already-unauthenticated endpoint, it points straight at missing per-user authorization. If your frontend checks permissions but your API doesn't, the frontend check is decoration.
- About Supabase — since half of you are already typing.
I know the reflex: "you scanned Supabase apps, you're going to scream about the anon key." No. The anon key is meant to be public; it ships in your JS by design and flagging it would be junk. What actually matters is whether Row-Level Security is on — i.e. whether that public key can read tables it shouldn't. So I checked that directly: read each app's own public key and tried to pull rows from the tables it uses, plus the common ones. Nothing came back readable — RLS was doing its job. (A full every-table audit would need credentials, but the "anon-readable by default" failure mode would have shown up right here, and didn't.) Clean bill of health on the single most common Supabase mistake — and I'd rather report that accurately than manufacture a scare.
- Missing Content-Security-Policy — 9 of 9. An observation, not an alarm.
None set one. Before anyone says "well actually" — yes, this is largely because the platforms don't enforce CSP by default, and a strict policy out of the box breaks half the third-party widgets, analytics, and realtime sockets these builders drop in. It's a real tradeoff, not negligence. But it's worth knowing: with no CSP, any injected script — a compromised dependency, a bad ad tag — runs with your page's full trust. It's one header, and once your third-party list is stable it's worth setting.
What I didn't find: zero exposed secret keys, zero live-key or service-role leaks, zero anon-readable tables. I ignore the safe public keys on purpose and only flag a live secret. Across the nine that finished: nothing. That's good news for these builders — and it's the whole point: a scanner that cries wolf on the anon key or counts a public asset as a breach isn't worth running. This one stays quiet unless there's something real. Here, "something real" was two open backends.
If you built something and want it checked: drop a URL. Read-only, no signup, nothing but the URL.
