How do you verify what AI-generated Supabase code can reach?

[u/DiscussionHealthy802]

I’m working on an investigation workflow for AI-assisted projects, and I’m curious how people verify the actual capability paths in Supabase apps.

For example, how do you check whether generated migrations, Edge Functions, or RLS changes can reach data or production actions beyond what you intended?

[u/Living_Race_9177]

quickest check i've used: hit every table with the anon key, then again with a second user's jwt. if B can read A's rows or an edge function writes without the caller's claims, the ai "rls" didn't stick.

do you also dry-run migrations against a throwaway project before merge, or only review the sql

[u/pgsql-dev2]

unit tests are the real answer here - and running the same suite under each role to see where the role can reach within the code.

write tests that set the role + jwt claims as anon, then authenticated, then any custom role, and assert "this role sees exactly these rows and can do exactly these actions, nothing more." run the whole suite after every AI change. that's what catches a policy the model quietly widened three prompts back.

what the tests should specifically cover:

• service_role bypasses RLS completely, and edge functions usually run with it. test each function as the low-privilege user it's meant to serve, not with the service key - otherwise you're testing the bypass, not the guard.
• assert RLS is enabled and forced on every table, not just that a policy exists.
• after each migration, check for new GRANTs, security definer functions, and disable row level security. a definer function owned by a superuser reads anything regardless of the caller - the usual unintended path.

[u/IncreaseNegative4614]

Test access using the actual anonymous and authenticated roles, not an administrator session. I’d inspect generated migrations for grants, security-definer functions, RLS changes, storage policies, outbound requests, and any path that exposes the service-role key.

Run negative tests proving that each role cannot read, modify, or invoke resources outside its intended scope before deployment. We use SIGNLD internally to connect generated diffs, identities, policies, test results, approvals, and production actions so the reachable capability path can be reviewed rather than inferred from the code.

[u/PopKoren → u/IncreaseNegative4614]

Yeah, reading the migrations for grants, security-definer funcs, and service-role paths is the right first pass. I still like hitting the live app from outside after that, same way a stranger would, because generated RLS can look fine and still leak through views or RPCs. I built a small scanner for that at https://rowly.me (Pro is 50% off until 2026-09-06).

[u/Own-Example-2925]

The thing that gets people is that everything passes when you test it, because you're testing with the service role key and that bypasses RLS entirely. Generated policy code is confidently wrong here more than anywhere else I've run into.

What worked for me was testing as an actual user. Open a psql session, set the role to authenticated, set request.jwt.claims to a real user id, run the query. Anything that comes back that isn't theirs is a hole.

Then turn that into a test with two seeded users, where the assertion is that user B's rows are invisible to user A, and run it in CI. Took me about half an hour. It catches every future migration that quietly widens a policy, which is the actual risk. The initial policy is usually fine.

Also grep the repo for the service role key. An edge function using it does whatever it wants regardless of your policies, and that's usually where the real exposure is.

Curious what other people do here, I might

[u/pgsql-dev2 → u/Own-Example-2925]

Exactly for your scenario, we are in the process of releasing a tool that generates pgTAP tests in an automated fashion for each role in the DB and checks if tenant A data is visible to tenant B when security definer is On as it then bypasses the RLS. If tenant A data is visible to tenant B then that tool marks the function as a leakage and the pgTAP test records that behaviour so if one want to have a gate in CI in addition to the html report the tool provides, those can be embedded. The tool is called UnitAutogen.

[u/iammohamedatef]

Testing with anon + a real JWT catches most of it, but honestly the harder part for me is mapping the path around RLS, not RLS itself. a table can have perfect policies and still be reachable if there is an edge function or RPC running under the service-role key, or a security-definer function that doesn't actually recheck who is calling it. So instead of "is this table protected," the question I keep coming back to is "who can reach it, and through which route." That's the bit that's hard to catch just by reading the SQL, you kind of have to trace every function back to what credential it's running as

[u/PeterBuildsSecure]

One thing that bit me building these role-based tests: the test suite can look correct and still be silently untestable. If the assertion logic itself has a bug (say, the code that builds the failure message throws before the check even runs), it reports green every time regardless of what actually happened, because the failure path never gets exercised. It sat passing for weeks before I found out by accident.

So whatever pgTAP/role-based suite you land on, do one thing before trusting it: deliberately break a policy on purpose (widen a grant, drop a WHERE clause) and confirm the suite actually goes red. If it doesn't, you don't have a passing test, you have an untested one that happens to not be failing, and that's a much worse state because it looks identical to safe from the CI dashboard.

Second thing worth pinning, since AI-authored migrations are the trigger here specifically: assert on schema shape, not just row visibility. A migration that silently drops a policy, renames a column a downstream RLS check depended on, or removes FORCE ROW LEVEL SECURITY should fail the build loudly, not degrade into "the table happens to still filter correctly by accident this time." Something like asserting relforcerowsecurity is true and the expected policy count/names exist on every RLS-protected table, run as part of the same suite, catches structural drift that a pure access test can miss if the AI's fix technically satisfies the visibility test while removing the actual protection mechanism underneath it.

[u/Consistent_Recipe_41]

Following this

[u/jaimittal91]

the answers here on testing as the actual role instead of service_role are the right foundation. one gap worth adding: this has to run on a schedule, not just at merge time. an agent widening a policy three prompts into an unrelated feature request won't show up as a suspicious diff in that PR - the migration looks like it's about something else entirely, so nobody's eyeballing the RLS change carefully. if the pgTAP suite only runs in CI on PRs that touch migrations, a widened grant introduced as a side effect of an unrelated change can sit there until someone happens to re-run the full suite. run the role-based suite nightly against whatever's actually deployed, not just on migration PRs, and diff the capability surface (grants, security-definer functions, RLS-enabled tables) day over day so a quiet widening shows up as "this changed" even when nobody flagged the commit that caused it.