The database I could see wasn't the database that mattered

A side project of mine — a review-content app built on Lovable with a Supabase backend — had a data bug: most products showed only summary cards where full in-depth reviews should be. Routine diagnosis, or so it looked.

I pointed my AI tooling at the database. The connector listed a Supabase project with the right name, the right tables, the right schema, plausible rows. Everything about it said "this is production." The diagnosis was forming, the fix was a few UPDATE statements away.

One habit saved the afternoon: before letting anything write, prove which datastore the running product actually reads.

The database I could see wasn't the database that mattered

The live app's JavaScript bundle has its Supabase credentials baked in — that's how these frontend-first stacks work. So I pulled the project ID out of the deployed bundle and compared it against what my tooling could see.

They didn't match. The project visible to my AI tools was a stale duplicate — same name, same schema, different product IDs, different data. Production was a different project entirely, one my tooling had no access to at all. Every "fix" I was about to apply would have executed cleanly, returned success, and changed absolutely nothing on the live site.

That's the worst category of failure: not the error, but the false success. I'd have closed the ticket, the bug would have remained, and I'd have trusted the broken state more afterwards, because hadn't I just fixed it?

The most dangerous fix is the one that succeeds in the wrong place.

How a duplicate like this comes to exist

Nobody creates a decoy database on purpose. It accretes: an early sync, a migration experiment, a tool that cloned the project during setup, and a year later there are two same-shaped databases and only the deployment knows which one is real. This is the oldest lesson in systems work — the twelve-factor app's dev/prod parity principle exists precisely because environments that drift apart produce code that "works" everywhere except reality. The modern wrinkle is that environment drift now bites a new audience: AI agents, which will confidently operate on whatever environment they can see.

And that's the structural point. The AI did nothing wrong. It was handed a connection, the connection contained a plausible database, and it reasoned correctly about the data in front of it. The wrongness lived one level up, in the unverified assumption that visible equals production. No amount of model intelligence fixes a mis-wired context.

The verification tax is the job now

This is the third entry in what's becoming a series. The revenue number that was convincingly wrong taught me to verify the data. The reply that was message 28 taught me to verify the fetch. This one adds the deepest layer: verify the target. Data, retrieval, destination — three different places for confident tooling to be silently wrong, and the failure always wears the same face: clean output, no error, full plausibility.

The working rule I've taken from it: an AI may diagnose from any database it can see, but it may not write until the target is proven — by reading the config of the running system itself, not the connection list of the tooling. The bundle, the environment variable, the deploy manifest. Ground truth lives where the product runs, a discipline I first wrote about in the anti-hallucination audit.

The actual bug, for the record, was found once I was looking at the real database: the content pipeline never ran its final step for newly imported products — and the orphaned duplicate project is getting deleted, so this particular trap can't reset itself.

Trust the model's reasoning. Audit its wiring.

Sources & further reading

External

The Twelve-Factor App — Dev/prod parity · OneUptime — How to implement environment parity

Related posts

The AI handed me a number that was wrong in the most convincing way · The AI said nobody replied. The reply was message 28. · The anti-hallucination audit · Lovable just fixed its biggest SEO problem