I built an AI system to verify 150+ credit card benefit records. Here's how hard accuracy actually is.

I launched Norte on the App Store yesterday. A wallet intelligence app that shows you the hidden benefits, credits, and coverage already inside your credit cards.

Before launch I spent weeks building a Claude Code skill to audit our database. The premise was simple: pull the card data, check it against official sources, propose fixes, human approves. AI proposes. I approve. Nothing ships unchecked.

What I didn't expect was how hard "official sources" actually is for credit card benefits.

An opaque industry by design

Most people assume credit card benefits are standardized. Visa Signature cards have Visa Signature benefits. Mastercard World Elite cards have World Elite benefits. The network sets the rules.

That's wrong.

Issuers override network baselines constantly. Two Visa Signature cards from different banks can have completely different baggage delay triggers, different purchase protection limits, different rental car exclusions. The only source of truth is the issuer's official benefit guide, sometimes hidden in multiple PDFs written in legal language that most cardholders never read.

When those PDFs aren't publicly available, there is no reliable source. I found this on multiple cards where the benefit guide is mailed to cardholders only. For those cards we could only rely on what the product page described.

What the AI audit system caught — and what it missed

The skill we built was genuinely useful. Across 150+ cards it caught:

• Field misplacements (durations stored in amount fields)

• Wrong canonical values (country names, confidence levels)

• Activation requirements from marketing copy instead of fine print — these are consistently different and consistently matter

• Rental car excluded countries missing or incomplete

• Network benefit assumptions applied to cards where the issuer had overridden them

What it missed — and this is the important part — was when it substituted third-party sources for official ones without flagging it. The skill said verify against official sources. Claude used Tavily and existing row data instead. No warning. Confident output that looked right.

What we did about it

Re-audited the top 15 most-saved cards manually with PDFs in hand. Core amounts and boolean flags were correct across the board. The issues were enrichment fields — missing exclusions, incomplete activation requirements, sub-limits not documented.

The fundamentals held. But I now have a rule I didn't have before: the AI audit output is a proposal, not a verification. The human check isn't a rubber stamp. It's the actual work.

Why this matters for Norte specifically

Norte is insurance-adjacent. When a user checks whether their card covers rental car damage and relies on that answer at the counter, wrong data has real consequences. That's a different standard than getting a cashback rate slightly wrong.

We added a confidence layer to the database — HIGH, MEDIUM, LOW — based on source quality. Cards where we couldn't get the official PDF are flagged. The AI assistant surfaces that uncertainty rather than hiding it.

It's not a perfect database. It's an honest one.

Where we are now

150+ cards across US, UK, and European markets. Top wallet cards verified against official benefit guides. Data quality improving with every audit session.

If you're building anything with AI on data that real people rely on — the verification layer isn't optional. And the verification layer needs its own verification layer.

Happy to share the skill structure or talk through what worked and what didn't.

Norte: norteapp.io