I open-sourced all the edge cases I've found while building my bank statement parser, Bankstatemently
I'm doing something crazy: I'm giving everyone a chance to build their bank statement parser faster
Today, I'm releasing the Bankstatemently Open Benchmark, the world's first open benchmark for bank statement parsing 🎉
It exposes all of the challenges and edge cases that I've come across so far while building my product. All of the secrets that other companies would call their IP
After processing many bank and credit card statements at Bankstatemently, I've run into plenty of edge cases. Affectionately, I've started calling them "quirks" and cataloged 37 of them so far
They range from:
• Credit card charges shown as positive numbers
• Thai Buddhist calendar dates (year 2568 instead of 2025)
• Transaction dates hiding inside the description text
• Day and month split across two separate columns
Why give all of this away?
Because the challenges aren't actually the secret. Everybody can now build a bank statement converter
The secret is solving all of them at once, reliably, and at scale. Any SOTA vision model can pull out transactions from a PDF. Interpreting the data is the hard part
But here is the problem: you can't test any of this against real data. Bank statements are private. You can't share them, benchmark against them, or publish them. So every tool claims "99% accuracy" and there's no way to verify
That's why I built the benchmark. 5 synthetic statements from 5 countries, 14 of the 37 documented challenges, each using a fictional bank with realistic formatting. Free and open-source
Download the PDFs. Run them through your parser. Submit to the evaluation API and get a detailed accuracy breakdown. You're free to do whatever you want with them
Why does this matter to me?
I care enough about accuracy that I built the evaluation system before I built the actual product. You can't have a financial document be a little bit wrong. People build their businesses on trusting your results
I'm planning on 10 more statements. What ends up in those 10 will partly depend on your feedback. If you know a parsing challenge that's not in the list, tell me
Read more: https://bankstatemently.com/blog/bankstatemently-open-benchmark-bank-statement-parsing
Or check out the public Github repo: https://github.com/bankstatemently/bank-statement-parsing-benchmark
