Teaching Fileo to tell a real document from a random photo
When you build an OCR-based automation system, there’s a surprisingly tricky question hiding in plain sight:
How do you know if an uploaded image is an actual document — or just a random photo that happens to contain some text?
In Fileo, OCR runs on every upload — it’s the foundation for everything that happens next.
But to avoid wasting resources (and to keep garbage out of the AI pipeline), I needed a way to score how “document-like” a file is after OCR has analyzed it.
⚙️ Multi-Signal Document Detection
The solution: a scoring system that combines multiple signals derived from the OCR output to decide whether something looks like a real document.
1. OCR Confidence (60 pts)
Based on Azure Document Intelligence’s own recognition confidence.
Real documents typically exceed 90 %, while photos with small or distant text drop to the 60–70 % range.
2. Text Coverage (40 pts)
A custom metric: how much of the image area is actually covered by recognized text?
Computed from the word-level bounding polygons (Shoelace formula for quadrilateral area).
This turned out to be a great differentiator between structured layouts (invoices, letters) and arbitrary photos.
Final document score = 0.6 × OCR confidence + 0.4 × text coverage score
Threshold: 45 / 100 → permissive on purpose — better to accept one extra photo than to reject a real invoice.
🧮 Debugging surprise: coordinate systems matter
At first, I assumed Azure returned normalized coordinates (0–1).
Wrong.
PDFs use inches (e.g. 8.5×11"), images use pixels (e.g. 5712×4284).
That mistake produced glorious “3,473,650 % text coverage” results before I fixed the scaling.
📊 Calibration
I tested the system with different document types to find reasonable thresholds:
Dense text (like book pages): around 49% text coverage → full credit
Typical documents (invoices, letters): usually 5–15% coverage → accepted
Photos with small labels or signs: below 1% → rejected
Turns out real documents almost never exceed 50% text coverage — margins, whitespace, and layout make a huge difference.
💡 Key takeaway
False negatives are far worse than false positives.
If Fileo misses a real document, that’s a failure — if it processes a random photo, no harm done.
So the guiding principle is:
When in doubt, process it.
