Ploutos AI

I build a tool that reads SEC filings and writes fundamental research on US stocks. The interesting engineering problem turned out to be nowhere near where I expected it.

Every US public company files financials in XBRL, a machine readable format, free, straight from the SEC. Which sounds like a solved problem: pull the tag, read the number, done. That works right up until you point it at a filer that does not look like the ones you tested on.

Here is the one that taught me. A company had migrated from filing as a foreign issuer under IFRS to filing a normal domestic 10-K under US GAAP. Its quarterly dividend rows were tagged with the annual period marker, and there was no summable trailing twelve months anywhere in the data. My code did the obvious thing, found what looked like an annual figure, and published a 2.24% yield against a real one of about 8.96%. Nothing errored. Nothing looked odd. It just quietly published a number that was wrong by a factor of four.

That was not an isolated case. A bank filed its quarterly dividend table inside the 10-K, and we returned $0.26 for the year. An insurer returned $2.83 against a real $4.55. A company that pays no dividend at all needs the answer to be "none", not zero, because zero is a claim and none is an absence. One large filer exposes no per share dividend tag whatsoever, and the only correct behaviour is to say so rather than to derive something plausible.

Then it got worse in a more interesting way.

A pre revenue biotech parks about three quarters of its liquidity in marketable securities. That position sits under an accounting element I had not mapped, so it read as zero, and the tool published a cash floor valuation of $1.43 when the real figure was $4.29. I fixed it, felt clever, and then found the same failure at a second company using a different element for the same line. Then a third, using a third element, where $25M of bank cash sat next to $234M of securities. Liquidity read ten times too small.

Three companies. One balance sheet concept. Three different tags.

That is the thing I wish someone had told me earlier. XBRL is not a schema that constrains filers into one shape. It is a vocabulary they choose from, and a tag chain that happens to cover the one ticker you tested is not a tag chain that covers the concept.

What actually fixed it

Not more unit tests. I had unit tests, and they all passed, because they were written against my own understanding of the data. A unit test proves your resolver is self consistent. It cannot tell you that your understanding is wrong.

What fixed it was a golden set: real SEC companyfacts payloads, frozen as fixtures, for the filers whose shapes break naive extraction. The foreign issuer mid migration. Two IFRS filers in different currencies. A retailer on a 52 or 53 week fiscal calendar. A June year end. A monthly dividend payer, where four times the quarterly figure is not the answer. A company that pays nothing. A company the SEC has no tag for.

The part that matters: every expectation in that file was checked against what the issuer actually disclosed, not recorded from whatever my code happened to return. That inverts what the test is for. A failure now means we disagree with reality, not that we disagree with a fixture someone captured on a Tuesday.

The whole set is a few hundred KB and runs offline, so it costs nothing to run on every commit.

The part that generalises

If your product reads data you did not produce, from sources that are allowed to disagree about shape, then your test suite is probably measuring the wrong thing. Mine was. Passing tests told me the code did what I meant. They could not tell me that what I meant was wrong for a quarter of the universe.

The bugs in this post were all found in production, by looking at outputs and asking whether the number was believable, not by anything going red. That is the uncomfortable bit. Every one of them shipped silently, and every one of them was a plausible looking number in the right units, on the right scale, next to a company name.

Happy to go deeper on any of it if it is useful to anyone.