We built our llms.txt to auto-generate from live data instead of writing it by hand
A few weeks ago we added an /llms.txt file to our site (buscasclientes.com) so AI crawlers like GPTBot, ClaudeBot and PerplexityBot get a clean, structured summary of what we do instead of trying to parse our rendered pages.
Most llms.txt examples I found online are static markdown files someone writes once and forgets. Ours is a Next.js route handler that builds the file on every request straight from the same data sources that power the site: blog posts array, success-case list, service categories, city landing pages, solutions list. If we publish a new blog post or add a new case study to our data file, it shows up in llms.txt automatically — nothing to remember to update by hand.
It also links out to a second file, llms-full.txt, with the full text of key pages inlined for models with large context windows, versus the link-only summary in llms.txt for lighter crawlers.
One thing we learned the hard way: keep a single source of truth for factual claims (company HQ city, in our case) and reference it everywhere instead of copy-pasting the same sentence into multiple files. We caught a stale "Madrid" HQ claim baked into both the copy-pasted description AND the llms.txt route itself, because it was written by hand in two places instead of pulled from one constant. Small thing, but it's exactly the kind of inconsistency that erodes trust when an AI model cross-checks facts about your company across sources.
Happy to share the route.ts code if anyone's building something similar.
You can see the live output here: https://buscasclientes.com/llms.txt
The stale HQ example would make a useful regression test: change the company fact in a fixture, then check the rendered page, llms.txt and llms-full.txt all agree. That would catch a future copy-paste reintroducing the same problem even after you've centralized the source.
The Madrid catch is the useful half of this, more than the generation. Nobody has yet shown a major crawler actually reading llms.txt, so before treating it as a channel I would check the server logs: count GPTBot and ClaudeBot hits on /llms.txt against hits on /sitemap.xml over a month. If the ratio is near zero, the file is documentation for humans and the fact consistency work is what is actually paying off.
Second check, cheaper: ask ChatGPT and Perplexity where the company is based, once a month, and log the answer. That is the only instrument that tells you whether the single source of truth reached the model. It also catches the browsing case, where the answer comes from Bing's index of your pages rather than from anything in the file, which for ChatGPT is the case most of the time.