Most companies' terms of service quietly ban what you're about to build
Before devdigest, the daily tech digest I build and run, could launch, I had to check something unglamorous: did any of the 100+ sources I pull from explicitly ban what I was about to do?
I expected the answer to track company size. Bigger, more corporate sites, I assumed, would be relaxed about this - they run their own RSS feeds, press APIs, developer-relations teams. Smaller, scrappier publications felt like the ones who'd actually care.
I read the real terms of service for every source, not the privacy policy, and quoted the actual clauses rather than assuming from "all rights reserved" boilerplate (which is close to universal and, alone, isn't real signal - it's the same language that would technically ban what every RSS reader does).
I had it backwards.
A VC-data company's terms ban using content "to train models (including generative artificial intelligence technologies)." A European startup-focused publication bans "any text or data mining or web scraping... for any purpose, including... training, fine-tuning or validation of AI systems." A major infrastructure company's own blog bans bots from "developing, training, fine-tuning... a machine learning model or artificial intelligence (AI) system." All explicit, all recently added, all naming AI training specifically, not generic anti-plagiarism language.
Meanwhile: an engineering-community site's terms say outright, "We permit the posting of a summary and then a link back." A university's news terms say it "offers RSS feeds for syndication purposes." A major code-hosting platform's terms explicitly carve public repositories out of any restriction at all.
The pattern held across the full source list: the sites closest to VC/startup data and big corporate infrastructure wrote the most specific, most recent anti-AI-training clauses. The engineering blogs and syndication-native sites, the ones you'd expect to be precious about their content, mostly weren't.
My best guess why: the well-lawyered sites have their own data-licensing business to protect, so they wrote the clause. The smaller publications' whole distribution model already runs on RSS and being linked to - being read is the product, not the risk.
devdigest only ever excerpts and links back, never reproduces full articles. Four sources still got removed outright once their terms explicitly named this exact pattern. Better to lose a source than build a paid product on top of an explicit no.
If you're building anything that reads other people's content automatically: check the real terms, not just the ones you'd guess would be strict.
