Im working on DocsKoala. it generates docs from merged github prs. before building out more automation i wanted to actually know how much of the problem was automatable versus how much genuinely needs a human, so i ran a real audit instead of guessing.
pulled 14,891 merged prs across 220 public repos. classified each as customer facing feature, internal, bug fix, or unclear. for the 3,623 customer facing ones, checked whether the pr diff and title alone gave enough to draft a doc, and separately whether it ever got documented at all.
1,386 features, 38.3%, were automatable based on the diff alone, no extra context needed from the dev. 541 features, 14.9%, never got documented period, automatable or not.
cost estimate if a human wrote docs for just the automatable chunk manually, 20-30 min each, comes to 462-693 hours and 34.6k-52k dollars across these 220 repos.
this number is basically why DocsKoala focuses where it does. it drafts automatically for the 38% that's clear cut, and flags the rest for a human instead of pretending it can guess context that isn't in the diff. figured people building similar tools or just curious about the "how automatable is documentation actually" question would find the raw split useful.
Thats nice!!! A 38% is absolutely wild
I think the most valuable part of this isn't the 38%.
It's that you identified the boundary where automation stops being trustworthy. A lot of AI products try to maximize automation. Defining where a human should take over often creates more confidence than squeezing out a few extra percentage points.