I could not tell if Google's AI Overview was citing my sites, so I built a way to check
I run a couple of small content sites. One of them answers questions about PFAS in consumer products, which is exactly the kind of informational query Google now answers with an AI Overview before anyone gets near a blue link.
Last spring my analytics started doing the thing everyone warns you about. Impressions held up. Clicks didn't. The obvious suspect was the AI Overview sitting above my results, summarizing an answer I'd written and citing somebody else. Obvious, but I couldn't prove it, and I couldn't have told you which queries it was happening on.
So I did the thing everyone does first. Incognito window, type the query, squint.
The manual version, and why it didn't last a week
Checking one query by hand takes about forty seconds. Search, wait for the overview to render, read the citation chips, write it down.
Forty seconds is nothing for one query. I had roughly two hundred I cared about across two sites, and the answer changes over time, so it's a chore that comes back every week. Two hundred queries is a bit over two hours of clicking, and what you get for it is a spreadsheet that starts going stale the moment you close it.
There's a worse problem underneath that one. Eyeballing doesn't turn into anything you can analyze later. I wanted to ask things like "across these forty queries, which domains does Google's AI cite most, and am I anywhere on that list." A folder of screenshots can't answer that.
What I actually needed was datasets for analysis
What I kept reaching for was structured output. One row per query, the answer text, and every cited source broken out with its title, link, and domain.
Once each check is a row, the questions I cared about get boring to answer, which is the point. Which of my pages get cited. Which competitor keeps turning up instead of me. Whether an overview shows at all, because Google skips plenty of queries entirely.
Building it was less about parsing than I expected and more about one quirk I didn't see coming. Google doesn't always hand over the overview with the search result. Sometimes it defers and gives you a short-lived token instead. That token dies in about a minute, so you can't hand it to a user and let them deal with it later; it has to get resolved inside the same run or the data is just missing. A naive scraper logs those queries as "no overview" when there was one sitting right there.
I'd guess that quirk accounts for most of the gap between a check you can trust and one that quietly under-reports.
Wiring it into AI agent workflows
The part I didn't plan was how much I'd end up using it through Claude instead of through anything I built.
I made it callable as a tool over MCP mostly to see if it'd work. Now it's how I actually use the thing. I ask in plain language, something like "is my domain cited in the overview for these ten queries," and the answer comes back in the conversation. No dashboard, no spreadsheet. For the recurring version I run it on a schedule and let the history pile up, which is the part that matters, because a single snapshot tells you nothing about whether you're gaining or losing.
Where it stands a month in
It's public on Apify now as the Google AI Overview API. Today it's at 162 total users, 80 of them active in the last month, and a little over 8,000 runs. Four reviews, all five stars, which I'll caveat myself: four is a small number and I'd rather say so than round it into something it isn't.
It costs a cent to start a run and a cent and a half per retrieval, so a normal query runs about two and a half cents and a deferred one about four. My original two hundred queries now cost a few dollars a week instead of two hours.
The two use cases I built first are still the two I run: checking whether a brand appears in Google AI Overviews, and seeing which sites an overview cites for a keyword. If you want the technical write-up rather than the story, that's over on Dev.to, and there's a shorter walkthrough on Peerlist.
The limitations are worth more to you than the pitch, so: Google mostly shows overviews for English, question-style queries, in a limited set of countries. A real chunk of queries come back with nothing. That's a legitimate answer rather than a failure, but coverage is patchier than you'd assume going in.
What I'd tell another founder
I spent about two months worried about a trend I couldn't measure and about a week building the measurement. That ratio still bugs me.
If something is quietly eating your traffic and you're guessing, turning the guess into a row in a table is cheap. Cheaper than the guessing was, anyway. Mine told me I was cited less than I thought and that one competitor kept showing up instead, which was annoying but at least it was actionable.
Two things people keep asking me
Someone asked me what AI agent workflow automation actually means here, because it gets used for everything. In this case it's narrow: you make a tool an agent can call, and the agent handles the fetching and the deciding instead of you clicking a UI. Mine calls the overview check, reads the citations, and tells me what moved since last week.
The other one is which platform lets you chain multiple AI agents in workflows. Anything that speaks MCP will let an agent call a tool like this directly, and the usual automation platforms cover the scheduled multi-step version if you'd rather keep a person in the loop.
The gap you closed is the one analytics never closes on its own: impressions without citation data are just a feeling that something is wrong. Once each query is a row with domains, you stop arguing with yourself about whether AI Overview is the culprit and start ranking which pages you are losing, and to whom. Treating the SERP as a dataset instead of a screenshot is the whole unlock, deferred tokens and all.
"Treating the SERP as a dataset instead of a screenshot" is a cleaner statement of the point than the one I wrote, and the bit about no longer arguing with yourself is the part that changed day to day. The argument was the expensive thing.
The ranking-which-pages-you-are-losing-and-to-whom half is where I still have work to do. A few people in this thread have pushed on the same gap, which suggests the citation data is only half useful until the competitor side is a first class column rather than something you read manually.
I like the fact that you built something to answer a very specific question instead of creating a broad analytics tool. What was the first signal that made you think people other than you might actually need it too?
The ratio you flagged, two months guessing versus a week measuring, is the whole post, and most founders have one of those sitting in their business right now. The risk with measurement tools is that curiosity churns: people check until they have an answer and then stop, which is why the scheduled history is the only part worth charging recurring for. I would stop selling the check and start selling the trendline, because nobody cancels the thing that tells them they lost three citations last week.
"Stop selling the check, start selling the trendline" is the most useful sentence anyone has said to me about this, and I think you are right that curiosity churns. Somebody runs it, learns they are cited four times out of forty, feels informed, and never comes back.
The uncomfortable part is that the pricing follows the retrieval right now, which is the churning behaviour, not the history. Per-retrieval pricing is honest and cheap but it is optimised for the one-off check.
On the ratio: two months guessing against a week measuring being the normal split is bleak and probably true. Your diagnosis that the delay is admitting the trend was real, not the tooling, matches my experience of it exactly.
162 users and 8,000+ runs is a solid amount of real usage for something this specific. Curious what users are actually doing most often with it.
That's a great question. I tried to reach out to other folks about what they're using and have built out a few examples here:
https://apify.com/johnvc/google-ai-overview-api/examples
Got it, I’ll take a look at the examples. I’d be interested to hear which use case ends up becoming the most common as usage grows.
You don’t need a tracker on day one. You need a prompt set and a spreadsheet.
Minimum viable check:
15 queries you actually care about (branded, category, comparison, objection).
Run them signed out, same location, once a week.
Record: AIO present? your domain cited? competitor cited? fact correct?
Separate “cited as source” from “mentioned in passing.”
Pay for a tool when two things are true: you have more than ~30 prompts, and you need history across engines, not just a gut check this week.
The strategic mistake is tracking citations like rankings. A citation with a wrong price or outdated feature can hurt more than being absent.
Log the cited domains and the answer text, not just whether you appeared. Freeze five buyer questions and rerun them weekly in a fresh chat, since the shortlist jitters between runs. After 30 days the mention rate reads as a trend instead of one lucky snapshot.
IMO most businesses have no idea it exists. I've watched this pattern across different accounts... impressions hold steady or even climb, clicks quietly drop, and nobody can explain why because the citation data isn't sitting where you'd expect to find it.
The naive scraper problem described lines up with what I'd guess is happening. Google isn't hiding this maliciously, it's just not built to be easy to audit from the outside, and a lot of tooling that claims to track this treats it like a simple scrape when it clearly isn't one.
Most businesses I talk to are flying blind on whether they're being cited at all, let alone how often or by whom. Anyone building a real way to check that gap is doing something useful. Curious what your accuracy looks like once you account for the token issue specifically.
The deferred-token detail is what separates this from the many "AI Overview checkers" that quietly log a miss as "no overview", and I'd keep leading with it over the pitch. The failure mode I'd guard against next is sampling variance: overviews are regenerated per request and shift with locale, session and time of day, so a single weekly check per query means some of the movement you're recording is noise rather than a real change in who Google decided to trust. Running each query a few times inside one run and reporting citation frequency (cited in 2 of 3 checks) instead of a binary flag would make the history much more defensible, and it hands you a volatility score, which is arguably more useful than any single snapshot. The other join worth building is query-level Search Console data next to your citation rows, because the decision people actually want to make is not "am I cited" but "does being cited protect my clicks, or does the overview suppress them either way". Since you now use it mostly through MCP rather than a dashboard, do you see the future as an agent-callable tool where history is a side effect, or is the scheduled diffing the part people will actually pay for?
Really useful approach. Instead of guessing whether AI Overviews are hurting traffic, you’re turning it into something measurable and trackable over time. The deferred-token issue is an https://shareincentivesplancalculator.com/ detail too missing those results could easily skew the whole analysis.
The attribution point Jaxin raised has a dirtier cousin: the click side of that ratio isn't clean either.
We run a remote jobs index and watch outbound clicks daily. A couple of weekends ago we logged a 457-click night, which would have been a record, except the clicks-to-sessions ratio came out at exactly 1.0. Humans never do that. It was bots, and it had been quietly inflating the baseline for days. So if clicks are sliding while impressions hold, it's worth confirming the earlier clicks were people in the first place.
The other half your dataset can't see is whether the AI crawlers fetch you at all. That one lives in the server logs, user agent plus status code per URL. Cheap to grep, and it separates being skipped from being outranked as a source.
That's an interesting problem. The impressions staying relatively stable while clicks drop seems like an especially useful signal to track. I can see why checking queries manually wouldn't scale once you have hundreds or thousands of pages.
I think the bigger challenge is separating "AI Overview caused the click loss" from other SERP changes. Having query-level citation data would make that much easier to validate.
This is a real gap - AI Overview citations don't show up in Search Console the way normal impressions do, so you're flying blind exactly where more and more traffic is going. One thing I noticed shipping a new domain recently: Google's AI Overview picked up my site within days while Bing hadn't indexed it at all after weeks, so the asymmetry cuts both ways - you can be cited in the place you can't measure and invisible in the place you can. Does your tool catch the case where the Overview paraphrases your content WITHOUT a visible citation link? That's the one I'd most want to detect, because it's the pure-extraction case where you get zero click credit.
I've been running manual AI visibility checks for indie SaaS founders asking ChatGPT which apps it recommends for a given category and mapping where each product is missing. The deferred-token problem you describe is something I noticed too when checking Google's AI Overview manually: the same query can return different results minutes apart, which makes spot-checking almost meaningless. What I found consistent though: if a product doesn't appear in Reddit threads and comparison articles, it doesn't appear in AI answers regardless of how good the product is. The structured history approach you built sounds like the right direction. Manual checks work for 5-10 products but don't scale.
Useful write-up. The next column I’d want beside the citation data is the normal Search Console position for the same query. If an AI Overview is present, a competitor is cited, and your page is already sitting around position 5-20, that feels like the highest-leverage batch to rewrite first. You’re close enough that Google understands the page, but the cited source is probably answering the question in a cleaner shape.
This is the best concrete suggestion in the thread and I do not know why I was not already doing it. Position 5 to 20 with an overview present and a competitor cited is a rewrite queue that sorts itself. Google already understands the page, so the gap is answer shape, not authority.
supafast asked in this thread whether AI Overview citations track classic top-10 rankings, and your column is how you would actually answer that rather than guessing at it, which is what I have been doing.
Ran into the same problem from a different angle: I kept seeing signups with no referrer and eventually traced some of them to ChatGPT and Perplexity answers. The thing that surprised me most once I started checking systematically was how volatile citations are: the same query can cite you on Monday and drop you on Thursday, so a single check tells you almost nothing.
Are you sampling AI Overviews on a schedule, or is it one-off checks per keyword? And have you noticed whether AIO citations correlate with classic top-10 rankings for the same query? In my experience they overlap way less than I expected.
Scheduled, though I got there late. Individual checks are what I started with and they told me almost nothing for the reason you name: the same query moves week to week, so one reading is noise.
Now it runs on a schedule and the history is the thing I actually read.
On the correlation with classic top-10 rankings, I do not have a clean answer and I am not going to invent one. My informal impression matches yours, that they overlap less than you would expect, but I have not joined the two datasets
properly. TomDigitalLab made the same point in this thread and suggested putting Search Console position next to the citation data, which is the obvious way to actually settle it. That is now on my list.
The no-referrer signups tracing back to ChatGPT and Perplexity is a good catch. @TCLeon511 made a related point about the referrer being the more trustworthy of the two signals.
Disclosure: LeadGrid is mine - B2B lead lists - and being cited by AI is the only acquisition channel that has actually worked for me, so I've been on the other end of this problem.
One column worth adding to the dataset: the referrer. Chats and overviews that link out send traffic with the source attached, and it's already sitting in everyone's analytics for nothing. Citation tells you the machine saw you. The referrer tells you a person moved. Those two don't line up as neatly as you'd expect, and when they disagree the referrer is the one I believe.
On the llms.txt comment above - I have one, and I can't point to anything it changed. What visibly moved for me was page shape: the question as an H2 with the answer in the first two sentences under it, and FAQPage/HowTo markup so the answer is machine-readable rather than inferred.
Also worth logging which of your pages gets cited, not just whether your domain does. Mine picks winners I'd never have guessed.
Any ideas on how we should change our approach here?
The referrer column is the one I did not have and should. You are right that they measure different things: citation says the machine saw you, referrer says a person moved, and when they disagree I would believe the referrer too.
On logging which page gets cited rather than just the domain: the Actor already returns each reference with its title, link and source, so per-page is in the data and I have just been collapsing it to domain level when I read it. That is my reporting being lazy, not a gap in the output. Going to stop doing that, especially given you say yours picks winners you would not have guessed.
Your llms.txt data point is worth more than the theory going around, since you actually have one and can see it changed nothing. Page shape and machine-readable markup matching your experience lines up with what I see too.
Nice build, I will surely use this. GEO and AI ranking will become more and more important. For the last months I run multiple AI pipelines that make all kind of articles.
Not just 'make an article' but 7 or more step pipelines. Measuring their usefullness and AI ranking is a *itch. Even with kinetixseo.com at my disposal it is hard.
Your API will come in handy, it is a genuine good and cheap way to check ranking. I pay way more per check.
The deferred-token paragraph is the most useful part of this, and I want to underline why. I build page-checking tools, and I hit the same class of bug last week: the fetch layer deduplicates repeat visits to a URL, and a repeat visit comes back as an empty stub pointing at the previous copy. My code read the stub as "this page has no content." That is a false negative wearing the costume of a truthful zero - it passed launch testing, because during launch every URL was a first visit, and then broke silently the first morning anyone re-checked a page.
The general shape, which your token quirk matches exactly: in measurement tools the dangerous failures are not errors, they are plausible zeros. An error gets noticed. "No overview," "no content," "not cited" all read as answers. The fix that held for me is the one you landed on - resolve the missing thing inside the same run, and treat every empty result as a claim that needs evidence rather than a fact. Your line about a check that quietly under-reports is the whole problem in six words, and "cited less than I thought" is only trustworthy because you did that work.
"Plausible zeros" is a better name than anything I had for it, and the dedupe-stub bug you describe is the same failure wearing different clothes. Both pass launch testing for the same reason: on the first run every case is a first case.
The thing that makes it nasty in this domain specifically is that "no overview" is a legitimate result maybe a third of the time, so you cannot treat the empty case as an error and alert on it. It has to carry evidence that it was actually checked.
I keep used_dedicated_api and retrievals_used on every row for exactly that reason, though after reading your comment and evanharland's I do not think that is enough metadata yet.
Turning each check into a row instead of a screenshot is the key insight here. In my own testing across ChatGPT, Perplexity, and Gemini, I’ve found that a brand can be mentioned without actually being recommended, and small changes in prompt wording can change the competitive set.
That made it useful to separate mentions, recommendations, citations, and the raw answer rather than treating visibility as a simple yes/no result. How are you handling query volatility over time — fixed queries, or multiple variations for each search intent?
Fixed queries today, and I think that is a weakness rather than a decision. The Actor takes a list and returns one row per query, so variations per intent works fine, I have just not been disciplined about generating them.
Your mentions / recommendations / citations / raw answer split is sharper than what I store. I keep the raw answer blocks and the cited sources, which lets you derive citation cleanly, but mentioned-without-being-recommended is genuinely invisible in what I collect today. That distinction matters most for exactly the commercial queries people care about.
The coverage caveat is part of the product thesis, not a footnote: “no overview” can be a legitimate result, but it can also be a measurement failure. I’d store run metadata that makes those cases distinguishable — country, language, device, latency, deferred-token resolution, and provider errors — then trend three states separately: no overview, overview with you cited, and overview with a competitor cited. That protects the weekly history from turning a flaky retrieval into a false SEO conclusion. The two-and-a-half-cent run is cheap compared with two hours of manual checking, but retention will come from turning snapshots into experiments. How are you detecting a missing overview caused by coverage or retrieval failure versus a genuine no-overview query?
Answering the direct question: partially, and not well enough. Every row carries ai_overview_present, a note when nothing came back, used_dedicated_api and retrievals_used, plus the gl, hl and location the run used. So a deferred-then-resolved query is distinguishable from a genuine no-overview query. What is not distinguishable today is a provider-side failure from a real absence, because both land as ai_overview_present: false with a note.
Your three-state trend is the right shape and I am going to steal it: no overview, overview with you cited, overview with a competitor cited. Those are three different business situations and I have been collapsing the last two.
The stronger paid unit may be a monitored intent set rather than an individual retrieval. Group queries by the decision they represent, assign each set a commercial value, and alert only when a source is gained, lost, or replaced. The useful report is then: what changed, which competitor replaced you, which page was cited, and what content or evidence differs. That turns citation tracking from an interesting dataset into a weekly operating decision.
Worth separating two things you're measuring as one. Ahrefs' February 2026 study found the top-ranking result takes a 58% lower click-through rate when an AI Overview sits above it - and that hit lands whether or not the overview cites you. So a citation trendline can sit flat and green while the clicks keep falling.
The row I'd want beside 'was I cited' is 'did an overview appear on this query', because that's the one that predicts the traffic. Being cited is the consolation prize rather than the fix.
On the ratio that bugs you: two months guessing against a week building is the normal split. The delay is almost never the tooling, it's admitting the trend was real.
You are right and it is the thing I got wrong in the post. I was treating "was I cited" as the metric when the one that predicts traffic is "did an overview appear on this query at all." Those are different columns and I only reported one.
I checked your number because I wanted to use it: Ahrefs, December 2025 data across 300,000 keywords, 58% CTR reduction for the top-ranking result when an overview is present, up from 34.5% in their April 2025 study. So the hit lands whether or not you get the consolation prize.
The Actor already returns ai_overview_present per query, so the data is sitting there and the reporting is what is missing. That is the next thing I change.
The impression/click divergence is the right signal but the harder question is what you actually do once you know whether you're cited or not.
If you're cited and still getting no click-through, the problem isn't visibility — it's that the AI Overview is satisfying the query so completely there's no reason to go further. The citation becomes almost worse than not being cited at all, because it means the content is doing work for Google, not for you.
If you're not cited, you know the optimisation target. But it changes with every update, so this needs to be something you run regularly, not a one-time diagnostic.
The question I'd add: not just "is my site cited" but "who IS being cited for this query." The site that's winning the citation slot is showing you exactly what content type and format Google's AI currently trusts. That turns the tool from a diagnostic ("why isn't this working") into a roadmap ("what are they doing that I'm not").
Are you tracking competitor citation patterns in the current version?
Yes, and it is the part I undersold in the post. Every row returns the full references array, so you get every cited source with its title, link and domain, not just a yes/no on your own site. There is a published task that does nothing but tally cited domains across a keyword list for that reason.
Your framing is better than mine though. I have been treating competitor citations as a column in a diagnostic; you are describing it as the roadmap, where the cited source shows you the content shape Google's AI currently trusts. That is a different product even though it is the same data.
On your first point, cited-but-no-clicks being worse than not cited: I think that is right and it is uncomfortable, because it means the citation metric I built can go up while the business goes down.
The detail that stands out most to me isn't the deferred-token quirk (though that's a great catch) — it's that you built this as an API/dashboard tool first and then found yourself living entirely inside the MCP interface instead. That's a pretty significant admission buried in one paragraph: the UI you'd normally spend the most time polishing turned out to be the part you don't use.
It makes sense in hindsight — "is my domain cited in these ten queries" is a question, not a workflow, and a chat interface that can just answer it and remember context between asks is a better fit than a dashboard you have to navigate to. But it raises a question for how you build the next thing: if the MCP tool is where the actual usage lives, does the dashboard/API layer become mostly a compliance/auditability layer for people who don't have an agent in the loop yet, or is there a real segment of your 162 users who prefer the UI version and would be annoyed if you deprioritized it?
Also curious whether the scheduled-history piece (the "what moved since last week" framing) is something the agent surfaces proactively on its own cadence, or only when you ask — because that distinction seems like the difference between a tool and something closer to a standing monitor.
You caught the thing I only half noticed myself. I built the API first and assumed the interface work would be a dashboard, and then I never built the dashboard because the MCP tool answered the question well enough that I stopped wanting one.
Your framing of it as a question rather than a workflow is exactly why. "Is my domain cited in these ten queries" has no steps. It is one question with a context I want remembered between asks, which is a chat interface, not a screen.
On the segment question, I do not have a good answer yet. The API gets used by people I never talk to and I cannot see whether they would have preferred a UI.
On proactive versus on-ask: today it is on-ask, and the scheduled runs just accumulate history I go look at. Your distinction between a tool and a standing monitor is the right one and I am on the wrong side of it.
the "I am on the wrong side of it" answer is honest, and I think it's actually the more interesting position than if you'd already built the proactive version. it means the accumulated history is sitting there, valuable, and just... not surfaced yet.
the thing I'd push on: going from on-ask to proactive isn't just a technical toggle, it changes what the tool is promising. right now if someone doesn't ask, nothing happens, no expectation is broken. the moment it starts surfacing things on its own cadence, you're implicitly promising "I'll tell you when something matters" — which means you now have to define what's worth interrupting someone for versus what just quietly gets logged. that's a harder design problem than the on-ask version, closer to what I'm dealing with confirmation-wise on the agent side (when does the system decide something's worth surfacing vs. handling silently).
would you default the proactive version to weekly digest, or something closer to threshold-based ("only tell me if citations dropped/appeared")? feels like digest is safer to ship first but threshold-based is the actual "standing monitor" you're describing.
Nice build — “eyeballing doesn’t turn into analysis” is exactly the problem. We hit the same blind spot: GA4 lumped everything under “google”, so we split ChatGPT, Perplexity and AI Overviews into separate sources, and separated crawler bots from real AI-driven visits. Once you do, the numbers finally mean something — it’s the gap that pushed us to build https://amami.dev.
The citation visibility gap is a real problem — even when you're being sourced, there's no native way to know without building something like this.
Running a review site with 100+ tool tests, I've found citation rate matters more than ranking position for AI-generated answers. A page at #3 that gets cited in Overviews drives more qualified traffic than a #1 that doesn't. The next frontier isn't just ranking — it's understanding which content structure gets pulled into AI responses, and why. This tool fills exactly that blind spot.
Search Console still folds AI Overview impressions and clicks into the regular web totals instead of breaking them out which is exactly why "impressions flat, clicks down" is as far as Google's own data will ever take you !Worth logging country and locale on every row though, since overviews aren't deterministic and Apify actors go out from datacenter IPs, so a citation that disappears next week might just be a different exit node. Does the actor let you pin region per run?
Yep, region is pinnable per run. Every run takes
glfor country, hl for language and an optional location, and all three come back on each row so you can see what a given result was actually pinned to.Your underlying point stands regardless: if you do not log those per row you cannot tell a real citation loss from a different exit node, and you will go rewrite a page that was fine. That is worth saying louder than I said it.
And you are right about Search Console folding AI Overview impressions into the web totals. That is precisely why "impressions flat, clicks down" is where Google's own data runs out and you have to go get the citation side yourself.
The structured history is more valuable than the one-time citation check. I would prioritize queries by business impact: high impressions, declining clicks, and pages tied to conversion. Then compare citation share against competitors over time and test whether content changes move it. That turns the tool from monitoring into a decision system for what to update next.
The measurement gap you describe is the one I hit hardest. I ran an automated content pipeline for two weeks — 18 GEO-optimized articles, daily publishing, technically correct setup — and zero AI citations. Or so I thought. My verification method was incognito spot-checks on 3-4 queries a week, which is statistically meaningless at scale.
The deferred-token behavior is the part I would have completely missed. A spot check that silently fails because the token expired looks identical to "no overview shown" in a manual audit. At scale that becomes systematic undercounting — you think you're absent from AI Overviews, but you might just be measuring wrong.
The ratio you close with stays with me: two months worried, one week building the measurement. That's the exact ratio I ran, except I never built the measurement at all. I optimized for six weeks before realizing I had no reliable way to verify the optimization was doing anything.
The MCP integration is the right architecture for this. Citation monitoring belongs in the same automated layer as broken-link checking — something that fires after publish and logs results, not a quarterly manual audit you remember to run.
Eighteen articles against three or four incognito spot-checks a week is the version of this that stings, because the optimisation was probably fine and you had no way to see it.
The part worth flagging for anyone reading: your spot-check failure mode and the deferred-token one compound. A check that silently fails looks identical to a real absence, so the error is not random, it is systematic undercounting in one direction.
You do not get noisy results, you get confidently wrong ones.
Agreed on where it belongs. Citation monitoring sits with the checks that fire after publish, not with the audits you mean to run quarterly.