To prove a fix worked, I first needed the day it went live
Under my last post: https://www.indiehackers.com/post/i-built-an-ai-visibility-tracker-then-stopped-using-it-myself-f981061340, seven people asked me some version of the same question: the assistants drift on their own, so when I re-run the same buyer questions, how will I know it was the change that moved them?
Half the answer is a control set. Read the questions a change was aimed at apart from the ones nothing touched. If both move, that's the assistants, not the change.
The other half took me longer to see: the day the change went live.
A re-test the day after a page goes up tells you nothing. The assistants probably haven't found it yet. A month later, "nothing moved" means something. Same result, opposite reading, and the only difference is a date.
When Passcite publishes a page, the date is there: an agency approves a draft and it goes into the client's WordPress. But it only publishes new pages. A new page can't break anything that's already there, and an edit can. So every edit, and every page on a site we can't publish to, goes to whoever runs the site.
That's where the date gets lost. The change goes out in an email, the client says "will do", and nobody learns when it happened, or whether it did. Not the agency, not me, and not the re-test.
So instead of an email, the agency sends the site owner a link. One per site, no account needed, and it can carry the agency's name instead of mine. For each change it has:
- Where it goes, and if it changes a page they already have, a preview of that page with the new section in place, outlined.
- Steps for whatever the site runs on. If the site is built in code, a line saying this one is for whoever maintains it, since the person holding the link may never have opened the code.
- The text, ready to paste.
And one button: "It is live — mark as done." Nothing else on the page changes anything, and that click becomes the change's date.
Off-site items, like claiming a directory profile, sit in the report as a checklist. Ticking one counts the same way.
A week after the first change, the same questions get asked again. The report lists each change with how long it had been live and which of its questions now name the client. The week is a guess, which is why the days are printed: "nothing moved after two days" shouldn't read as "nothing works".
None of these re-tests has run yet. Whether a change like this makes an assistant change its answer is still what my product rests on, and it's still unproven.
One question for each side again:
If you run an agency: when you hand a client something to do on their own site, how do you find out it's done?
If you've built something that measures a change someone else makes: where does your "after" date come from?
The free report needs no signup: https://passcite.com/free-report
I'd make the witness evidence the page itself, not only the date someone reports. Store the intended change, the first observed public version, and the retrieval time separately. Then freeze the question set and compare repeated runs against an untouched control. That keeps mention, citation, and recommendation from being collapsed into one noisy “AI moved” score.
That's where it ended up after praneetbrar's comment above. The text of each change is kept when it's handed over, and the date is the first time a check finds that text in the public HTML, not the click. Who an assistant recommended and who it only cited are counted separately too, so there's no single "AI moved" number.
Dating it from the first time the text shows up in the public HTML is the right anchor. That's the moment the evidence actually exists for anything reading the page, not when someone pressed publish.
The "same result, opposite reading" line is the useful part. I've burned cycles calling a change a miss when the only real miss was the clock.
What helped me was treating the go-live date as part of the test protocol, not a footnote. For anything I expect assistants (or search, or a competitor watcher) to pick up, I write down: what changed, the exact publish timestamp, the earliest re-check date, and a control set that should not move. Then I refuse to score the experiment before that window.
Curious how you're encoding that date today — just a field on the page record, or something that blocks the re-test UI until the window opens?
A field on each change, filled in by the check, not by the click. There's no re-test button to hold back either: it's scheduled a week after the first change goes live and runs on its own, so the agency can call it off but can't score it early.
Once agencies start using the new workflow, what behavior would show the implementation date is solving the measurement gap—more completed changes, cleaner re-tests, or greater willingness to run the analysis repeatedly?
Fewer changes in a re-test with no date on them. Whether agencies keep running re-tests is the real signal, but that's a couple of months out.
That re-test behavior is probably the stronger signal once the workflow has enough history. If you’re open to it, what’s the best email to reach you on?
Sure — what did you have in mind? Easiest is the contact on passcite.com.
Thanks! I’ve just sent it over.
Looking forward to hearing your thoughts whenever you have a chance.
On the after date, I wouldn't trust the button by itself. I've had people mark a page live while the old version was still cached, or click it meaning "I'll do it tonight."
What I do on my own changes is dumber. I keep the sentence I'm waiting to see, and a small check hits the public URL until that sentence is actually in the HTML. First time it shows up is the date I write down. The button is a nudge. The date should come from the page, not from the person.
I'd still wait longer than feels reasonable before calling a re-test a failure. Two days after a page change, "nothing moved" has mostly meant it hasn't been picked up yet, not that the change was useless.
You're right. A click can mean "done" or "doing it tonight", and nothing checks which. I'm changing it so the button only starts a check, and the date is the first time the new text actually shows up in the page's HTML.