2
1 Comment

Tried answering my own week-to-week noise question, landed on a 2-week rule

Yesterday's open question was how to tell a real slow decline from normal week-to-week noise, without waiting so long the run is already lost.

What I landed on: instead of comparing this week to last week, require 2 consecutive weeks below a rolling variance band (trailing 8-week mean minus about 1.5x stdev) before the gate calls it a real decline instead of one bad week.

Reran the same 6 pages from the CTR-delta test. It now catches the slow bleeder that day 11 missed, and doesn't add any new false alarms on the 4 thin pages or the 2 winners.

Caveat: I only have two examples of "how many weeks is enough": 1 week was false-alarm-y, 2 weeks works so far on a sample of one page. If this is already a solved problem in stats, I'd rather use the real name for it than keep reinventing thresholds.

multisiteseo.com if you want to see where this lives.

on July 8, 2026
  1. 1

    What stood out to me is that you're questioning whether the evidence has actually earned the rule, not just whether the rule fits the current data.

    A lot of products end up shipping thresholds that explain yesterday's examples but don't generalize. Treating this as a decision-confidence problem instead of just a tuning problem feels like the more durable approach.