I did 5 things at once and 800 pages got indexed in a week. I have no idea which one actually caused it.
This is not a clean case study. I want to say that upfront.
A clean case study would be: "I changed one thing, here's exactly what happened." This is not that. This is: "I changed five things at roughly the same time, something worked, and now I'm trying to reverse-engineer which one it was."
Background: I'm building YourAIFinder — an AI tools directory — on evenings and weekends while working full time in Mumbai. At the end of May, I had around 400 pages indexed after nearly 6 weeks live. Decent, but slow. The other 1100+ pages were sitting in "Discovered — currently not indexed" limbo.
Then around June 10–15, something happened.
That bar chart went almost vertical. In roughly one week, Google indexed around 800 additional pages. I went from 400 indexed to 1,500 indexed. It has held since then.
Around that same time, here's everything I did — in no particular order because I was doing all of it simultaneously like a person with no scientific method:
-
Fixed internal linking
I had hundreds of pages with zero inbound internal links — completely orphaned. Pages Google had found via sitemap but had no reason to prioritise since nothing on the site pointed to them. I did a sweep and made sure every page had at least 2–3 internal links pointing to it from relevant category and task pages. -
Resubmitted my sitemap
Nothing fancy. Just went into GSC, removed the old sitemap entry, resubmitted fresh. Sometimes the simplest things. -
Published a batch of new content
Around 8–10 new blog posts and comparison pages went live in that window. My theory: new content being actively crawled might have pulled Googlebot deeper into the site, which then triggered re-evaluation of the orphaned pages nearby. -
Backlinks from DA 79–96 domains went live
Around that time, my IndieHackers product page, GitHub profile, and a Dev.to article all went live with links back to youraifinder.com. None of these are editorial links — they're profile and contextual links. But they were the first real external signals the domain had ever received. -
Manually requested indexing for my top 20 priority pages
I went through GSC one by one and hit "Request Indexing" on the pages I most wanted indexed. This is supposed to queue them for faster crawling — I genuinely don't know if it cascaded to nearby pages or not.
All five of these happened within roughly the same 5–7 day window.
What happened after the spike:
- Impressions: 9K → 18K
- Clicks: 6 → 117 (real humans, not me)
- Average position: 34.4 → 30.3
- Pages still not indexed: 265 — same domain, same site structure, just... not yet
The honest answer to "what caused it":
I think it was the internal linking fix combined with the new backlinks arriving at the same time. My working theory: external links gave Google a reason to allocate more crawl budget to the domain, and the internal linking fix gave Googlebot somewhere useful to go once it arrived. The new content and sitemap resubmission probably helped too, but I think those were supporting factors rather than the trigger.
I could be completely wrong. There's no control group here.
What I still don't understand:
Why those 1,500 pages and not the other 265? They're all on the same domain, submitted in the same sitemap, with similar internal linking. If anyone has a theory for what separates "indexed" from "discovered but not indexed" on a site that's already past the initial crawl — I'd genuinely love to hear it.
Has anyone seen a similar pattern — a delayed indexing spike on a new domain after a specific combination of changes? What was your read on what caused it?
800 pages in a week is impressive. The tricky part is figuring out why they got indexed. A proper SEO audit can help identify whether technical fixes, internal linking, sitemap changes, or content improvements made the difference. That’s exactly the kind of analysis I built into SERPSpur.
Thanks! Agreed that the attribution is the hard part — five changes in one window and no control group.
If you've seen this pattern across other sites, I'd be curious what you'd expect to find. Specifically: what separates the pages that got indexed from the 265 that didn't, when they're on the same domain, same sitemap, same template, and similar internal linking? That's the bit I can't crack.
ok adin builds
Honest writeup and the reverse engineering instinct is exactly right even without a control group.
My read on what actually triggered it: the internal linking fix was the primary cause and the backlinks were the accelerant. Here is why I think that.
Google does not just discover pages, it has to justify the crawl budget spend on them. Orphaned pages sitting in "Discovered, currently not indexed" are essentially pages Google found but deprioritised because nothing on the site was vouching for them internally. When you fixed the internal linking, you gave Googlebot a signal that the site architecture considered those pages important enough to reference. The backlinks arriving simultaneously gave Google a reason to allocate more crawl budget to the domain at exactly the moment the internal structure made that budget worth spending.
The timing gap between your fixes and the spike also fits this pattern. It was not instant because Google had to recrawl the pages that now had internal links pointing at them, reassess their priority, and then action the queued indexing. That process typically takes five to ten days on a newer domain, which matches your timeline almost exactly.
On your remaining 265 unindexed pages: the most common separator between indexed and stuck in discovery on a directory site is thin content differentiation. If those 265 pages look structurally similar to each other and to already indexed pages, Google may be holding them back while it decides whether they add distinct value. Worth checking whether the unindexed ones have genuinely unique content or whether they follow a template that produces near identical pages at scale.
Impressive jump for a side project built on evenings and weekends.
This is the most useful reply I've gotten on this, thank you.
The framing that fixed something for me: I was treating internal links and backlinks as competing explanations when they're not competing at all. Backlinks bought the crawl budget, internal linking made it worth spending. That also explains the dead six weeks before — I had a sitemap full of pages and nothing on the site vouching for any of them.
And your thin-content read on the 265 looks right. I checked after reading this and almost all of them are programmatically generated comparison pages — same template, differentiating content mostly pulled from structured fields rather than written. The ones that did get indexed skew heavily toward category and task pages that have actual prose on them.
So the question isn't really why Google won't index them. It's whether they currently deserve it, and the honest answer is probably not yet. Useful thing to find out before I spent another month building more of them.
This comment was deleted 2 months ago