1
0 Comments

What Do You Do When a Website Doesn't Have a Sitemap?

I was doing a competitor audit recently and ran into something surprisingly common: websites with no obvious sitemap.

No sitemap in robots.txt, no standard /sitemap.xml, and no mention of one anywhere in the source code.

For small sites it's not a huge deal, but when you're auditing dozens of domains, manually hunting for sitemaps becomes a time sink.

I've noticed a few different scenarios:

Sites that forgot to generate a sitemap
CMS installations with non-standard sitemap URLs
Dynamically generated sitemaps
Sites that intentionally hide them
Large websites with multiple sitemap indexes

I'm curious how everyone handles this.

When you can't find a sitemap immediately, what's your process?

Check robots.txt first?
Crawl the site yourself?
Use search operators?
Look for CMS-specific patterns?

Would love to hear the approaches other SEOs and indie hackers use when a sitemap isn't easy to find.

on June 13, 2026
Trending on Indie Hackers
6 weeks solo, 2 rejections, finally live but nobody told me marketing would be this hard User Avatar 140 comments I spent more time setting up cold email than actually selling. Here is what fixed it. User Avatar 38 comments I just wanted to taste AI coding tools. A week passed. User Avatar 24 comments I built a PDF API because every team I know has a haunted corner of their codebase they never want to open User Avatar 19 comments A pattern I keep seeing in EdTech: traffic isn't usually the problem. User Avatar 17 comments Building LinkCover – Day 3: Payment is live. No more building, time to sell. User Avatar 16 comments