GSC and Bing impressions both dropped to ~0 within a day of each other. Crawling continued, no DNS/hosting changes on our end. Anyone
Small SaaS (owelet.app, on Vercel), steady 80+/day impressions on both Google Search Console and Bing Webmaster Tools until late July. Then both cratered within a day of each other and I can't find a cause I can act on.
Timeline:
July 26: normal, GSC healthy
July 27: GSC drops to single digits, stays there through today (Aug 2). Bing still stable
July 28: Bing drops by about half
July 29-30: Bing hits flat 0 and stays there
We shipped only content and frontend-level changes in that window (calculator copy rewrites, JSON-LD breadcrumbs, footer/internal linking, one line added to a middleware public-route allowlist for an analytics path). Nothing touching DNS, registrar, Vercel settings, or infra-level redirects. I went through every page type with curl and Playwright, including spoofed Googlebot/Bingbot user agents in case of bot-specific blocking: clean 200s everywhere, no noindex, no unexpected redirects, no hydration errors, identical responses for bots vs browsers. The one robots.txt edit happened July 31, four days after the drop started, so it can't be the initial cause.
The interesting part is in GSC's own Crawl Stats:
Crawl request volume never stopped. Still 10-45 requests/day through the whole July 27-30 window, in line with plenty of earlier healthy days
But the Response table shows a nonzero DNS error rate (0.59%) and "unknown (failed requests)" at 0.73%
The Hosts table: owelet.app (essentially all crawl traffic, 6000+ requests) is flagged "Problems in the past." Our www subdomain, which gets almost nothing, shows "No problems"
Security and Manual Actions: clean
And in Bing's Site Explorer export, the homepage's last successful crawl is stuck on July 26, which lines up almost exactly with when GSC started dropping
Working theory: some transient DNS/host-connectivity blip around July 26-27 that both crawlers hit independently, and both engines are now throttling impressions while they reassess host reliability. GSC's own label says "past," not current, and nothing on our end shows an ongoing problem.
What I'm asking:
Has anyone seen this exact combo before: crawling continues normally, impressions crater on Google and Bing simultaneously, host status "problems in the past," small but nonzero DNS error rate, no DNS/hosting changes made?
What actually makes Google's DNS error metric tick up when you didn't touch DNS? Registrar-side, Vercel edge, cert renewal, DNSSEC weirdness?
Once the underlying issue is gone, is there anything beyond Request Indexing and waiting that genuinely speeds up recovery?
What else would you check?
Happy to share crawl stats screenshots or response headers if useful.
Everyone here is debugging the infrastructure, so let me ask the boring question first: what happens when you split those impressions by query and by page? When I pulled my own 90-day GSC export I had 227 clicks, and 216 of them were a single brand query hitting the homepage — the headline number was really one query wearing a trenchcoat. If your 80/day was concentrated anything like that, a drop to near-zero is one query losing position, and every edge-routing theory in this thread is a dead end.
The most useful outcome may be closing the observability gap for the next incident rather than proving the July cause from today’s healthy deployment.
I’d add a few lightweight external canaries from different regions that periodically record:
DNS resolution result and latency
resolved IP
TLS details
status and response headers
body hash for /robots.txt, /sitemap.xml, and one key page
the deployment ID or release version serving the response
One important distinction: spoofing a Googlebot user agent tests your middleware branch, but it does not reproduce Google’s resolver or network path. For requests claiming to be Googlebot or Bingbot, logging them and verifying the source through reverse and forward DNS would give you much stronger evidence.
This will not restore the impressions today, but it means the next drop can be correlated with the exact deployment and edge state instead of reconstructed afterward.
Are you currently retaining enough external evidence to identify what each verified crawler was actually served at a specific time?
The timing with that middleware allowlist change lines up too well to be coincidence, I'd start there before DNS. Bots don't send the same cookies or headers a logged-in browser sends, so a one-line "public route" check that looks harmless in review can quietly short-circuit or redirect requests that don't look like a normal user. Two quick checks: curl a few key URLs with a Googlebot user-agent and diff the status/headers against a normal browser request, and check whether that middleware or anything downstream of it is now setting x-robots-tag noindex on paths it touches. Also worth checking if the new analytics allowlist rule is matched before or after your existing routing, order-of-operations bugs in middleware chains cause exactly this kind of "nothing changed but suddenly it did" symptom.
Yes, this exact pattern happens more often than people realize, especially on Anycast edge platforms like Vercel.
What you're describing is a classic "transient edge availability blip" that triggered crawler back-off algorithms on both engines simultaneously.
What Caused the Drop
Anycast / Edge Routing Blips Vercel routes traffic across global edge nodes. During routine edge routing re-configurations, SSL certificate auto-renewals, or upstream provider maintenance (AWS/Cloudflare), an edge node can briefly drop DNS resolution or fail TCP handshakes for specific geographical bot IPs. While human users on regional networks served by warm edge nodes see 100% uptime, Googlebot and Bingbot hitting different global edge resolvers hit timeouts.
The robots.txt Safeguard Trigger If Googlebot or Bingbot attempts to fetch /robots.txt during a 30-second window when a DNS query or edge connection fails, both crawlers treat the host as unstable. Google's protocol dictates that if /robots.txt cannot be reliably verified due to host errors, it halts or severely throttles indexing and serving to protect user experience.
Why the Metric Showed 0.59% DNS Error Google's DNS error metric ticks up when its distributed resolvers hit a timeout (SERVFAIL, NXDOMAIN, or socket timeout) resolving your apex domain's A or CNAME records. When diagnosing regional DNS propagation or edge node failures across global resolvers whether for an app like yours or a local domain like lexingtonappliancerepairkycom you'll often find that the error occurred at specific edge resolver locations while standard dig commands from your local terminal return clean 200s.
What to Check Now
IPv6 / AAAA Records: Ensure you don't have lingering or conflicting AAAA records in your DNS provider if Vercel is handling your primary routing.
DNS TTL: If your TTL was set high (e.g., 86400 seconds), lowering it to 300–600 seconds ensures crawlers clear stale edge DNS caches faster.
Vercel Deployment Logs: Filter your logs around July 26–27 specifically for 502/504 edge response codes or middleware execution timeouts during bot fetches.
How to Accelerate Recovery
Since the GSC host status now shows "problems in the past," the underlying connection issue is resolved. Now it's a matter of rebuilding crawler confidence:
Force Key Page Re-crawls: Use the GSC URL Inspection tool to request indexing for your homepage, primary landing pages, and key calculator routes.
Resubmit Sitemaps: Re-submit your sitemap.xml in both GSC and Bing Webmaster Tools to trigger a fresh crawl queue.
Submit via Indexing APIs: Use the Bing Indexing API to push your main URLs directly.
Monitor Crawl Stats Daily: Watch the "Response" distribution in Crawl Stats. Once 200 OK responses sit at 100% for 5 to 7 consecutive days, impressions typically rebound sharply as the engines restore host trust scores.
Everything you checked reconstructs the present. curl and Playwright today test the deployment that is live now, which cannot tell you what got served on the 26th.
We had blog routes on Vercel return 404 for about 23 hours while main was green and every local check passed. Nothing on the live site showed it afterwards.
Worth pulling the deployment list and edge logs for July 26-27 rather than probing the site. Vercel keeps per-deployment status and the actual response codes served to Googlebot, which is the only direct record of what the crawler hit. A failed build or an alias pointing at a bad deployment looks exactly like this.
This is the angle I hadn't thought of. You're right that probing the live site today tells me nothing about what Googlebot actually hit on the 26th. Going to pull the Vercel deployment list and edge logs for that window now. The 404-for-23-hours-while-main-was-green scenario is exactly the kind of thing that would be invisible to everything I checked. Will report back what the logs show.
A sudden drop in both GSC and Bing impressions usually points to an indexing, visibility, or technical SEO issue rather than just hosting. I’d check crawl patterns, index status, and SERP changes. Tools like SerpSpur can help audit technical issues and identify potential ranking drops quickly.
Hi,
This exact pattern (simultaneous GSC + Bing impression drop with "Problems in the past" DNS errors on Vercel) is actually something several devs on Vercel/Cloudflare Edge setups encounter. Here is what is likely happening under the hood and what you should check:
What causes the DNS error spike on Vercel without DNS changes?
Vercel uses Anycast Edge routing. During automated SSL cert renewals or regional edge node updates, specific edge nodes can experience transient SERVFAIL or DNS resolution timeouts (>2s) specifically for Googlebot/Bingbot crawler IP ranges (which hit edge nodes concurrently from multiple regions), while regular browser traffic remains unaffected. Even a tiny 0.5%-0.7% failure rate triggers GSC's "Problems in the past" flag.
Why did Google & Bing throttle impressions simultaneously?
Both Googlebot and Bingbot have automated host-health protection mechanisms. When host connectivity or DNS resolution fails even briefly, the search engines temporarily suppress/throttle impressions to prevent overloading a seemingly struggling host.
What else to check:
Hope this helps give you some clarity on the infrastructure side!
This is the most technically specific explanation I've seen for this pattern and it fits precisely. The Anycast edge SSL cert renewal causing transient SERVFAIL for crawler IPs while browser traffic is unaffected explains why everything looked clean on my end. The 0.59% DNS error rate being enough to trigger the "Problems in the past" flag also tracks. Going to check Vercel middleware execution logs for 502/504s during that window and cross-reference with the DNSSEC check. Already submitted the sitemap and requested indexing on core pages. Will stay stable on infra and let the 7-14 day clean crawl window do its thing.
If crawling is still happening normally, I'd first check whether there were any recent changes to robots.txt, meta robots tags, or canonical tags that could affect indexing. Since both Google Search Console and Bing dropped around the same time, it's worth looking for a common cause rather than treating them separately.
Appreciate you jumping in. robots.txt, meta robots, and canonicals were all on my checklist early, nothing changed in that window and GSC shows no coverage issues. The simultaneous Google and Bing drop is what pushed me toward the infrastructure theory since a robots or canonical issue would typically only hit one engine or show up in coverage reports. Still useful to have a second set of eyes confirming those bases are worth checking.
Your working theory sounds spot on. I’ve seen micro-glitches on Vercel/Anycast DNS before where regular users don't notice a thing, but bot crawlers hit a quick wall. Once GSC flags "Problems in the past," it usually takes anywhere from a few days to two weeks of clean crawls for rankings and impressions to bounce back.
Yeah the continued crawling is what makes this harder to read definitively. Few days to two weeks is what I'm expecting at this point. Fingers crossed it's the short end of that range.
The simultaneous Google/Bing drop is the part that makes the infrastructure theory tempting, but continued crawling makes the interpretation less obvious.
What would you need to see over the next few days to distinguish “both engines are temporarily reassessing host reliability” from “the DNS errors are incidental and the visibility loss has a different cause”?
Good question. What I'd need to see: GSC DNS error rate going to 0% consistently over the next 7 days would confirm the underlying issue is gone. If impressions start recovering after that, infrastructure theory confirmed. If DNS errors go to 0% but impressions stay flat, then something else is suppressing visibility and I need to look harder at the ranking side, not the crawling side. That's the test I'm running right now by just waiting and watching the crawl stats daily.