How I Reverse-Engineered HowLongToBeat for My Steam App (Without Getting Banned & Scaling Cheaply)

I recently wanted to add a simple but highly requested feature to my Steam storefront app, steamAF: showing HowLongToBeat (HLTB) estimates right on the game detail cards.

The goal was simple: show Main Story, Main + Extras, and Completionist times so users can decide if a game fits their backlog schedule.

The problem? HLTB does not have a public API, and they actively try to break scrapers.

Here is a full breakdown of how I built a robust, scalable, and cheap integration from the client click down to the database cache, ensuring my app doesn't break even if HLTB changes their internal endpoints.

1. Bypassing HLTB’s Anti-Scraping Defenses

This was the hardest part. HLTB’s internal search endpoint is deliberately designed to frustrate scraping.

Rotating Path Segments: The endpoint path changes periodically (search → seek → ouch → s → bleed).

Per-Visit Tokens & Honeypots: You can't just POST a search query. You first have to hit an /api/<segment>/init endpoint to fetch a token and a honeypot key/value pair (hpKey / hpVal).

The Solution: I built a two-step fetcher. First, GET /api/bleed/init to grab the token and honeypots. Then, POST /api/bleed passing the token in the headers (x-auth-token, x-hp-key, x-hp-val) and injecting the honeypot into the JSON body.

Note on maintenance: I hardcoded the current segment (bleed) as a constant. The ceiling here is that when HLTB rotates the segment again, my fetch will 404. But because I built the UI to degrade gracefully, the playtime row simply won't render. No errors, no broken UI. To fix it, I just grep their _next JS chunk for the new segment and update one line of code.

2. Fuzzy Matching the Right Game

HLTB returns an array of candidate rows. Picking the right one isn't straightforward because Steam titles and HLTB titles don't always match perfectly, and HLTB’s search chokes on weird punctuation (searching "Assassin's Creed" with a curly apostrophe returns 0 results).

The Solution: I sanitize the search query by stripping weird punctuation and sending only alphanumeric word tokens. Then, I run a 3-tier matching strategy (inspired by the open-source hltb-for-deck plugin):

Exact Steam AppID match: Checking profile_steam (often null these days, but great when it hits).

Exact Normalized Name match: Stripping all symbols and comparing lowercase strings.

Fuzzy Match (Levenshtein distance): If exact matches fail, I calculate string similarity. If multiple games have the same distance, I tie-break using popularity (comp_all_count).

I added a minimum similarity threshold of 0.5 to prevent false positives (e.g., searching "Hades" and accidentally showing the playtime for "Saint Seiya: The Hades"). If it doesn't meet the threshold, I treat it as "no entry exists."

3. Scaling Cheaply: Caching & Rate Limiting

If 10 users click on Cyberpunk 2077 at the same time, I cannot send 10 requests to HLTB. I would get rate-limited instantly, and it's terrible for performance.

To make this practically free and instantly fast, I put everything behind a Supabase Read-Through Cache acting as a proxy layer.

Massive TTL: Game completion times rarely change. I set the cache TTL to 30 days.

Single-Flight Lease (RPC): When a cache is stale or missing, the first request claims a "lease" in the database. Any concurrent requests for the same game will just see the stale data or get a "pending" response. This completely eliminates cache stampedes.

Global Token Bucket: All outbound requests to HLTB pass through a global token bucket, ensuring the entire app stays under a strict request limit per minute.

Caching "Null": If a game genuinely doesn't exist on HLTB, I cache that null result too. This prevents the server from repeatedly querying HLTB for an obscure indie game that will never yield a result.

4. The Client Side (React)

The API route itself is a thin proxy. On the frontend (Next.js), when a game detail card mounts, a useEffect fires the fetch.

The state is tri-state:

undefined: Loading (shows a skeleton UI).

null: No HLTB entry exists (the row hides itself completely).

object: Renders the playtime visually.

To save even more client-side requests, I store the result in a module-level Map in memory. If a user opens a game, closes it, and reopens it in the same tab session, it reads directly from RAM.

Takeaway

By combining a smart 2-step token fetcher, aggressive Levenshtein string matching, and a rock-solid single-flight Supabase cache, you can build reliable features on top of undocumented, hostile endpoints without sacrificing UX or burning money on server costs.

Would love to hear how other indie hackers handle undocumented APIs or deal with cache stampedes!