My RAG pipeline was technically solid. My customer discovery was zero. Guess which one mattered.

Three weeks. That's how long it took to go from "journalists need better research tooling" to a deployed, self-hosted RAG pipeline with hybrid retrieval, async ingestion, and a tiered API.

It worked. It was overbuilt. Both things were true.

Two journalists actually used it. One gave feedback. That feedback taught me more than three weeks of building. Here is what I got wrong.

1. Hybrid retrieval is great. Knowing when to use it is better.

Atlas combines pgvector for semantic search, PostgreSQL FTS for keyword matching, and Reciprocal Rank Fusion to merge results. The RRF math looks deceptively clean:

def reciprocal_rank_fusion(rankings: list[list[str]], k: int = 60) -> dict[str, float]:
    scores = {}
    for ranked_list in rankings:
        for rank, doc_id in enumerate(ranked_list):
            scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank + 1)
    return dict(sorted(scores.items(), key=lambda x: x[1], reverse=True))

Looks smart. Feels smart. Is only as smart as the queries you test it against.

I tuned k against synthetic queries I invented myself. Reader, that is not tuning. That is guessing with a math costume on.

What I would do instead: ship vector-only search first. Log every real query that comes in. Add RRF when you can actually see what it's failing at. You will save yourself three days of tweaking a constant against vibes.

2. Celery was 80% overkill and 100% my fault.

Atlas ingests RSS feeds via Celery workers, Redis broker, retry logic, dead letter queues — the full distributed task queue experience:

@celery_app.task(bind=True, max_retries=3, default_retry_delay=60)
def ingest_feed(self, feed_url: str):
    try:
        entries = parse_feed(feed_url)
        for entry in entries:
            process_and_embed(entry)
    except Exception as exc:
        raise self.retry(exc=exc)

This is fine code. It is also three Docker services, a Redis memory budget, and worker state I had to debug at 11pm on a Tuesday.

What I actually needed:

scheduler = BackgroundScheduler()
scheduler.add_job(ingest_all_feeds, 'interval', minutes=30)
scheduler.start()

That's it. APScheduler, a Postgres queue table, done. 200 lines instead of 600. Zero Redis. I would have shipped the actual features a week earlier.
Celery is great when you have scale problems. I had a laptop and a dream.

3. I built tiered API access before I had users. Peak founder delusion.

Week 2. Zero users. I built this:

TIER_LIMITS = {
    "free":       {"requests_per_day": 50,   "results_per_query": 5},
    "pro":        {"requests_per_day": 500,  "results_per_query": 20},
    "enterprise": {"requests_per_day": 5000, "results_per_query": 100},
}

async def check_rate_limit(api_key: str, tier: str):
    usage = await get_today_usage(api_key)
    limit = TIER_LIMITS[tier]["requests_per_day"]
    if usage >= limit:
        raise HTTPException(status_code=429, detail="Limit exceeded")

Clean implementation. Genuinely pointless at the time.

It felt like building a business. It was actually avoiding the harder question: does anyone want this? A single flat API key with a # TODO: tiers later comment would have been completely fine. Ship the core. Find one person who finds it useful. Then figure out how to charge them.

4. "Self-hosted" is simultaneously a feature and a support ticket.

Privacy-conscious newsrooms love self-hosted. They also need a 45-minute Docker Compose walkthrough before they can try it:

services:
  api:
    build: .
  postgres:
    image: pgvector/pgvector:pg15
  redis:
    image: redis:7-alpine
  celery_worker:
    build: .
    command: celery -A app.worker worker
  celery_beat:
    build: .
    command: celery -A app.worker beat

Five services. Every journalist I wanted feedback from had to set this up first. I was gatekeeping my own user research with a YAML file.

A hosted version with a strong data policy would've given me 10x more conversations in the same time. Learn first. Self-host later.

5. The actual mistake: I built before I talked.

Week 1 was: get pgvector running, wire retrieval, get results back. Reasonable.

It should have been: call three journalists and ask how they actually research stories.

Because here is what I found out after shipping — journalists don't want a search box. They want signals pushed to them. "Monitor and surface" not "search and retrieve." That's a fundamentally different product shape, and no amount of clever RRF tuning fixes a wrong mental model.

The retrieval pipeline was solid. The customer discovery was zero. These have very different consequences.

Atlas is open source at github.com/PreethaRaj. If you are building RAG pipelines or newsroom tooling, I would like to compare notes.

What's the worst thing you have overbuilt before talking to a single user?