💸 How we cut the running cost of our AI chatbot (without cutting quality)

1️⃣ Dropped the bigger models we didn't need

We were defaulting some flows to full-size models (gpt-5.4, gpt-5.5) when the task — answering FAQ-style questions from scanned site content — didn't need that much horsepower. Swapped those out for gpt-5.6-luna, OpenAI's new nano-tier model built specifically for high-volume, cost-sensitive workloads. Same job, fraction of the cost per request. 💰

2️⃣ Same move on the Gemini side

Moved to gemini-3.5-flash and gemini-3.1-flash-lite instead of the older/discontinued previews. Flash-tier models are consistently the sweet spot for chatbot Q&A — you're not doing complex reasoning, you're doing retrieval-augmented answers, so paying for a frontier model is usually wasted spend. ⚡

3️⃣ Trimmed the Mistral lineup to one model

We used to expose several Mistral variants. Cut it down to just mistral-large-3. Fewer options isn't just simpler for users — it's fewer configs to test and support, which matters a lot when you're a team of one. ✂️

4️⃣ Added Anthropic as a genuinely cheaper alternative

claude-haiku-4-5 is now available as a provider option — for a lot of chatbot use cases it's a strong quality-per-dollar pick, so users who were maybe overpaying elsewhere now have another lever to pull. 🎉

🧠 The bigger lesson

The real cost lever isn't picking "the best model" — it's matching model tier to task. A chatbot answering "what are your opening hours" doesn't need the same model as one doing multi-step reasoning. Most of our savings came from routing the boring 90% of queries to the cheapest capable model, not from any one provider being magically cheaper.

🙋 Curious if other indie hackers running AI features have found other levers — caching, prompt shortening, self-hosted models for narrow tasks? What's worked for you?