Smaller models are good enough — and that's the whole point

I was using OpenAI for everything. Every AI call in every project went through the same API. It worked, it shipped, nobody complained.

Then I actually looked at what those calls were doing:

• Classify this support message.

• Extract the date from this user input.

• Translate this sentence.

• Decide if this text is spam.

• Summarise this paragraph into one line.

90% of my calls were tasks like that. Narrow, repetitive, well-defined. The kind of task where the expected output is small and predictable every single time.

I was using one of the most computationally expensive systems ever built to do the AI equivalent of checking if a light switch is on or off.

We've been here before

This is not a new pattern. The industry has a long history of defaulting to the most powerful tool available until someone points out it's unnecessary.

1. VMs to Containers: We used to spin up full virtual machines for every workload. Then containers showed us most of those workloads didn't need a full OS underneath them.

2. Containers to Serverless: Then serverless showed us most of those containers didn't need to run 24/7 for a function that fires twice a day.

Every time, the insight was the same: the tool was right for hard problems. It was just wrong as the default for everything.

Frontier models are extraordinary. They can reason, write, code, and analyse. That's genuinely impressive and there are tasks that need exactly that. But most production AI calls are not those tasks. We just defaulted to frontier models because they were there, they worked, and we didn't have to think.

It's the same instinct as spinning up a dedicated server to run a cron job. Nobody questions it until someone asks why.

What actually runs in production?

Not demos. Not experiments. The AI features that actually run on every user action in real apps are basic:

• Classify this input

• Extract these fields

• Translate this message

• Transcribe this audio

• Read text from this image

• Summarise this into one sentence

• Yes or no based on this content

These tasks are not hard. A model doesn't need to understand nuance, handle ambiguity, or reason through complexity to route a support ticket into one of five categories. Smaller, purpose-built models — instruct models, STT models, OCR models — handle these accurately, fast, and at a fraction of the cost.

The reality: The gap between a frontier model and a smaller model on open-ended creative work is real and significant. The gap on a classification task with a clear schema is close to zero in practice.

The cost of the wrong default

When the default is always the biggest model available, a few things happen:

• Token anxiety becomes permanent: Every API call has an unpredictable cost that scales with input length, output length, and whatever the provider decides to charge next month. You watch the bill with mild dread as usage grows.

• Team mindset locks into "AI = Expensive": New developers joining the project assume this is just how it works. Nobody asks whether the task actually needs it.

• Configuration overhead never goes away: Model version, temperature, max tokens, system prompt tuning. Reasonable for something that needs it. Excessive overhead for a function that checks if a message is a complaint or a question.

We optimise everything else

We learned to be smart about physical resources. Thermostats, LED bulbs, sleep timers — we optimise heating and lighting in our homes without thinking twice about it. Not because we're environmentalists, but because wasting resources on something that doesn't need them is just bad engineering.

Yet we route millions of "is this spam?" checks through models that could write a novel.

If enough developers start routing tasks to appropriately-sized models, it also changes the market. Big providers currently have little pressure to simplify billing or compete on price for simple tasks because the default behaviour is to send everything their way regardless. That changes when the demand signal changes. This is not idealism — it's just how markets respond to volume.

What I built

I got tired of setting up the same calls, picking models, and counting tokens for tasks that are fundamentally the same every time. So I built a flat-rate API that handles the model layer automatically. You describe the task in plain English, send your content, get the result back.

• No model selection.

• No token counting.

• No SDK (Plain HTTP, any language).

It covers the tasks that actually run in production — classification, extraction, translation, summarisation, with transcription and OCR coming. The model is chosen for the task automatically. The price is flat monthly.

It's called a8k.me. Still early and honest about it. But the thesis is the point: most of what your app needs from AI is boring and predictable. The infrastructure should match.

I'm curious to hear from other devs here: How much of your current AI bill is actually going toward complex reasoning versus basic utility tasks like extraction or routing? Have you tried moving any of it to smaller models yet?