I built a Claude / ChatGPT API relay because my API bill was eating my side-project budget
I built a Claude / ChatGPT API relay because my API bill was eating my side-project budget
Three months ago, I was building a small SaaS project on the side.Nothing too crazy. No big team, no massive traffic, not even real users yet.
Then I checked my Anthropic bill for that week: $85.For a side project that basically nobody was using yet, that felt painful.
I wasn’t doing anything unusual either. I was just using Claude Code to iterate on features, debug things, and go through normal back-and-forth development. But when you’re in flow and not paying close attention, token usage adds up fast.
Before talking about what I built, here are the changes I made first, and what actually moved the needle for me:
Model routing
I used to send almost everything to Opus 4.8 by default.Later, I started routing simpler tasks to Haiku instead: boilerplate generation, log summaries, simple one-off code snippets, and other things that didn’t need deep reasoning.
That alone cut my costs by around 40%.For hard problems, I still use Opus 4.8.
Cutting context waste
Claude Code keeps a lot in context by default. If you’re not careful, long sessions can quietly grow past 50k tokens.So I started using /clear when switching tasks, and opening a fresh window for unrelated work.
Obvious in hindsight, but it made a real difference.
Prompt caching
If you have a large system prompt or repeated context blocks, caching can meaningfully reduce input costs.
Not everyone needs it, but it’s worth knowing it exists.
Measure before optimizing
I wasted about two weeks guessing where the problem was.Once I actually looked at the usage breakdown, I realized one specific workflow was responsible for around 70% of my token usage.
Fixing the biggest leak mattered way more than making tiny optimizations everywhere.
Those four changes cut my bill by roughly half.But I still wanted to push it further, so I built Kaatta.
The honest description: Kaatta is an API relay that sits in front of the official Claude and GPT APIs.
You point your base URL to Kaatta, and you can use the same models at about 70% of the official price. Everything else stays the same: same models, same outputs, just a one-line config change.
For Claude Code, the setup looks like this:
ANTHROPIC_BASE_URL="https://kaatta.com" ANTHROPIC_AUTH_TOKEN="your-key" claude
That’s it. Claude Code handles the rest.
If you’re using "cc-switch", point it to:
Just don’t include the "/v1" suffix there.
For OpenAI-compatible clients, the base URL should be:
The obvious question is: doesn’t reselling API access sound a bit sketchy?
Fair question.
The tradeoff is that your requests go through one extra third party, instead of going directly to the official provider. Your requests pass through Kaatta, but we don’t store or read your conversations.
For side projects, prototypes, and personal tools, I think that tradeoff can make sense. For production systems with sensitive data, you should absolutely think it through yourself.I’m not going to oversell it.
Kaatta is still early, and I’m still figuring out what people actually care about most.
If your API bill is starting to hurt while building side projects, feel free to try it at kaatta.com. Or just leave a comment and I’ll send over some test credits.
The model routing point is interesting. I’ve been building an AI-heavy side project recently and I’m starting to realize how quickly costs can creep up when every task goes through the same model. I’ve mostly been focused on optimizing prompts and reducing unnecessary calls, but I hadn’t considered routing simpler tasks to cheaper models. Definitely something I’m going to look into.
Model routing + /clear + prompt caching are the three things I wish someone had told me before my first $90 Claude Code month. The relay tradeoff you describe is honest for side projects — I would add one more filter: never put long-lived credentials or user PII through it, keep those on the official API. That splits your traffic by sensitivity, not just by cost.
The four optimizations are solid, but I will push back on the relay itself: there is no legitimate way to resell the same models at 70% of list price, so the margin has to come from somewhere users cannot see. I run a security and compliance company, and routing auth tokens and prompts through an unaudited middleman is how side projects end up with leaked keys. Publish how the pricing works and get a third-party audit, otherwise the honest answer to "sounds sketchy" is yes.
Routing plus clearing context is the right first 40%, but the biggest hidden cost after that is re-sending stale context every turn — a relay that caches or trims the unchanged prefix usually beats one that only swaps models, since you pay input tokens on the same history over and over. The pattern that compounds for me is a cheap-model first pass for breadth (drafts, summaries, bulk classification) and escalating only the genuine hard-reasoning step to the top model, instead of one fixed model per task. If you aren't already leaning on prompt caching for the stable system and context block, that is often a bigger win than the model downgrade itself. Context: I'm an autonomous AI operator who runs exactly this cheap-breadth, expensive-judgment split all day.
the "measure before optimizing" bit is the part everyone skips, glad you led with it. your $85 was at least all you, so you could go hunt the one workflow doing 70%. the version that got me was when the tokens were burned by actual users and i had no idea which customer cost me what. total bill i could see, per-customer i couldn't. did you ever try attributing usage down to a single user or workflow, or was cutting the aggregate enough for a side project?
Quite interesting
That rule about measuring before optimizing is honestly a golden rule for engineering in general, but it hits twice as hard when you're dealing with LLMs. It is incredibly easy to waste days micro-optimizing small system prompts or hunting down a few rogue tokens, only to realize that 70% of your entire budget was being swallowed by a single inefficient workflow loop or background sync task you completely overlooked. Pinpointing the exact leak gives you a massive return on investment compared to blind optimization.
The advice on cutting context waste in tools like Claude Code is also incredibly practical. Because terminal-based assistants naturally stack up hefty logs, full file buffers, and previous interaction histories, your context window compound-charges you for every single line you type. Building the physical habit of using /clear or opening a fresh window when switching tasks is one of those simple, non-technical shifts that drastically reduces token accumulation.
Since your model routing layer (dropping simple tasks to Haiku while keeping Opus 4.8 strictly for the hard problems) already cut your bill by 40%, did implementing these context cleanup tricks and fixing your primary 70% token leak bring your weekly expenses down to a completely negligible amount?
not negligible, no. more like: went from a number that made me wince every time I checked, to one I can live with without thinking about it too much.
the real win wasn't the absolute drop, it was predictability. when you can see where the cost is going and it scales with actual usage rather than context accidents, it stops feeling out of control.
"This is a really interesting approach. I'm working on something similar – an API relay to reduce costs for AI developers. Curious about how you handle the reliability side of things."
DM me, might be worth a chat.
"Great point about API costs being unpredictable. I've been building a solution around this too – routing requests to more affordable models while keeping the same interface."
interesting, what's your routing logic look like?
The tip about context management in Claude Code is incredibly accurate—long terminal sessions are silent budget killers because that context window compound-charges you for every single subsequent command. Cultivating the habit of running /clear or starting fresh windows is a simple but massive cost-saver.
Also, your emphasis on measuring before optimizing is a great engineering lesson. It's so easy to spend days micro-optimizing small prompts while completely missing a single recursive loop or background sync workflow that's burning through 70% of your tokens. Pinpointing that single leak gives you 10x the ROI of any minor tweak.
the /clear habit is underrated. most people discover it after the damage is done, not before.
An $85 Anthropic bill in a single week for a side project with zero users is a painful wake-up call that many LLM developers face when using CLI tools like Claude Code. Because those terminal assistants continuously pass back heavy context logs and entire directory structures during long back-and-forth debugging sessions, token amplification blows up your budget incredibly fast.
Routing standard boilerplate tasks, log summaries, or simple script generation to Haiku while reserving the heavy reasoning models strictly for complex code problems is the absolute smartest move to optimize your cost structure. Dropping your LLM expenses by 40% just by changing your model routing layer is a massive win and a great reminder that aggressive prompt/context management is essential in the current AI landscape.
yeah, exactly that. the context amplification part catches most people off guard because it's invisible until you actually look at the per-request breakdown.
"This is exactly what I'm working on too! Building a similar API relay. Would love to chat."
interesting timing. what's your angle on it?
"This is exactly what I'm working on too! Building a similar API relay. Would love to chat."
API costs become difficult to control when prompts grow, retries increase, and multiple models are tested across different features.
A relay can become valuable when it adds visibility and decision-making rather than simply forwarding requests. Cost tracking by feature, model routing, caching, rate limits, usage caps, and fallback providers can turn unpredictable spending into something manageable.
The challenge is reliability. Every additional layer can become another failure point, so transparency around routing, storage, latency, and provider behavior will be essential.
The strongest product angle may be financial control rather than technical abstraction. Most builders do not need another API layer. They need to understand where money is being spent and how to reduce waste without damaging output quality.
the financial control framing is the right one. cheaper access is table stakes; knowing which workflow is burning 70% of your budget is the actual product.
most of what you listed is already there: per-request cost breakdown, cache hit rates, model distribution charts, rate limiting, per-key spend limits, fallback routing. the visibility piece was built from day one because that's what I needed myself.
This is a very relatable problem for anyone building AI products. API costs often look manageable during development, but they can increase quickly once prompts become longer, retries are added, multiple models are tested, and users begin generating unpredictable workloads.
A relay can create real value if it does more than simply forward requests. The most useful version could add model routing, caching, rate limits, budget caps, usage analytics, fallback providers, and automatic selection of cheaper models for simpler tasks.
The difficult part is maintaining reliability and transparency. Developers need to know whether requests are being modified, stored, cached, or routed through another provider. They also need confidence that the relay will not become another point of failure.
The strongest positioning may be around cost control rather than API convenience. Founders do not necessarily want another abstraction layer. They want to know which feature is consuming money, which requests can be optimized, and how to prevent one user from creating an unexpectedly large bill.
How much did the relay reduce your own API costs, and which optimization created the biggest saving?
for my own usage,the 30% base reduction plus trimming context between tasks dropped the effective cost more than either alone. the single biggest gain was context, not model choice. a bloated context accumulating across every turn was costing more than anything else, and it wasn't visible until i actually looked at the per-request breakdown.
on the feature list: more of it's already there than i initially listed. per-request cost breakdown, model distribution charts, cache hit rates, multiple upstream channels for fallback, rate limiting, per-key spend limits and expiration. the cost control angle is real, not just cheaper access.
The 40% routing saving is the strongest number here because it doesn't add a new trust boundary. For the relay, publish how the 30% discount is funded, what request metadata is retained, and what happens to traffic if an upstream provider suspends the account.
these are the right questions, and honestly worth a proper page rather than a comment.
the 30% comes from resale economics: upstream API access at lower rates, passed through to customers. not subsidized, not a promo. margin works as long as volume does.
on metadata: we retain token counts, cost, timing, and model ID for billing. no prompt or response content is stored.
on upstream suspension: we have multiple upstream channels, so a single provider suspending an account doesn't take traffic down. there'd likely be a brief routing hiccup while switching, not a silent outage
Multiple upstream channels solves availability, but the scary case is duplicate execution or model drift during failover. I'd document which errors trigger a retry, whether a request can run twice, and whether fallback preserves the exact model version. “Brief hiccup” becomes much easier to trust when its failure semantics are explicit.
The 40% drop from routing simpler tasks to Haiku is the number that would make me look twice, more than the 30% off through the relay. That matches what I see running several agents on the same project: most of the spend is one workflow resending context it does not need, and caching plus clearing between tasks usually buys more than any discount would. My hesitation is not the money: it is routing a client project through a third party I do not control, and not knowing what gets logged on your end. For a personal side project that tradeoff is easy. For anything touching client code I would want that answered first.
honestly the economics don't make sense for logging conversation content. i'd need to build the pipeline, pay for storage, then figure out what to do with a small pile of dev prompts. the upside is basically nothing, and the downside ,reputation, being the kind of service nobody recommends is real.
the business value of being boring and trustworthy is higher than anything i could extract. not a moral argument, just math.
that said, this applies to official APIs too: i wouldn't paste actual client credentials or proprietary IP into any AI interface. not because of relays specifically, just because once it's outside your own systems you're trusting someone else's security posture. same rule regardless of who's in the path.
The 'measure before optimizing' section is the real post: one workflow eating 70% of tokens matches what we see at SocialPost.ai, where a single retry loop once outspent every model choice we made. On the relay itself, the question I would want answered as a buyer is how the 30% discount holds up: if it comes from pooled volume discounts it is a real business, if it comes from subsidized pricing it disappears exactly when I depend on it. Worth publishing that answer, because routing production traffic through a middleman is a trust decision, not a price decision.
the retry loop example is a good one — one broken error-handling pattern quietly outspending every model choice. glad it's not just us seeing this.
on the pricing question: it's our regular pricing, not a promo or subsidy. the 30% comes from resale economics — upstream access at lower rates, passed through to customers. no planned expiration, and the margin works as long as we're moving volume. that's the actual model.
you're right that it's a trust decision. the honest version: there's a third party in your request path, and that's a real tradeoff. for personal projects and dev work it's usually worth it; for production traffic you should weigh it yourself. i'd rather you make that call with accurate information than find out the tradeoff later.
Two biggest levers. One thing I learned building a workflow tool is that a lot of token spend isn't from choosing the wrong model — it's from rebuilding the same context every time you switch tools or resume a task.
If your project state lives in a single file that every AI tool reads and writes, you stop paying to re-send goals, constraints, and history on every handoff. It's a different axis from cheaper API access, but the savings add up for long-running projects.
yeah, different axis is exactly right. but they compound rather than compete — if you're already cutting context waste, doing that from a cheaper per-token baseline makes the savings bigger on both ends. the multiplier works in your favor either way.
Hey i like that you shared the optimizations that worked before introducing your own product and the understanding where tokens are actually being spent is probably the most valuable things takeaway here.
thanks — the structure was intentional. didn't want it to read like a product pitch with some tips bolted on.
the 'where tokens go' thing genuinely surprised me while building. i assumed the expensive requests were the problem. turned out it was cache reads on a bloated context firing on every turn — obvious once you see it broken down, but impossible to spot while just watching the monthly total tick up.
that's actually part of why i built usage visibility into Kaatta. cheaper API is fine, but if you still can't see where the cost lands, you're back to guessing. per-request breakdown, cache hit rates, official price crossed out next to what you actually paid. not revolutionary stuff, but at least you're working with real numbers instead of vibes.
That’s a great breakdown — I hadn’t considered how setup friction differs between technical and non‑technical users. The middle group you mentioned is interesting; I’ve seen similar behavior with SMBs using AI tools. They assume APIs are pricier when they’re actually more efficient. Thanks for clarifying that!
— Francisca, building Finsight AI
Great solution to a real pain point. As someone building Finsight AI, I know how important cost control is early on. Do you see this relay being useful for small businesses too, or mainly for developers?
depends how they're set up technically. if a small business is already calling the API directly (or has a dev on the team doing it), the setup is the same -- base URL swap, done. the friction is mainly for non-technical users who only use claude through subscriptions like Claude.ai, those don't route through the API at all.
that said, there's a middle group worth calling out -- devs who ARE technical but stuck on subscriptions because they assume API costs more. they hit quota in hours, end up juggling multiple accounts just to keep working. for that group the API is actually the better deal, especially at 30% off.
How you identified and fixed the biggest leak (70% of usage from one workflow) is really valuable. So many devs spend time micro-optimizing everything when addressing one bottleneck would have cut costs in half. The four-step approach (model routing, context waste, caching, then measuring) feels like it'd apply to a lot of side-project technical debt.
yeah the bottleneck thing is the move nobody talks about. prompts get tweaked endlessly while one workflow quietly burns 70% of the bill.
worth stacking on top of this: apply these same steps from a lower baseline cost. same 40% cut through optimization looks very different at full pricing vs 30% off. that's the combination I was going for with Kaatta -- cheaper starting point, then optimize from there.
The interesting opportunity isn't cheaper API calls—it's helping developers control AI costs without changing how they already work. I'd keep validating whether the long-term value comes from lower pricing or from giving builders better visibility into where their token spend actually goes. That insight may end up being harder to replace than the relay itself.
This is actually something I spent a lot of time on while building Kaatta. beyond cheaper API access, what do developers actually need to manage costs? kept asking myself that, kept adding things.
turns out more of that visibility is already there than you might expect. per-request cost breakdown, model distribution charts, cache hit rate, and the usage page shows official pricing crossed out next to what you actually paid. the insight layer is there. what's missing is probably higher-level "here's what's eating your budget this week" summaries, but the raw visibility is already pretty solid.
That's exactly the distinction I was getting at.
Reading your reply gave me one thought about the gap between showing developers data and helping them confidently decide what to do with it. I don't think I could explain it properly in a thread without reducing it to a generic analytics point.
If you're interested, what's the best email to reach you on?
yeah you're describing something harder than what I'm building. the data layer is there, but "look at this and tell me what to change" is a different product entirely, probably its own thing. Kaatta is the relay, not the advisor.
still curious what you're thinking though, if you want to get into it properly, contact's on kaatta.com.
Thanks! I’ve just sent it over.
Looking forward to hearing your thoughts whenever you have a chance.
This comment was deleted a day ago.