LLM Ops

Hey IH! 👋

I just launched LLM Ops - a cost tracking and optimization platform for companies using AI APIs (Anthropic, OpenAI, Google).

Why I built this:

A founder friend recently told me his Gemini API bill jumped from a few hundred dollars to $6,000+ in one month. Zero visibility into what caused it. No breakdown by feature, no alerts, nothing. Just a massive surprise bill.

I spent time at AWS managing $2B+ in infrastructure spend, and I saw this pattern constantly with cloud costs. Now it's happening with LLMs - but moving 10x faster.

The problem:

Most startups building AI features have NO IDEA:

- What their actual per-request costs are

- Which features/teams are burning money

- When costs are spiking until the bill arrives

- That output tokens cost 3-10x more than input tokens

LLM pricing is deliberately confusing. A model advertised as "$10 per million tokens" actually costs $40+ for typical usage patterns.

What I built:

LLM Ops gives you:

✅ Real-time cost dashboards (by model, team, project, feature) for Anthropic, OpenAI and Google Gemini

✅ Anomaly detection & alerts (catch $6K bills before they happen)

✅ Department-level cost allocation

✅ Optimization recommendations

✅ 60-second proxy integration (no code changes)

What I'm working on:

- Automated optimization (smart routing + caching)

- Multi-cloud support

- Custom alerting rules

- Team budgets and spending limits

Would love feedback on:

- What LLM cost pain points are you facing?

- What features would make this a must-have for you?

- How much would you pay for 40-60% cost savings?

Check it out: https://cloudidr.com/llm-ops

Also built a free LLM pricing comparison page (60+ models): https://cloudidr.com/llm-pricing

Happy to answer any questions! 🚀

---

Background: Former AWS EC2 Capacity Products leader, managed $300M ARR. Built cloud infrastructure at scale for 6 years before starting CloudIDR.