Agnos LLM gateway proxy

Hey IH! 👋

I'm building Agnos Proxy - the first gateway-agnostic control plane for LLMs.

The Problem That Drove Me Here:

Every LLM gateway is a bet. You pick one and you're locked in.

• LiteLLM → Great reach (100+ providers). But 2026 CVEs proved your keys are at risk when translation code runs next to your vault.

• Bifrost → Blazing fast (sub-100µs). But limited provider support.

• Portkey → Beautiful UI + prompt management. But pricing doesn't scale.

• Kong → Enterprise-grade. But retrofitted for LLMs, not built for them.

You shouldn't have to choose.

The Insight:

Most gateways fuse two jobs:

1. Control plane (keys, budgets, guardrails, audit) - your valuable asset

2. Data plane (translation to providers) - fast-moving third-party code

When they're fused, a vulnerability in the translation code can reach your entire key vault. This is exactly what happened with LiteLLM in 2026.

The Solution: Agnos Proxy

Agnos decouples these jobs. Your control plane + encrypted vault stay in your infrastructure. Translation engines (LiteLLM, Bifrost, Portkey, Kong, built-in) become swappable, stateless commodities behind one endpoint.

What This Means For You:

✅ Gateway-Agnostic - Swap or combine ANY gateway with one config change
✅ Security - Keys stay in your vault, engines hold zero keys
✅ One Endpoint - Your apps never change, no matter which gateway runs underneath
✅ Performance - ~1ms overhead
✅ Cost Control - Hierarchical budgets + rate limits + per-request cost attribution
✅ Guardrails - PII/secret detection, CEL rules, block/redact/audit
✅ Observability - OTEL traces, Prometheus metrics, live dashboard
✅ MIT-Licensed - Self-hosted forever

Install in 60 Seconds:

bash

curl -fsSL https://raw.githubusercontent.com/siva010928/agnos-proxy-oss/main/install.sh | sh

Repo/Website Links:

• Website: https://agnos-llm-gateway.site

• GitHub: https://github.com/siva010928/agnos-proxy-oss

• Live demo (playground): https://agnos-llm-gateway.site/app

• Repo: https://github.com/siva010928/agnos-proxy-oss

• Youtube: https://www.youtube.com/watch?v=cGCz44h_D-k

Why This Matters For Indie Hackers:

The gateway market never stops moving. Prices change, providers get added, CVEs get disclosed, new gateways launch. Agnos makes sure you never have to bet on one gateway again.

Whether you're building a side project or scaling to thousands of requests/day, Agnos gives you the flexibility to adapt without rewriting your app.

What's Next:

• 🔜 v1.0 - Stable APIs + production hardening

• 🔜 Managed cloud (for teams who don't want to self-host)

• 🔜 Enterprise SSO (OIDC)

• 🔜 More engine adapters

Would Love Your Feedback:

• What gateways are you using?

• What providers do you need support for?

• What's your biggest pain point with LLM infrastructure?

Drop your thoughts below! 👇