luoluoAI

The first version of luoluoAI looked straightforward: accept an OpenAI-compatible request or a Claude Messages request, send it to the right provider, and return the response.

The happy path worked quickly. The bugs showed up at the edges:

• SSE chunks do not always line up with message or JSON boundaries.
• OpenAI tool_calls and Claude tool_use blocks carry similar information in different shapes.
• A provider can report the same failure with a completely different status, error body, or finish reason.
• Partial streams, reconnects, and parallel tool calls are especially easy to mishandle.

We ended up building a small normalization layer around those cases. Internally, we keep one event model, translate tool-call arguments incrementally, normalize usage and finish reasons, and keep provider-specific errors from leaking into the application layer.

The main rule became: provider-specific payloads should stop at the gateway boundary. The application should not need a separate streaming parser every time it adds another model.

It is working well for our own agent workloads, but the long-tail cases still need more testing: reconnects after partial streams, long-running tool loops, and unusual provider error responses.

If you are running coding agents against more than one model provider, how are you handling this today—one adapter per application, or a shared gateway?

We run a mix of coding agents and automation workflows across Claude, GPT, DeepSeek, Grok, Gemini, and GLM. In one recent month, our aggregate usage crossed roughly 30 billion tokens.

I expected model cost or output quality to be the biggest headache at that scale. It wasn’t. The real drag was the plumbing around the models: API keys scattered across accounts, different authentication and billing systems, streaming responses that failed in different ways, and incompatible tool-call formats.

We were also missing a basic answer to a basic question: which agent or project was actually consuming the budget?

That is why we built luoluoAI. It started as an internal gateway that puts provider-specific behavior behind one contract. Existing tools can use an OpenAI-compatible endpoint, while Claude integrations can use an Anthropic Messages-compatible endpoint. The gateway handles model configuration, SSE normalization, tool-call translation, and usage tracking in one place.

The happy path came together quickly. Most of the work ended up in the edges: partial streaming chunks, event ordering, reconnects, provider-specific errors, and translating between OpenAI tool_calls and Claude tool_use blocks without breaking downstream applications.

We have now opened a limited public version. It is still evolving, but it has been stable enough for our own day-to-day agent workloads, and I’m interested in seeing where it breaks for other setups.

If you run agents across multiple model providers, where do you keep this compatibility layer today: inside every application, or behind a shared gateway?