Stop Learning Prompts

We built half our pipeline on a model we don't control. One Sunday, it stopped answering.

I've been building with AI for over a year. We're a small team in Seoul, and for most of that time I ran on an assumption I never bothered to examine: the frontier model would always be one API call away. Always on, metered, available to anyone with a credit card. Like electricity.

Then one Sunday, it wasn't.

A model we'd built half our pipeline around simply stopped answering us. No outage. Every status page was green. It just wouldn't talk to a team sitting in Seoul.

---

The access came back. The thought it left behind didn't.

By lunch the next day (me, Sam, and Jax picking at our food), we weren't talking about downtime anymore. We were talking about something heavier. If access to the best models can be drawn along a border, then a startup in San Francisco and a startup in Seoul aren't in the same race. One of us gets to assume the frontier is always there. The other has to plan for it vanishing on a weekend.

We even said the quiet part out loud: maybe someday we'd have to move to the US just to keep our supply of intelligence. Building a company on the hope that a border stays open isn't a strategy. It's a bet, and it's not one we want to make.

---

Here's what I got wrong before that weekend.

There's a well-known discipline for the work that makes a model reliable: harness engineering. Building structure around an LLM call so the output stays consistent even when the model underneath isn't the smartest in the world. In our product, the piece that does this for code is called Charlie. It defines what good output looks like, checks the model's work against it, and catches failures before a human sees them.

My mistake: I'd been quietly thinking of harnesses as a coding thing. They're not. Every domain needs its own: a harness for video, a harness for documents, a harness for the boring office tasks a normal employee does all day. Stack those together, put a human-friendly face on them, and people stop seeing "modules." They see a persona. That stack of personas is what we call autosquad.

---

The same weekend pushed a second decision we'd been putting off: running a local model.

Our model bill has been climbing for months, and now access itself was in question. A local LLM had always felt like a science project, something to play with later. Those two pressures together turned it into a requirement. Yes, a local model is dumber than the frontier. That's fine. Closing the gap between "the model we're allowed to run" and "the output we need to ship" is the entire job of the harness.

---

So here's the advice I'd give my past self, and anyone building right now: stop taking prompt courses.

I've watched newer models read the exact same prompt completely differently from the ones before them. Memorizing clever phrasing is studying for a test that gets rewritten every few weeks. What actually compounds is the system around the call: the harness, the mesh of agents, the feedback loop that grades the output and feeds it back in. That's the part that survives a model swap.

Where we stand now: we're building so the model is the swappable part, not the foundation. Engineering, plus model, plus autosquad. The model is the piece we don't control. The engineering and the orchestration are the pieces we do.

We're not betting on always having the best model. We're building so we don't need one.

---

We're Team Sisyphus. We build AI-native tools and run our own team on them before we ship. DevKanban is live at sisyphus.team.

If your access to the frontier model disappeared this Sunday, which part of your product would still be standing on Monday, and what do you wish you'd already built?