Why a tiny AI regex tool still runs on DeepSeek (and when I’d switch)

Mate, the JS validation is solid, but the better signal might be the gap between the first output and the regex users actually keep after editing. Quietly logging those final versions would give you a private quality metric that failure rate alone misses.
Until that gap starts looking wide, sticking with one default still feels right.
Hope it keeps working well for the people who need it.

As a web developer, I actually like this approach. For a small codegen tool, the important thing isn't the model name it's whether the generated regex consistently works in the actual JS environment.

I’d keep one default model for now too. If users start correcting patterns frequently, that gives you a much better signal that it’s time to test another model. Keeping the model behind the same structured schema also makes switching much easier later.

The if it doesn’t work on the page, it’s a failed generation rule is especially interesting. That kind of product-level validation seems more valuable than simply choosing a popular model.

One default seems right while the problem surface is narrow. I’d define an explicit switch threshold before adding a picker: for example, the share of generations failing JS validation, user correction rate, p95 latency, and cost per successful pattern. A small weekly sample of held-out prompts would keep the decision from becoming anecdotal. Keeping the schema constant is a strong choice too—it lets you swap models without making users relearn the product.

Keep the single default. Every option you expose is a support thread waiting to happen ("why did model X give a different answer"), and at your stage the SQLite cache is doing more for your margins than any model swap would. Same principle I see on the infra side: bills aren't high because founders picked the wrong service, it's the uncached repeat work. Your cache is the cost optimizer, the model is just the engine. Switch when the error rate costs you users, not before.

Genuinely useful to read, I'm making a similar call on a tiny tool where the output can be checked mechanically. On your last question: as a user I'd rather see one default and no model picker — a picker on a small tool makes me wonder whether the author knows which one is actually good. Has anyone asked you for a model choice, or is that a worry ahead of demand?

I’d keep one default and validate the failure mode with a same-day loop: find 3 users with real regex tasks, run them through, log where they hesitate, and expose model choice only when a repeated failure blocks a paid use case. I packaged that first-customer loop here → https://promptworks-storefront.vercel.app

I'd stick with one default. A model picker turns a failed regex into a debate about models instead of a problem you can fix. Your JS validation already gives you a clear failure signal. Track prompts that compile but later get edited or abandoned. Add a second model only if one type of job keeps failing. Until then, the picker mostly gives users another reason to question the result.

Nice constraint-first framing. I’d instrument the failure modes before switching: invalid-regex rate, repair-pass frequency, latency, and cost per successful conversion. Then keep a tiny regression set of real queries and run it across models weekly. That makes a switch evidence-based rather than brand-driven. Hiding the model picker is also a great UX choice.

The JS validation is a useful guardrail, but do users still have to edit the generated regex often enough for model quality to become the real bottleneck?