My product writes five escalating emails. I finally read them in order.
Nudge chases unpaid invoices: a ladder of reminders pinned to the due date, each written from that invoice's real context, the voice hardening as it ages. I've been building it for weeks and had never once sat down and read the whole sequence end to end against a single invoice.
I did that today. $3,400, brand identity work, a mid-size client - fictional, but shaped like the real thing. Five rungs: a heads-up three days before due, friendly at day 3, firm at 14, serious at 30, final notice at 45.
Three findings, in increasing order of how much they bothered me.
1. One generation in three failed.
Three of ten calls came back GatewayInternalServerError after three retries and fell through to the built-in template. My gateway account is on the free tier, which reaches exactly one model - claude-3-haiku, two generations old - and apparently not reliably.
I had been describing this as "AI-written reminders". It is closer to "AI-written about seventy percent of the time".
2. The templates were sometimes better than the model.
This is the one I didn't expect. Compare the firm rung. The model wrote:
This invoice is a formal matter that requires your attention.
The template it fell back to:
Could you let me know either way - is it scheduled for payment, or is something holding it up on your side?
The second is better. It's specific, answerable in five seconds, and doesn't sound like a letter from a bank. The model's version is what you get when you ask a language model to sound serious: it reaches for the register instead of the content.
I had assumed the AI was the product and the templates were the fallback. Read side by side, the templates are carrying more than I credited them with, and the escalation ladder - not the prose - is doing most of the work.
3. It signed an email with a name I never gave it.
The sender is "Sol". The business is "Sol Kim Design". The prompt tells the model who it is writing as by saying "From: Sol at Sol Kim Design", and the model signed off:
Regards,
Sol Kim
I don't have that surname. Nobody typed it. The model composed a person out of two fields.
It's a bug I fixed a week ago wearing better clothes. That one guessed a business name from a Gmail address and produced "Lebagbag", which at least looks wrong. An invented surname doesn't look wrong - it looks like a person. Nobody proofreading would catch it, and the client may well know the sender has no surname like that. On a demand for money, that's a bad thing to be wrong about.
Fixed the way I've fixed the others: checked in code rather than asked for in English. Only the last few lines count as a sign-off, a configured full name is left alone, and the name appearing mid-sentence isn't a signature.
The pattern, if there is one
Every quality rule I've written into the prompt has eventually been broken by the model. Every one I've moved into a code check has stayed fixed. Four so far - banned filler phrases, shouty subject lines, a bare link stranded below the signature, and now the sign-off.
I don't think that's a prompt-engineering skill issue. An instruction is a preference and a check is a guarantee, and the gap between them widens the closer you get to something irreversible. An email going out in someone else's name is about as irreversible as this product gets.
The ask
Still the thing I can't test alone: I have no idea whether these emails are good enough to send to a real client. Nobody has used this on a real invoice, mine included.
So - if you have an invoice that is genuinely late right now, send me the amount, the due date, and whatever you've already sent. I'll write you the next three follow-ups in sequence and send them back. No signup, nothing to install, and I don't need access to anything.
What I get is the one thing I can't fake: whether this reads like something a person would actually put their name on.
Helpful post. How did you get your first bit of traction?
Clear and practical, thanks. Did anything surprise you along the way?
The distinction between a prompt instruction and a code-level guarantee is probably the most important part of this.
I’ve seen the same pattern in AI products: prompts are good at expressing intent, but they’re a poor place to enforce invariants. If a wrong name, price, URL or other customer-specific field can actually reach production, I’d treat that as application logic rather than prompting.
The other interesting takeaway is that your fallback templates aren’t really a fallback if they consistently produce better outcomes. That suggests the escalation logic and the constraints around the message may be doing more of the product work than the generation itself.
I’d be curious whether you eventually measure this by reply/payment rate rather than perceived writing quality. That seems like the real test for this kind of product.
The end-to-end testing point is probably the biggest lesson here for me.
It’s easy to test each generation individually and think the system works, but the actual product experience is the sequence: what happens at day 3, then day 14, then day 30, and whether the tone and information still make sense as the situation changes.
I also like the distinction between instructions and guarantees. Anything that must never happen—like inventing a person's identity—probably belongs outside the model as a deterministic validation rule.
It makes me think of AI less as the system of record and more as one component inside a workflow where the important constraints are enforced by the application itself.
I’ve started treating every generated sequence as a single artifact, not five isolated outputs. I read it once as the recipient, then run checks for identity, dates, commitments, and an explicit next action before any send. The model can draft tone, but those invariants belong in code or a deterministic template.
Really relatable. How much time do you put into this each week?
Really relatable. How much time do you put into this each week?
The invented surname is the scary failure here because it looks polished enough to survive review. I would treat identity fields and money terms as protected values, not prompt context: render names, invoice amounts, and dates from structured data after generation, then reject output containing an unapproved person or value. That is why DictaFlow cleanup stays focused on transcription noise. Once a model starts speaking for the user, it is doing a different and much riskier job.
"An instruction is a preference and a check is a guarantee" is the clearest way I've seen that stated.
The templates outperforming the model on the firm rung is the finding worth sitting with. The firm escalation has a specific job: get an answer, either "it's coming" or "something's wrong." Your template nailed that. The model defaulted to register — sounding serious — which is what you get when you prompt a model to escalate without defining what escalation is supposed to accomplish. "Formal" and "effective" are not the same thing, especially in invoice chasing.
What the model did well is probably the earlier rungs where the goal is warmth and context, and where small variations are acceptable. What the template does better is the structural moves: the answerable question, the specific next step, the tone that gets someone to respond rather than defer.
The 70% reliability problem is a real blocker for anything revenue-adjacent. A client who gets three identical fallback emails doesn't know why, but they'll notice the pattern, and it undermines the credibility of the whole sequence.
The sign-off bug is the kind of thing that sounds minor until the wrong person notices. In an invoice context, where the power dynamic is already sensitive, an invented name on a payment demand is the exact wrong thing to get wrong.
If you're building out the validation step — send me three real sequences when it's ready. Happy to punch holes in them.
Really relatable. How much time do you put into this each week?
Nice, this makes a lot of sense. What's been the most surprising part of it so far?
Really solid approach — curious how you're thinking about this, what's been the hardest part to figure out so far?
Really interesting approach. The distinction between giving AI instructions and actually guaranteeing the behavior is important. AI for ambiguity, code for certainty feels like a great principle for building reliable SaaS products.
The point about moving important rules from prompts into code really resonates.
I’ve seen a similar pattern while working on AI-driven lead follow-up and automation systems at https://nexiobit.com/ . AI is great at handling conversations, but critical rules like timing, escalation, customer data, and human handoff should stay deterministic.
Sometimes a simple template with the right workflow is more reliable than giving the model too much freedom. The AI can handle the language while the system handles the boundaries.
This is great work — reminds me of some of the calls I've had to make building Xstream4K. What would you do differently if you started over?
Nice work shipping it. What has been the biggest challenge since launch?
What made you pick this stack over the alternatives?
Nice work shipping it. What has been the biggest challenge since launch?
Clear and practical, thanks. Did anything surprise you along the way?
Helpful post. How did you get your first bit of traction?
Good write-up. What would you do differently if you started again?
Solid lesson. Which channel has worked best for you so far?
Your template gives the client an easy way to explain a delay, like an invoice waiting for approval. That matters if you want to keep working together after it's paid. The sender can be clear about what's owed without assuming bad intent.
This is useful. How are you finding your first users so far?
Appreciate the honesty here, most people only share the wins.
For the first real invoice users, what would validate Nudge beyond approving the copy—actually sending the sequence and seeing whether it changes payment behavior?
Good point. Did you test that with users before committing to it?
Appreciate the honesty here, most people only share the wins.