My product writes five escalating emails. I finally read them in order.
Nudge chases unpaid invoices: a ladder of reminders pinned to the due date, each written from that invoice's real context, the voice hardening as it ages. I've been building it for weeks and had never once sat down and read the whole sequence end to end against a single invoice.
I did that today. $3,400, brand identity work, a mid-size client - fictional, but shaped like the real thing. Five rungs: a heads-up three days before due, friendly at day 3, firm at 14, serious at 30, final notice at 45.
Three findings, in increasing order of how much they bothered me.
1. One generation in three failed.
Three of ten calls came back GatewayInternalServerError after three retries and fell through to the built-in template. My gateway account is on the free tier, which reaches exactly one model - claude-3-haiku, two generations old - and apparently not reliably.
I had been describing this as "AI-written reminders". It is closer to "AI-written about seventy percent of the time".
2. The templates were sometimes better than the model.
This is the one I didn't expect. Compare the firm rung. The model wrote:
This invoice is a formal matter that requires your attention.
The template it fell back to:
Could you let me know either way - is it scheduled for payment, or is something holding it up on your side?
The second is better. It's specific, answerable in five seconds, and doesn't sound like a letter from a bank. The model's version is what you get when you ask a language model to sound serious: it reaches for the register instead of the content.
I had assumed the AI was the product and the templates were the fallback. Read side by side, the templates are carrying more than I credited them with, and the escalation ladder - not the prose - is doing most of the work.
3. It signed an email with a name I never gave it.
The sender is "Sol". The business is "Sol Kim Design". The prompt tells the model who it is writing as by saying "From: Sol at Sol Kim Design", and the model signed off:
Regards,
Sol Kim
I don't have that surname. Nobody typed it. The model composed a person out of two fields.
It's a bug I fixed a week ago wearing better clothes. That one guessed a business name from a Gmail address and produced "Lebagbag", which at least looks wrong. An invented surname doesn't look wrong - it looks like a person. Nobody proofreading would catch it, and the client may well know the sender has no surname like that. On a demand for money, that's a bad thing to be wrong about.
Fixed the way I've fixed the others: checked in code rather than asked for in English. Only the last few lines count as a sign-off, a configured full name is left alone, and the name appearing mid-sentence isn't a signature.
The pattern, if there is one
Every quality rule I've written into the prompt has eventually been broken by the model. Every one I've moved into a code check has stayed fixed. Four so far - banned filler phrases, shouty subject lines, a bare link stranded below the signature, and now the sign-off.
I don't think that's a prompt-engineering skill issue. An instruction is a preference and a check is a guarantee, and the gap between them widens the closer you get to something irreversible. An email going out in someone else's name is about as irreversible as this product gets.
The ask
Still the thing I can't test alone: I have no idea whether these emails are good enough to send to a real client. Nobody has used this on a real invoice, mine included.
So - if you have an invoice that is genuinely late right now, send me the amount, the due date, and whatever you've already sent. I'll write you the next three follow-ups in sequence and send them back. No signup, nothing to install, and I don't need access to anything.
What I get is the one thing I can't fake: whether this reads like something a person would actually put their name on.
Helpful post. How did you get your first bit of traction?
Helpful post. How did you get your first bit of traction?
Helpful post. How did you get your first bit of traction?
Helpful post. How did you get your first bit of traction?
The end-to-end testing point is probably the biggest lesson here for me.
It’s easy to test each generation individually and think the system works, but the actual product experience is the sequence: what happens at day 3, then day 14, then day 30, and whether the tone and information still make sense as the situation changes.
I also like the distinction between instructions and guarantees. Anything that must never happen—like inventing a person's identity—probably belongs outside the model as a deterministic validation rule.
It makes me think of AI less as the system of record and more as one component inside a workflow where the important constraints are enforced by the application itself.
Helpful post. How did you get your first bit of traction?
Helpful post. How did you get your first bit of traction?
Helpful post. How did you get your first bit of traction?
Helpful post. How did you get your first bit of traction?
Helpful post. How did you get your first bit of traction?
Clear and practical, thanks. Did anything surprise you along the way?
The distinction between a prompt instruction and a code-level guarantee is probably the most important part of this.
I’ve seen the same pattern in AI products: prompts are good at expressing intent, but they’re a poor place to enforce invariants. If a wrong name, price, URL or other customer-specific field can actually reach production, I’d treat that as application logic rather than prompting.
The other interesting takeaway is that your fallback templates aren’t really a fallback if they consistently produce better outcomes. That suggests the escalation logic and the constraints around the message may be doing more of the product work than the generation itself.
I’d be curious whether you eventually measure this by reply/payment rate rather than perceived writing quality. That seems like the real test for this kind of product.
The middle point is the one I've been avoiding saying out loud, and you've said it more precisely than I did.
If the fallback consistently produces better outcomes then it isn't a fallback - it's the product, and the model is a garnish I've been describing as the engine. That reframes what Nudge actually is. Not "AI writes your chase emails" but "a schedule and a set of constraints that happen to be rendered as prose", where the generation is the least load-bearing part. The ladder, the cancellation guarantee, the per-client tone, the withdrawal of the excuse over time - none of that needs a model.
I don't think I'm ready to act on that fully, and I want to be honest about why rather than pretend it's a considered position: the AI is most of what makes the landing page interesting, and four competitors in this space are all selling the same sentence. Dropping it is commercially scary in a way that has nothing to do with whether it's true. Which is exactly the kind of reason I should distrust.
What I can do cheaply is stop lying about the ratio. "AI-written" when it is AI-written seventy percent of the time and the other thirty is often better is a claim I'd rather not defend.
On measurement - yes, and specifically days-to-payment on invoices that were already late when the sequence started, not reply rate and not perceived quality. Someone else in this thread pushed me on the same thing and my answer is that I can't do it yet: no user has sent one of these to a real client, so there is nothing to measure. The honest interim test is narrower and binary - will a real person put their own name on what this wrote and press send to a client they want to keep. Nobody has done that either.
And on invariants belonging in application logic: four of them are now checks rather than prompt lines, and every single one was a prompt line first that the model broke. I have no counter-examples in the other direction. Not one rule has held because I asked nicely.
I’ve started treating every generated sequence as a single artifact, not five isolated outputs. I read it once as the recipient, then run checks for identity, dates, commitments, and an explicit next action before any send. The model can draft tone, but those invariants belong in code or a deterministic template.
Reading it once as the recipient is what I did for the first time this week, and it is the reason there was anything to post. Five rungs had each passed their own review in isolation for weeks. Read in order, as one artifact, three things fell out immediately - the duplicate at day 3, the invented surname, and the fact that the fallback was outwriting the model on the firm rung. None of those are visible one email at a time, because none of them are about a single email.
What I'd add to your list, from the thing that actually bit: read it as the recipient with the dates consistent. My first pass had a fixture where dueDate and daysOverdue contradicted each other, and the model quietly reconciled them by inventing a date. I spent a while convinced it couldn't compute weekdays before realising I'd handed it two facts that couldn't both be true. An invariant check would have caught that in a second; my eyes did not.
On the split - identity, dates, commitments, explicit next action in code, tone from the model - that is now almost exactly where I've landed, with the addition that money belongs in the first group too. Someone else in this thread made that point and it was the gap: I was checking the name and trusting the figure, when the figure is the one the client can argue with.
The part of your process I don't have is the explicit-next-action check. Right now the templates all contain one by construction and the model is merely asked for one, which by my own argument means it is a preference rather than a guarantee. I tried writing that check early on and deleted it - it couldn't distinguish a date the sender was requesting from a date merely mentioned, so it rejected every template for quoting the due date. Still unsolved. If your version handles that distinction I'd like to know how, because I suspect the answer is that the next action has to be a structured field the email renders, not prose the checker reads back out.
Really relatable. How much time do you put into this each week?
Really relatable. How much time do you put into this each week?
The invented surname is the scary failure here because it looks polished enough to survive review. I would treat identity fields and money terms as protected values, not prompt context: render names, invoice amounts, and dates from structured data after generation, then reject output containing an unapproved person or value. That is why DictaFlow cleanup stays focused on transcription noise. Once a model starts speaking for the user, it is doing a different and much riskier job.
You found the half I'd left out, and it was the more dangerous half. Went and implemented it before replying.
What I had was a blacklist: check the sign-off, reject a surname the sender never gave. What you're describing is the inverse - approve the set of values allowed to appear, reject anything else. I'd only applied that to the name, and the name is not the field that costs money.
Nothing was checking the amount. The model was handed the right figure and trusted to repeat it, which is the same trust I'd already watched fail three times on wording. A reminder that misstates what is owed is worse than one that misstates who sent it: the client can only conclude the sender doesn't know what they're owed, and a figure they can point at is a reason not to pay today rather than a reason to pay.
So money is now an approved value. Any currency-marked token in subject or body that isn't the invoice's own total rejects the draft, with spacing and a trailing .00 normalised away so "$3,400" and "$ 3,400.00" both pass. Invoice numbers, dates, sort codes and account numbers carry digits too, and none of them is a claim about the debt, so only currency-marked tokens count. Tests cover a wrong amount, the same amount written three ways, a body full of non-money digits, and every built-in template.
Dates I haven't done and I'm not sure the whitelist shape fits them. The model legitimately invents one - the date it asks the client to pay by - and that date is supposed to be new. What's checkable is narrower: any date it states as fact must be the issue date or the due date. That's a real check, it's just a smaller one than "reject unapproved values", and I'd rather say so than claim the pattern generalises cleanly.
On your last line - once a model starts speaking for the user it's doing a different and much riskier job - that's the sentence I'd have wanted at the start of this project rather than a week in.
"An instruction is a preference and a check is a guarantee" is the clearest way I've seen that stated.
The templates outperforming the model on the firm rung is the finding worth sitting with. The firm escalation has a specific job: get an answer, either "it's coming" or "something's wrong." Your template nailed that. The model defaulted to register — sounding serious — which is what you get when you prompt a model to escalate without defining what escalation is supposed to accomplish. "Formal" and "effective" are not the same thing, especially in invoice chasing.
What the model did well is probably the earlier rungs where the goal is warmth and context, and where small variations are acceptable. What the template does better is the structural moves: the answerable question, the specific next step, the tone that gets someone to respond rather than defer.
The 70% reliability problem is a real blocker for anything revenue-adjacent. A client who gets three identical fallback emails doesn't know why, but they'll notice the pattern, and it undermines the credibility of the whole sequence.
The sign-off bug is the kind of thing that sounds minor until the wrong person notices. In an invoice context, where the power dynamic is already sensitive, an invented name on a payment demand is the exact wrong thing to get wrong.
If you're building out the validation step — send me three real sequences when it's ready. Happy to punch holes in them.
You called the duplicate before I found it, and it was worse than you described.
The default ladder carries the friendly tone at both day 0 and day 3. The friendly template said "due today" regardless of the day. So the day-3 reminder was byte-identical to the day-0 one, and it told somebody three days late that their invoice was due that morning. Not three identical fallbacks - two, plus a factual error about the one number that matters.
The cause turned out to be worse than the instance. Each template stated the timing itself, so each could be wrong on its own, and Settings lets a workspace put any tone on any offset - firm three days before due cheerfully announced the invoice was "3 days overdue".
Fixed by computing the standing once and sharing it, long form for the body and short for the subject. Which also means no two rungs can repeat now, because every tone carries a number that moves. Tests assert it for the built-in ladder and for any tone duplicated across two offsets, since the guarantee has to hold for ladders nobody has configured yet.
You're also right about where the model earns its place, and I had it backwards. I assumed the AI was the product and the templates were the fallback. Your split is better: the structural moves - answerable question, specific next step - are deterministic work, and the model's value is warmth and context on the early rungs where variation is fine and being wrong is cheap.
Taking you up on the offer. Same invoice throughout - $3,400, brand identity work, issued 25 August, due 8 September. Subject lines included because you're right that a client notices the pattern before the content.
Day 0 - "Invoice INV-2026-041 - $3,400.00, due today"
Day 3 - "Invoice INV-2026-041 - $3,400.00, 3 days overdue"
Day 7 - "Invoice INV-2026-041 - $3,400.00, 7 days overdue"
Day 30 - "Invoice INV-2026-041 - $3,400.00, 30 days overdue"
Day 45 - "Final notice - invoice INV-2026-041"
Punch away. The one I'm least sure of is day 30: "I'd rather sort this out directly than escalate it" names a consequence by gesturing at it, which may be worse than either naming it outright or leaving it alone entirely.
Really relatable. How much time do you put into this each week?
Nice, this makes a lot of sense. What's been the most surprising part of it so far?
Really solid approach — curious how you're thinking about this, what's been the hardest part to figure out so far?
Really interesting approach. The distinction between giving AI instructions and actually guaranteeing the behavior is important. AI for ambiguity, code for certainty feels like a great principle for building reliable SaaS products.
The point about moving important rules from prompts into code really resonates.
I’ve seen a similar pattern while working on AI-driven lead follow-up and automation systems at https://nexiobit.com/ . AI is great at handling conversations, but critical rules like timing, escalation, customer data, and human handoff should stay deterministic.
Sometimes a simple template with the right workflow is more reliable than giving the model too much freedom. The AI can handle the language while the system handles the boundaries.
This is great work — reminds me of some of the calls I've had to make building Xstream4K. What would you do differently if you started over?
Nice work shipping it. What has been the biggest challenge since launch?
What made you pick this stack over the alternatives?
Nice work shipping it. What has been the biggest challenge since launch?
Clear and practical, thanks. Did anything surprise you along the way?
Helpful post. How did you get your first bit of traction?
Good write-up. What would you do differently if you started again?
Solid lesson. Which channel has worked best for you so far?
Your template gives the client an easy way to explain a delay, like an invoice waiting for approval. That matters if you want to keep working together after it's paid. The sender can be clear about what's owed without assuming bad intent.
This is the comment that reframed the thing for me, and it cuts against how I'd been describing my own product.
I'd been treating the diagnostic question - "is it waiting on an approver, a PO number, or the next payment run?" - as a way to get information. You're pointing out it mostly functions as a way to hand the client a socially acceptable exit. They may not know which of those it is either. What the menu does is establish that the delay has an administrative explanation, before either side has to consider that it might not.
Which means its value is almost the opposite of what I thought. It isn't diagnosis, it's a shape the client can step into. And that explains something I couldn't account for earlier in this thread: why the template beat the model on that rung. The model was trying to sound like escalation. The template was offering a door.
The part I'd push back on slightly: it only works while the explanation is still plausible. At three days, "it's stuck in approval" is almost certainly true. At forty-five days across six ignored emails it has become a thing you are both pretending, and continuing to offer it starts reading as either naivety or sarcasm. So the same sentence that protects the relationship early corrodes it late.
That's the actual job of the ladder, stated better than I've managed to state it: not increasing pressure, but withdrawing the excuse one step at a time, slowly enough that the client can take the exit before it closes.
I don't think the current rungs get the withdrawal rate right - the jump from "I'm assuming it just got buried" at day 3 to "I haven't been able to get a payment date from you" at day 30 is a long way with only one stop in between. Noting that rather than claiming it's solved.
This is useful. How are you finding your first users so far?
Appreciate the honesty here, most people only share the wins.
For the first real invoice users, what would validate Nudge beyond approving the copy—actually sending the sequence and seeing whether it changes payment behavior?
Approving the copy validates nothing, and I think you've been asking me this in one form or another for a week. So here is the answer I'd actually hold myself to.
The thing to measure is days-to-payment on invoices that were already late when the sequence started. Not open rate, not reply rate - those measure whether the email was interesting, which is not the job. If an invoice sat at 40 days for a month and a half and gets paid eleven days after the first reminder, that is the claim. If it gets paid on day 60 the way it was probably going to anyway, the product did nothing and I'd rather know.
The honest problem with that measurement is n=1 and no control. One freelancer, one client, one invoice that might have been paid that week regardless. I can't randomise somebody's receivables, and I'm not going to pretend a single recovered invoice proves anything.
What I think it does prove, and the only thing I'd claim from it: whether a real person, looking at what this wrote, is willing to put their own name on it and press send to a client they want to keep working with. That is a binary and it is currently untested. Nobody has done it - me included, because I have no clients to chase.
Two things I'd need from a first user to make even that much real:
The reply the client sends back, if any. Not the payment - the reply. That's where you learn whether the diagnostic question worked or whether it read as a form letter. A client who answers "it's stuck waiting on a PO" has told you the email did its job even if the money is three weeks out.
Whether they edited it before sending, and what they changed. Every edit is a line the product got wrong about their voice or their relationship with that client. That's better signal than the payment date and it arrives immediately.
So the offer in the post stands and I'd rather have one of those than ten more people telling me the ladder sounds sensible.
That’s a much more useful validation framework. Could be worth comparing notes on this by email sometime, if you’re open to it.
Good point. Did you test that with users before committing to it?
Appreciate the honesty here, most people only share the wins.