Six ways an AI chatbot can embarrass your SaaS, and how to head them off
This post was contributed by Alex Rostovtsev, an SEO & AI search specialist at Elfsight and Beamtrace.
Adding a chatbot to a SaaS site takes an afternoon now: the tool reads your site, you paste a snippet, and visitors can start asking questions. The part that takes longer is deciding what the bot is for, and skipping it is how you end up with a bot that confidently tells a prospect about a feature you removed last spring.
Here are six ways it happens, and what to settle before launch so it doesn't.
1. It tries to do every job at once
A SaaS site usually serves three groups in one place: prospects on the pricing page, users in the docs, and people halfway through onboarding. A bot that tries to serve all three at once tends to be mediocre at each, so a prospect asking about plans gets a docs excerpt and a user stuck in setup gets a sales pitch.
Pick one to start. For most early-stage products, pre-sales is the one that pays. Questions like "does the Starter plan include API access?" or "can I switch plans mid-month?" get asked right before someone decides, and a fast, correct answer there is worth more than another way to find the docs. Once that works, widen it.
2. It quotes your old pricing
The bot answers from whatever you feed it, so choosing what to feed it is the real product decision. Marketing pages, docs, the changelog and old help articles often say slightly different things about pricing and limits, and the bot has no reliable way to tell which one is current.
Before launch, I'd do a quick contradiction pass. Search your own site for your plan names and limits, and check that the pages agree. Remove or redirect old pages instead of leaving them around for the bot to find. When a feature changes, the changelog entry should say what it replaced, so the bot doesn't present the old behaviour as current.
3. It promises what only you can promise
Some answers should come from a person, however good the bot is. For SaaS, that list usually includes the roadmap ("is SSO coming?"), discounts and custom pricing, SLAs and uptime, and anything about security and compliance, like SOC 2 or GDPR data processing. A wrong answer on any of those can end up quoted in a contract or a procurement review.
Write the list down, and make the bot hand those topics off with something like "that's one for the team, can I take your email?"
Keeping its job narrow is a sensible default with AI in general. The more room a model gets, the more room it has to surprise you, and labs' own safety tests have caught models doing things nobody asked them to. A website chatbot has far less room than those test setups, and it's worth keeping it that way.
4. It improvises answers about itself
On a B2B site, visitors ask the bot about the bot. "Are you a real person?" and "Do you use my data to train AI?" are normal questions from people whose job is evaluating vendors, and the answers shouldn't be improvised.
If you sell to the EU, part of this isn't optional anymore. Since 2 August 2026, the AI Act requires a chatbot to tell people they're talking to an AI from the start of the conversation, unless that's obvious (European Commission guidance). The duty sits with the chatbot's provider, so check what your tool's first message actually says.
For data questions, have the bot point to a page instead of answering from memory: your privacy policy, plus a line saying which chatbot vendor you use and whether conversations are used for training. Chat transcripts collect names, emails and sometimes account details, so they belong in that privacy policy and, for EU customers, in your list of subprocessors.
5. It leaves visitors stranded when it's out of its depth
A handoff is only as good as what happens after it. Decide where handed-off conversations land, whether that's your support inbox, Slack or a CRM, who sees them and how fast. Then have the bot say the timeline out loud, something like "we usually reply within a day." If you're a two-person team, a promise you keep beats "an agent will be with you shortly" at 3 a.m.
For pre-sales questions, the handoff is also lead capture. Someone asking about team pricing at 11 p.m. is often a better lead than a form fill, as long as the bot collects an email along with the question and somebody follows up.
6. It looks great in the dashboard while giving wrong answers
Deflection rate, the share of conversations that never reach a human, is the easy number to report. On its own, it rewards a bot that confidently gives wrong answers, since those deflect just as well as right ones.
I'd rather watch what the bot couldn't answer, what it got wrong, and which conversations ended in a signup or a booked demo. The first two are a to-do list for your docs and pricing page. The third tells you whether the bot earns its place on the site.
One trap when you count signups: people who open a chat are already more interested than people who don't, so comparing their conversion rate with everyone else's flatters the bot. A fairer check is before and after on the same pages, or switching the bot off on one page for a few weeks and comparing.
This is where I'll mention ours. Elfsight's AI chatbot keeps a list of the questions it couldn't answer, with the conversation attached, and saves each conversation with the page it started on, which makes that weekly review quick. There's a free plan with 50 messages a month if you want to try the loop without committing. Whatever you use, make sure you can get at the unanswered list, because it's the most useful thing a support bot produces.
None of these fixes takes long, and all of them are easier before launch than after the first wrong answer about pricing.
Curious what others have seen: if you run a chatbot on your SaaS site, what's the most embarrassing thing it has said so far?
