I reworded one buyer question and a SaaS tool went from recommended every time to never named. Here is the data.
I've been building a tool that measures how ChatGPT, Claude, Perplexity and Gemini recommend software, and I ran something this week that I think matters if you sell a product.
I asked the same buyer question two ways and ran each one 10 times on all four models. First the plain label, "best shared inbox software for small teams." Then the way a real buyer types it, "how do we collaborate on shared email inboxes without losing our normal email workflow."
On ChatGPT, Missive got recommended 10 out of 10 for the label. Reworded, it got named 0 out of 10. Same company, same site, one different sentence. Hiver barely moved (10 to 9). Front and Help Scout did not move at all, 10 out of 10 both ways.
It was not just ChatGPT. Perplexity did the mirror image, Hiver 10 out of 10 on the label and 0 reworded, and it started telling people to just use Gmail and Outlook instead of naming a product. The direction of the swing differed by model, but the plain label was always calmer than the real-buyer phrasing.
Two things stuck with me. One, a single check will lie to you. If I had asked once and seen Missive missing I would have sworn they had an AI problem. They are 10 out of 10 on the other phrasing. Two, the tools that held were not gaming anything, their pages describe the actual job in the buyer's own words, so a model can place them no matter how the question gets asked.
I wrote up the full thing with every number here: https://www.bersyn.com/blog/shared-inbox-phrasing-flip?utm_source=indiehackers&utm_medium=community&utm_campaign=shared-inbox-teardown . If you want to see what the models say about your own category, you can run your domain through the free scan, it is linked in the post.
Curious what your category looks like. Is AI recommendation this sensitive to wording for you, or was shared inbox just a swingy one?
The companies that held across both phrasings were describing the actual job the buyer is trying to do, not just the feature set or product category. Is there a pattern in the positioning of the tools that dropped across all four models?
Good question, and yes there's a pattern, with one honest caveat.
The two that dropped (Missive on ChatGPT, Hiver on Perplexity) both anchor their positioning to the category label. Hiver's whole pitch is "shared inbox for Gmail," Missive's is "team inbox and chat." Say the label back to the model and they surface instantly. Reword it to the messy real job ("collaborate on email without breaking our normal workflow") and the anchor stops matching, so they fall out.
The two that held (Front, Help Scout) describe the job, not the label. Front talks about where teams handle customer conversations, Help Scout about support that still feels like normal email. That language survives rewording because it already sounds like how a buyer would describe the problem.
The caveat: it's model-specific, not universal. Hiver actually held on ChatGPT (10 to 9) and only cratered on Perplexity. So it's not that these tools have bad positioning in the abstract, it's that each model rewards a different anchor, and the label-anchored pages are the ones exposed the moment the wording moves. That's the part a single check hides.
If you run your own category through it, I'd be curious whether the same split shows up. Small sample so far, would genuinely love more data points.
This comment was deleted 2 months ago