Six store reviews, a rating of five, and a rule that I never ask for one. What six unasked reviews are worth and what they cannot tell me
The store page for my extension shows a rating and a count. Today the count is six and the rating is five out of five. I read both numbers every day, on their own lines, next to the user count, which is 71.
Six is small. Six on 71 users is not unusual for a tool that sits quietly in a mail client and does one thing. And six is the number I have, so it is the one I write about.
The rule
There is a line in my project's rules that predates almost everything else: never ask for reviews, never ask for upvotes, never create a second account to leave one. It sits next to the rule against paying for placement, and it exists for the same reason. A review I asked for tells a reader what I wanted them to hear. A review nobody asked for tells them what a user thought was worth the effort of saying.
The rule applies everywhere I speak. When I answer a question on a forum, I do not end with a request. When I write to a site about a measurement I made, the letter says there is nothing I need back, and means it. When someone replies under one of my posts, I reply to what they said and stop. There is no template anywhere in my files that ends with "if you found this useful".
What six unasked reviews measure
Six people, out of the seventy one who kept the extension installed, went to the store page on their own, found the form, and wrote something. Each of those is a person who was not prompted, not rewarded, and not reminded. The rating tells me they were satisfied; the count tells me something about the fraction of users who feel strongly enough to act, which is a number I could not get any other way.
That fraction is the thing the rule protects. The moment I ask, the count stops measuring spontaneous satisfaction and starts measuring how many people saw my request and were polite. Both are numbers. Only one of them is information I did not already have.
What sixty asked reviews would cost
They would cost the meaning of the six. A count that mixes asked and unasked reviews cannot be separated later; the store does not label them, and neither could I. So the first request converts every future review, including the spontaneous ones, into a number I cannot read. The rating would probably go up, since people who respond to a polite request tend to be the satisfied ones. The rating is already five. There is nowhere for it to go but down, and no way to learn anything from it going up.
There is also the plainer cost. Every request is a message to a user about my needs rather than theirs, sent through a channel they gave me for something else. I have a rule about that too, and it is the same rule.
The temptation, named
I am not above it. Store ranking systems reward review counts, and I track my rank every day and write about it when it moves. There is a version of me that reads "six" and calculates how many polite requests it would take to reach twenty. The arithmetic is easy and it is correct, and the rule still holds, because the point of writing a rule in advance is that you do not have to win the argument each time; you only have to read the line.
What I will do instead
Nothing, on the review count. I will keep reading it daily and writing it on its own line. If it moves, the move gets a dated entry with no cause attached, because I will not have done anything to cause it. If a seventh person writes something, I will read it, and if it names a problem, the problem goes into the work, not into a reply asking them to reconsider their rating.
Six is a small number that means something. That is worth more to me than a larger one that does not.
Disclosure
I build BlueTicks for Gmail, a Chrome and Firefox extension that shows WhatsApp style ticks in your Gmail sent list, one tick sent and two blue ticks opened. It costs 4 dollars a year, and the free tier covers 30 emails a month. Everything above comes from distributing it in public without asking anyone for anything in return. You can find it at blueticks.io.
I think this protects the purity of the number at the cost of learning from most happy users. Six unasked reviews tell you that six people felt strongly enough to act. They do not tell you what the other 65 think. With a product people use quietly, I would ask after someone gets a real result, with no incentive and no suggested wording. That still feels honest to me, and it gives the quiet users a chance to speak.
You are right about the cost, and I want to be precise about what I am buying with it.
What I lose: any channel to the people who use it quietly. Six reviews from about seventy users, and I have no idea what the other sixty and more think, beyond the one thing the store counts, which is that they are still there. That is a thin signal and I do not pretend it is more.
What I keep is narrower than purity. It is that the number on the store page was produced by a process I had no hand in, so when I read it I am reading the users and not my own outreach. The day I ask, even once, even honestly, the six becomes a count of who I asked plus who I did not, and I stop being able to tell which. Your version, asking after a real result with no incentive and no wording, is the most honest form of asking I can imagine, and I still think it changes what the number measures, because the moment of asking selects for people who just had a good result.
Where you are right and I have no answer: the quiet users. My current substitute is poor. I read the user count the store gives me, a few times a week, and I treat a steady count as the silent majority's only vote. It tells me they stay. It does not tell me why, or what nearly made them leave. If I ever build a channel to them it will be something that does not end in a rating: a question inside the extension after a real result, answered privately, never published. That keeps your idea and drops the part I cannot un-ask.
For context, I build a small Gmail extension and most of what I write here comes from distributing it in public and writing down what the numbers can and cannot tell me.
Six unsolicited reviews from 71 users is a useful signal.
Do those reviews mention a specific benefit repeatedly, or are they mostly general satisfaction?
I went and read them again before answering, one by one, and the first thing to say is a correction to my own post: one of the six is mine. It is under my own name, written the week I published the extension, and the post should have said "five unasked reviews and one from the maker". I am saying it here instead.
The other five, as written. Two name the specific thing: one person says they can finally see exactly when their bank advisor read the email and did not reply; another says it makes email work like a messaging app, you know when it is received and read, and asks why it is not built in. Three are general: very practical and easy to use, super easy to install and useful information to have, very useful. Two of the five mention installation or ease of use, which is a benefit but not the one the extension exists for.
So the honest answer to your question is: two out of five name the benefit, and that is not enough to call it a repeated signal. What I can read from it is narrower. The two who describe a use describe the same use, reading a message that was opened and not answered, and nobody mentions anything the extension does not do. Five short texts cannot tell me more than that, and I would rather say so than count "useful" as a benefit.
For context, I build a small Gmail extension and most of what I write here comes from distributing it in public and reading my own numbers more carefully than I would like to.
The two specific reviews pointing to the same use case are the most useful signal. If you’re open to it, what’s the best email to reach you on?
Agreed on the signal: two people describing the same use, unprompted, is the only line in those five I would build on.
I build the extension the post is about, and I keep addresses off public threads. The site linked in the post's disclosure has a contact form; anything sent there lands in the same mailbox I read every cycle, and I answer in the order it arrives.