The hardest part of my product isn't the tech — it's deciding how much to explain

The "proportional to surprise" heuristic is really sharp — you've basically found that explanation isn't about completeness, it's about earning trust at exactly the moment trust is being tested. Obvious answers don't need defending; counterintuitive ones do.

I ran into a similar tension with an experiment feature in my app — comparing hydration habits over a week. Early version explained the methodology every time (sample size, comparison logic). Nobody cared until the result contradicted what they expected, and then they wanted the entire reasoning, immediately. Ended up doing something close to your model: quiet by default, but the moment a result surprises the baseline, that's exactly when we surface "here's why this isn't what you expected."

Your test — can a new user explain WHY unprompted — is the right bar. Curious whether you're measuring that through post-recommendation surveys, or inferring it some other way (e.g. whether they override the suggestion less often over time)?

Your proportional rule makes sense, but I'd make the trigger clear in the wording too. If a recommendation surprises someone, lead with the consequence: "Use the card with fewer rewards here because the other choice could hurt your credit." Then add a short "show me the math" option for anyone who wants more detail. We ran into something similar with DictaFlow's cleanup. People want clean text first, but they also need a clear signal when AI changes more than punctuation. The explanation is worth adding when you're asking someone to go against their instinct.

great

The thing I would change is your success metric. Whether a new user can explain why the tool picked a card matters less than whether they actually use the card it told them to, because the explanation exists to change behavior, not to be recalled. At SocialPost.ai we found users never wanted the reasoning behind an output, they wanted one concrete number that made it believable, so "this saved you $40" outperformed every factor breakdown we tried.

I’m dealing with a similar challenge while building a browser debugging tool. What seems to work best is showing the result first, then the strongest evidence, while keeping the full technical trace optional. I really like your idea of asking users to explain the recommendation back to you. Clicking “more details” shows curiosity, but it doesn’t necessarily mean they understood anything. I’d also be interested in whether users can tell the difference between something the product knows and something it’s only suggesting.

The line I use is the cost of being wrong. In a pre-quote workflow, an obvious-looking draft can hide assumptions that change the price, so I lead with the proposed action, then surface only the missing fact that would change it: ‘Booking requires X integration; confirm which system before you quote.’ Everything else stays behind a detail view. I would add one behavioural test to your teach-back measure: can the user spot and correct a wrong input before following the recommendation? If they can repeat the explanation but cannot change the decision when the premise is wrong, the interface explains well but still protects badly.

We have seen the same with analytics answers: users only drill into the why when the result contradicts their mental model. Proportional-to-surprise works, but surprise is hard to predict, so we made depth on-demand and treat every expand as a signal of where trust is missing.

Acceptance alone can be misleading here. A clear explanation might make people follow a confident but wrong recommendation more readily.

I’d test calibration with paired cases: one counterintuitive recommendation that is correct, and one equally plausible recommendation built from a deliberately stale or incorrect input. After the one-line why, ask users what drove the choice and whether they would follow or override it.

The explanation works if it increases justified acceptance and justified rejection. If it only raises compliance, you have made the system more persuasive, not more trustworthy.

The "recommends AGAINST the obvious choice" moment is the one I keep failing at too, so that is the part I want to push on.

I run a price tracker. Its highest-value output is telling someone a discount is not real, or that today is a bad day to buy. Same shape as your problem: the answer people did not come for needs more justification than the answer they did.

What helped me was splitting two things I used to show as one blob.

1. Why the answer is what it is. One line, always visible. Yours might be "the 3% category cap resets next month."

2. Why you should believe me. Not reasoning, provenance. For me that is "we read this price on 31 separate days, here are the dates." It is not the logic, it is the receipt.

When I showed reasoning, people argued with it. When I showed dated observations, they stopped, because there is nothing there to argue with. Either we wrote it down or we did not.

So my guess about trust going down: the extra information was more reasoning, not more provenance. Reasoning invites the reader to check your work. Provenance invites them to check reality, which is a much cheaper ask.

This is a different lever from the one further up this thread about making the instruction editable, and I think both work for the same reason - they move the thing being trusted away from your judgement and onto something the user can verify or control.

One thing that surprised me: I also publish what I cannot judge. Listings without enough history get labelled "not enough data" and get no verdict at all. I expected that to read as weak. It does the opposite.

For the against-the-grain moment specifically, does your one line change shape? Mine does. For a normal answer it states a fact. For a contrarian one it states a comparison: "cheapest we recorded is $1.22, today is $1.31." The comparison does the persuading, not the reasoning.

There is a version of this where the explanation moves from the output to the input.

My product is a desktop utility with one-shot AI actions, rewrite this, check that. People did not want reasoning attached to results. What they wanted was to know what the tool had been told to do. So every action's instructions are readable and editable: open the action, read the exact instruction, change it if you disagree. That did more for trust than anything I attached to outputs, because it turns "do I believe this reasoning" into "these are my instructions being followed". It also dissolves the override question this thread keeps circling. A user who dislikes an output does not argue with an explanation, they edit the instruction and run it again.

The second lever is refusal. When a configured model cannot do a job properly, mine says so and names the fix rather than returning a degraded attempt. That refusal buys more credibility than most successes.

Your criteria are presumably your moat, so full disclosure may not transfer. Is there a slice, weightings on fees versus rewards maybe, that you could hand users to edit, so disagreement becomes input instead of churn?

I think you're onto something with proportional reasoning. The explanation probably shouldn't just answer “why this card?” it should answer “why should I trust this recommendation?”

The interesting case is exactly the one you mentioned: when the recommendation goes against someone's intuition. That's probably where a short explanation has the highest value.

I'd also test whether the explanation needs to change depending on the user's confidence. A new user may need more context, while a returning user who's already learned how the product thinks may want almost none.

Your measure of whether someone can explain the decision back to you is a much better signal than whether they clicked “show more.”

A short explanation would help me if it named the detail about my situation that drove the choice. If that detail is out of date, being able to correct it would help me trust the next recommendation.

Thank you. This is better than I hoped. I thought I was alone in this, but— three different domains
(logistics, SEO, checkout) all landing on the same principle
is a strong signal that it's real.

Genuinely grateful for this — you've all sharpened how I'm
thinking about it now and to keep moving forward.

When I ran into this building logistics and compliance tools, showing the full calculation logic usually overwhelmed users and actually created doubt. In my experience, people only want deep reasoning when the recommendation conflicts with their intuition, like picking a route that seems slower but avoids customs delays. What worked was staying quiet on predictable answers, and only surfacing a single clear 'why' when disagreeing with the user's default choice, leaving full breakdowns behind an optional click.

We build something with the same shape, an SEO tool whose entire premise is showing what to fix first rather than four hundred problems, and your trust finding matches ours. More reasoning reads as less certainty, because a full factor list invites the reader to audit you and an audit they cannot finish feels like hedging.

On the tension you have not solved: I think the error is treating explanation as a constant. It should scale with how surprising the recommendation is. When the answer matches what someone already expected, the why is noise. When it contradicts them, the why is the product, because confidence on its own just reads as a mistake.

The useful part is that this is computable. You already know the obvious choice, it is the card they would have reached for. So the system knows when it is disagreeing with the user, and that is your signal to expand, rather than one fixed line for everything.

The counterintuitive recommendations seem like the real test of the explanation layer. When the tool goes against a user’s obvious card choice, does a concise “why” actually increase acceptance of the recommendation, or just make the decision easier to understand afterward?

Your explain-back test measures comprehension, but I would pair it with a behavior test. For a counterintuitive recommendation, measure whether the user follows it before seeing the explanation, whether the concise reason changes the choice, and whether opening the full math changes it again. That separates understanding from actual trust. Have you seen whether the one-line explanation changes which card testers choose?

The trust going down when you showed more reasoning is really interesting. I wonder if the goal is less “explain the decision” and more “give me enough evidence to feel comfortable with it.” Especially when the recommendation goes against what I expected.

This matches what I keep running into. Full transparency feels honest, then people bounce because they have to do the thinking twice.

Your proportional rule is the useful bit. Default quiet when the answer is obvious, spend the words when you are asking someone to trust a counterintuitive call. That is closer to how people already take advice from a person than dumping the whole model on them.

The unprompted "can they explain why" test is a good bar. If they cannot say it back in one sentence, the UI failed even if the logic was perfect.