Anecdotes May Be Charming, But Are They Reliable?

Emre Soyer
4 min readApr 21, 2022

--

ulleo/Pixabay

How can you make others pay attention to, understand, and then possibly share your ideas?

It’s not easy. There are a lot of ideas, but not enough time and attention. The topic and the product should be attractive. But that’s not enough. They should also be presented in a fascinating way.

Presenters who make a boring subject interesting stand to win over those who blandly describe an interesting subject.

A common way to grab attention is to rely on alluring anecdotes. Unique events with captivating details make gripping stories. Anecdotes get frequently used to prove one’s point in business books, TED talks, advertisements, YouTube videos, or tweets. They can involve impressive successes, catastrophic failures, or some other settings that excite the audience.

But when does anecdotal evidence help decisions? When does it not?

Costs and Benefits of Relying on Anecdotes

Anecdotes offer clear advantages to presenters of ideas:

  • They make the message memorable, inspiring, relatable.
  • Presenters don’t have to lie: The information may be selected but accurate.
  • There’s no shortage of impressive anecdotes. A skilled presenter can use an event from one domain to make a point about an issue in another domain. In fact, the larger the distance between the two domains, the more interesting the story arguably becomes.

There are some benefits also for decision makers, who apply those ideas:

  • They can remember the message easily and use it as an inspiration.
  • They can learn about what’s possible, be it an extreme case of success or failure.
  • They can use anecdotes to justify their decisions to others.

But anecdotes can subtly get in the way of competent decision making:

  • The more unique an event, the less it would generalize. Hence, the more extraordinary an anecdote, the less relevant it would be for decisions.
  • Anecdotes tend to underestimate the uncertainty involved in decisions. Anecdotal evidence can attribute many chance events to specific behavior, which can lead decision makers to invest in costly strategies that don’t yield the desired outcomes.

Overall, presenters of ideas tend to gain more from anecdotes than the decision makers that apply those ideas.

Using Anecdotes as Signals

Anecdotes are data points and should be treated as such. But telling of anecdotes transforms them into something more than plain data. Hence, decision makers should not make them the basis of their decisions, but take them as signals to ask some data-related questions:

  1. How representative is the anecdote?

If the story involves an outlier, a one-of-a-kind situation, a considerable deviation from the norm, then the insights would be about what can happen in the limit of a particular setting. The lessons wouldn’t translate well to instances where the situation and the aims are significantly different. Distinguishing between representative and unrepresentative anecdotes would save decision makers time and energy.

For example, you read a fascinating story about an eccentric billionaire or a world-renowned athlete. How many people, who wish to be like them, actually end up being like them? Are their situation and aims similar to yours? If the answers tend toward “not many” and “no,” then the anecdote is not really representative.

2. What does the statistical evidence say?

Some anecdotes may claim to be representative: They may involve a detailed account of one of many cases that happen in the same domain, and the situation and aims can match those of the decision maker. But instead of inspiring immediate action, such anecdotal evidence should trigger a check that is based on wider data, to better understand the underlying phenomena and see which lessons are more valid and general than others.

For example, there may be a lot of companies that fail in a given domain. A detailed account of one of these can help a decision maker pay attention to the phenomenon. But this initial interest should spark a curiosity for statistical analyses within the domain, rather than the specifics of the particular case.

3. Who is the presenter?

Decision makers should beware of presenters who constantly opt for unrepresentative anecdotes to sell ideas. The benefits of this practice are highly asymmetric: Presenters get the attention they seek, while the insights tend to be unreliable for decisions.

As for potentially representative anecdotes, decision makers can see if the presenters make an effort to provide further relevant data. It may be acceptable to lure people in with the charm of an anecdote. But a valuable consultant or advisor would have to back the claim with reliable statistics.

Ultimately, decision makers who use anecdotes as signals rather than guidance would incentivize presenters of ideas to feature representative examples and reliable data to make their points, which would further improve future decisions.

PS. We resisted the urge to use any anecdotes to make our points in this post :)

A version also published on Psychology Today

--

--

Emre Soyer

behavioral scientist, co-author of The Myth of Experience