Wisdom of Crowds Conditions
Make groups smarter by protecting diversity, aggregation, and incentives
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 6
- Confidence
- 98%
A crowd becomes intelligent only under specific conditions. First, the participants need diverse perspectives and decision rules so their errors are not all pointed in the same direction. Second, the organization needs a mechanism that actually extracts and combines their information. Third, participants need incentives that reward being right and penalize being wrong. Independent estimates can then cancel one another's errors, and larger samples generally improve collective accuracy. The mechanism fails when one condition breaks, especially diversity: social conformity makes views correlate, turning independent judgments into a uniformly positive or negative crowd. Before trusting consensus, assess the process that produced it rather than treating agreement itself as evidence.
Origin
Michael Mauboussin connects James Surowiecki's wisdom-of-crowds conditions to markets, classroom estimation exercises, and diversity breakdowns in The Tim Ferriss Show.
Core principles
- 01Independent perspectives create useful error diversity
- 02Information has no value unless it is aggregated
- 03Consequences improve the quality of judgments
- 04Correlated views can turn a wise crowd into a mad one
How to run it
- 1
Define the judgment
Choose an estimation or forecasting problem that can benefit from multiple perspectives. Separate it from tasks where a single qualified specialist has a clear advantage.
Pro tip Ask whether participants can reasonably form estimates without copying one another.
Watch out A crowd is not automatically better than an expert.
- 2
Build cognitive diversity
Include people who use different perspectives, mental models, training, and decision rules. Do not treat visible demographic variation as proof of cognitive variation.
Pro tip Seek smart participants who approach the problem differently.
Watch out People who look different may still have nearly identical training and models.
- 3
Preserve independence
Have participants make their judgments before hearing the group's views. This prevents conformity from correlating otherwise useful errors.
Pro tip Use private written estimates before any group discussion.
Watch out Uniform enthusiasm or fear is a sign that diversity may have broken down.
- 4
Aggregate the information
Combine the independent inputs with an appropriate mechanism, such as an average, median, modal choice, or market price. Match the mechanism to the form of the question.
Pro tip Use the median when extreme estimates could distort an average.
Watch out Information left in people's heads contributes nothing to the collective answer.
- 5
Align incentives
Create rewards for accuracy and penalties for error. Use money, reputation, scoring, or another consequence appropriate to the setting.
Pro tip Track results so participants receive feedback on calibration.
Watch out Incentives that reward agreement can destroy independence.
- 6
Audit the conditions
Before trusting the collective result, check diversity, aggregation, and incentives again. Treat a violation of any condition as a reason to reduce confidence.
Pro tip Pay special attention to diversity because Mauboussin identifies it as the condition most often violated.
Watch out A large sample cannot repair systematically skewed judgments.
In the wild
At a fair, Francis Galton collected hundreds of guesses about an ox's weight. Although individual guesses varied, the crowd's average or median came within 1% of the actual weight, illustrating how independent errors can cancel when many estimates are aggregated.
→ The collective estimate was much more accurate than Galton expected.
Mauboussin asks a class of roughly 60 or 70 students to estimate the contents of a jar. The collective answer is typically off by 2% to 10%, while a randomly selected individual is usually off by about 50%.
→ Aggregating diverse estimates substantially improves accuracy.
Triple Crown excitement attracted casual bettors who wanted a winning souvenir ticket. Their correlated enthusiasm pushed Big Brown's implied probability to 77%, despite historical base rates and speed figures suggesting a much lower probability.
→ The diversity breakdown produced poor market odds, and Big Brown finished last.
Common mistakes
Counting appearances instead of cognition
Social-category diversity is measurable, but cognitive diversity is the component Mauboussin says the problem-solving literature identifies as crucial.
Letting the group anchor itself
Discussion before independent estimates can correlate views and remove the error diversity that makes aggregation useful.
Using a crowd for an expert task
A plumber or mathematician is more efficient than a mixed crowd when the problem has a clear specialist solution.
Is it for you?
Best for
It is best for estimation, forecasting, and complex questions where multiple informed perspectives can be combined.
Not ideal for
It is not ideal for tasks with one demonstrably skilled expert and a clear technical answer.
From the transcript
“the wisdom of crowd says crowds are wise when three conditions are in place a we have diversity of the underlying agents or heterogeneity”
“second is an appropriate aggregation mechanism”
“And then the third is incentives, which are rewards for being right and penalties for being wrong.”
From the episode
#659: Michael Mauboussin — How Great Investors Make Decisions, Harnessing The Wisdom (vs. Madness) of Crowds, Lessons from Race Horses, and More
Michael Mauboussin