In his book, The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations (title long enough for you?), James Surowiecki argues that decisions made by a crowd are generally better than those made by any single individual in the group.

Crowd

Seems like a lot of theoretical hogwash until you see this thesis put to action in the real world via a prediction market. A prediction market (also called a decision market) is, as its name implies, a market created specifically to predict the likelihood of a specific outcome. They are most successful when participants have something invested in their decisions. Money often does the trick, but is not necessary.

The Hollywood Stock Exchange, a prediction market focused around the film industry, demonstrated the potential accuracy of such markets when it went 32 for 39 in predicting 2006’s Oscar Nominees (in the major categories that are the only ones people care about). They didn’t do so bad in 2007 either.

It’s no surprise then that there is a lot of research going on in predictive markets. Preliminary results show they seem to work very well when properly structured.

Contrast the success of prediction markets to another decision making process that involves a group. A recent comment by a reader got me thinking on this subject as he described one of the symptoms of this process.

If you try to satisfy all parties, you’ll end up with mediocre product that does not satisfy everybody.

An even more strong way to state this is to say that trying to satisfy everyone leaves almost everyone unsatisfied.

This is one typical result of a decision making process known as Groupthink. Another colorful term used to describe it is decision by committee. These terms do not, by any means, have a positive connotation.

Group Think 

There seems to be a paradox here between the power of markets and the ineptitude of Group Think. Why do these two somewhat similar means of decision making have such a wide variance in accuracy?

Perhaps counter-intuitively, it has a lot to do with the coercive effects of seeking consensus in decision making. Group think is often hamstrung by seeking consensus whereas participants in an effective market are largely independent of one another.

Before I continue, let me head off that insightful comment on how I am trying to promote anarchy and save your fingers the pain of some typing by pointing out that this is not an indictment of all consensus driven decision making. In many cases, consensus is absolutely required. It doesn’t help if a decision is optimal but nobody is willing to participate in the result of the decision. This sort of decision making is especially essential in one particular form of decision making - negotiations,which is well covered by the book Getting to Yes: Negotiating Agreement Without Giving In.

The problem lies in applying negotiation to make decisions that should not be made by consensus. At least not without applying non-consensus based fact gathering first, so that negotiations can occur using data gathered in a dispassionate manner.

So what makes groupthink particularly inadequate? Social psychologist Irving Janis identified the following eight symptoms of groupthink that hamstring good decision making:

  1. Illusion of invulnerability – Creates excessive optimism that encourages taking extreme risks.
  2. Collective rationalization – Members discount warnings and do not reconsider their assumptions.
  3. Belief in inherent morality – Members believe in the rightness of their cause and therefore ignore the ethical or moral consequences of their decisions.
  4. Stereotyped views of out-groups – Negative views of “enemy” make effective responses to conflict seem unnecessary.
  5. Direct pressure on dissenters – Members are under pressure not to express arguments against any of the group’s views.
  6. Self-censorship – Doubts and deviations from the perceived group consensus are not expressed.
  7. Illusion of unanimity – The majority view and judgments are assumed to be unanimous.
  8. Self-appointed ‘mindguards’ – Members protect the group and the leader from information that is problematic or contradictory to the group’s cohesiveness, view, and/or decisions.

One symptom not listed here, but probably fits in with #6 is the fear of causing offense.

One theme in common in the list is that many of these symptoms are the result of seeking some form of consensus with others within the group. The net effect is that all of these symptoms provide incentives not to deviate from the group.

Prediction Markets counter these symptoms through independence and proper incentive. Because the participants in a market are not directly working together, there is no way for peer pressure to burrow itself into the decision making process. There is no fear of offending others in such a market. Putting something valuable (such as money) on the line (or having a huge investment in the correct outcome, such as predicting disaster) has an eye-opening habit of making people truthful and helps to avoid self-censorship etc…

Another necessary factor in the success of markets, as pointed out in an insightful comment to this post, is diversity. Markets and committees that lack diversity of viewpoint fail to take advantage of all available information in a meaningful way and succumb to a form of tunnel vision. This type of phenomena is very evident in a leader who surrounds him/herself with sycophants and thus makes decisions based on what they want to hear rather than fact.

Prediction markets are not perfect and are not suitable for all decisions. In fact, they are probably not suitable for most decisions as the cost to set up a market isn’t justified for everyday decisions like whether you should get the bottle of beer or the pitcher of sangria.

However, understanding what makes predictive markets work and what the symptoms of groupthink look like can be a great benefit the next time you are making a decision within a group and start to see groupthink emerging.