Weekend reading: The Wisdom of Crowds vs The Black Swan

By Shaun Hendy 25/06/2010 2

The Black Swan by Nicholas Taleb (Random House, 2007, 366 pages).
The Wisdom of Crowds by James Surowiecki (Anchor, 2005, 336 pages).

Do markets work?  Writing from perspectives that precede the recent financial crisis, Nicholas Taleb, Wall Street trader turned academic, and James Surowiecki, economics correspondent for the New Yorker, offer insights into the strengths and weaknesses of market forces. 

Surowiecki opens with an account of a pre-industrial version of that school gala staple, guess the number of jelly beans in the jar.  While Surowiecki’s seventeenth century protagonists are interested in the weight of an Ox, rather than a quantity of sweets, he notes that the average guess of a crowd in either case is typically very close to the actual answer.  The first half of Surweicki’s book is devoted to understanding under what conditions crowds are able to perform such feats of wisdom. 

What has this got to do with markets?  Well, for a market to allocate goods efficiently, the agents trading in such a market must act rationally to maximise their own well being.  However, experiment after experiment has shown that people do not always behave rationally, and while there is only one way to act rationally, while there are a myriad of ways to act irrationally.

Surowiecki’s wise crowds offer a possible way out: in the same way that the average guess of a crowd can get close to the true weight of the ox, maybe a crowd of irrational agents can, on average, behave rationally.      

In the second half of the Wisdom of Crowds, Surowiecki investigates under what conditions groups might fail to be wise.  For instance, he suggests that individuals in a crowd must be able to make their decisions independently, without influencing others.  If this condition is not satisfied, bubbles can grow and pop in a market as individuals follow others rather than make their own decisions.     

In fact, it is the failure of crowds, and in particular markets, that concerns Nicholas Taleb.  In The Black Swan, Taleb is scathing of the modern financial system that systematically underestimates the risks of rare but high-impact events.  For instance, the models used by traders to value complex financial products assume that stock prices will fluctuate as if they were undergoing a ‘random’ walk, precluding the possibility of rare but high-impact events.

How do such high-impact events occur? Under Surowiecki’s conditions, where the decisions of a large ‘crowd’ of traders are independent, the fluctuations in stock prices may be well approximated by a random walk as the financial models assume.  The crowd acts as if wise and markets operate efficiently.

However, if the decisions of traders become correlated, if traders start to follows the decisions of others, then the central theorem no longer applies — stock prices will cease to fluctuate according to a random walk.  Under such conditions bubbles can form, leading to rare but high-impact collapses such as that experienced in the recent financial crisis. 

Indeed, stock market data shows that just prior to, and during a crash, the decisions of traders become highly correlated.  Surowiecki devotes several chapters to discussing the consequences of such ’group-think’, including an in depth post mortem of the dangerous consensus that developed within NASA’s management team that lead to the Challenger disaster.

Ultimately, Taleb is concerned with more than just bubbles and group-think. His ‘Black Swans’ include any low-probability but high-impact event, including the arrival of far-reaching innovations like the internet, wars or terrorist attacks. As I have discussed previously, such rare events seem to be important for progress in science; the ‘Black Swan’ of science lead to scientific revolutions.

Can crowds ever be wise in the face of ‘Black Swans’?  Taleb is pessimistic; Surowiecki would offer a conditional maybe.  Many of us (although not all) would agree that stockpiling supplies of was a prudent response to the possibility of a high mortality flu pandemic that threatened in the winter of 2009.  Yet to perform a cost-benefit analysis of this precaution of the type used by financial markets to hedge against risk is clearly impossible.  If one were charitable, one might say that governments fell back on a precautionary principle; Taleb would argue that such principles could be usefully applied to markets.             

Both books offer insights into the way people interact and make decisions.  Despite being written prior to the events of late 2008, both offer insight into the recent financial crisis.  As Taleb notes, we have a compulsion to invent post hoc narratives for each ‘Black Swan’ and the recent crisis is no exception.  It is quite possible a reading of these two books will leave you with a deeper insight into recent events than anything that has been written in their aftermath.    

2 Responses to “Weekend reading: The Wisdom of Crowds vs The Black Swan”