A crisis of overconfidence

    Are humans hard-wired for overconfidence - and does that help explain the financial crisis?

    By David Ewing Duncan

    Thousands of years ago, our ancestors fanned out from Africa, convinced that something better lay over the next hill. They brazenly faced down a dangerous world of beasts much larger and fiercer than they were, and their boldness as hunters probably contributed to some species -- saber-toothed cats and the mammoth, for instance -- going extinct.

    While there may appear to be little similarity between those prehistoric times and today, it turns out that the same audacity that made us effective predators also may have driven the relentless advance of Homo sapiens from caves to condos and from primitive haggling to hedge funds.

    Arguably, it's our overconfidence that's kept us going, and now a new paper titled "The Evolution of Overconfidence" from researchers in Scotland and San Diego suggests that this distinctly human trait may be vital to our success.

    But this predilection to overestimate one's abilities, adversaries, and outcomes also has a dark side, since not everyone can be the best merely by thinking they are. Take drivers: Over 90% believe they are above average at maneuvering an automobile. Or college professors: 94% of them claim they are above average teachers.

    While some may certainly be assessing themselves accurately, that simply can't be true of all of them. But their overconfidence can still be an asset, boosting resolve, ambition, morale, and tenacity, even for the average driver or professor.

    Yet because overconfidence by definition makes it difficult to accurately evaluate risk, it has also contributed to disasters that range from getting eaten by long-fanged felines 15,000 years ago to global warming, the Vietnam War, and the catastrophic response to Katrina.

    In the recent financial collapse, we had an epidemic of overconfidence in just about every quarter (realtors, homebuyers, investors, bankers, lawmakers, and the list goes on). In the lead-up to the crash a year ago, the warning signs were clear as debt piled-up, housing prices and sales began to slide, and market upswings became detached from the real economy.

    Still, as late as July 2008, Treasury Secretary Henry Paulson was saying: "Our banking system is a safe and a sound one."

    The human proclivity to overconfidence has led experimental psychologists to puzzle over how a trait that can be so destructive remains so potent in our gene pool.

    One explanation comes from the authors of "The Evolution of Overconfidence," evolutionary biologist Dominic Johnson of the University of Edinburgh and political scientist James Fowler of the University of California at San Diego. Using a mathematical model that compares the overconfident with the more tempered, they conclude that the audacious usually reap more rewards than those who rationally assess risk.

    Johnson and Fowler calculate that when the potential upside is at least twice that of the cost to compete, overconfidence is the best strategy. "This is because overconfident individuals make more challenges when there is uncertainty about the strength of opponents (and thus the outcome of conflicts)," write Johnson and Fowler, "while less confident individuals shy away from many conflicts they would win."

    If true, this contradicts the prevailing economic and social theory of the last 30 or 40 years that individuals and societies tend to make rational choices, balancing costs and benefits in a reasonable manner -- whether they are buying a new car or going to war.

    The Johnson-Fowler scenario may also tell us why the rational choice theory seems to have failed so spectacularly in the past year or two. Seen through their model, financial traders were taking ever more fantastic risks -- usually with other people's money -- apparently believing they would win big. Like drivers, most assumed they would be not only above average, but way above average.

    But as we've learned, there's an outer limit that seems to occur when overconfidence becomes too extreme and the ratio of costs to rewards becomes dangerous or potentially suicidal. Ancient Greeks called this brand of overconfidence "hubris" -- an excessive pride, confidence, and ambition that Greek myths and stories constantly warn against for both mortals and gods. (There were no hedge fund traders back then.)

    But there's no formula to predict the moment when overconfidence becomes hubris, and optimization reels into disaster. Had there been, our distant ancestors might have avoided killing off every last one of those saber-toothed cats -- and in today's turbulent (and self-inflicted) economic times, we might have some sense of just how brash we need to be to repair the damage done by our own brashness.