The New York Times
March 11, 2006, page B1

The Future Divined by the Crowd

By JOE NOCERA

LAST Sunday was Oscar night, which of course meant movie stars, glamour, Jon Stewart's jokes and - drum roll, pleaase! - Michael J. Mauboussin's annual economics experiment. Or rather, his annual economics eye-opener.

Mr. Mauboussin is a well-known Wall Street strategist, now with Legg Mason, who gained a following for an offbeat, scholarly newsletter called The Consilient Observer, in which he tackled such diverse subjects as what investors can learn from Tupperware parties and why stress can lower investment performance. His work is sometimes dense, but always original.

Since 1993, Mr. Mauboussin has also been an adjunct professor at Columbia Business School. Every year, as the Academy Awards approach, he asks his students to vote for the winners in 12 categories, not just the biggies but some relatively obscure ones, like best film editing and best art direction. Then, after the Oscars have been awarded, he tallies the results and compares the students' predictions with the winners.

This year, the pick that got the most votes - the consensus pick,, he calls it - turned out to be right in 9 of the 12 categories, including, amaziingly enough, film editing and art direction. And yet, of the 47 students who participated, only one matched the accuracy of the consensus. None did better, and most did much worse; according to Mr. Mauboussin, the average number of correct answers per ballot this year was only 4.1. "It has never failed," he said. "The consensus invariably does much better than the average student."

The point of Mr. Mauboussin's little experiment is to illustrate the power of so-called prediction markets, in which groups of people guess or bet on something, with the results aggregated into a consensus. Prediction markets, while not perfect, are surprisingly accurate - certainly more accuraate than individual experts or polls, research has found.

The granddaddy of prediction markets, the Iowa Electronic Market, which the University of Iowa has run since the late 1980's, allows people to make election predictions. The consensus almost always beats the polling data; in the last presidential election, for instance, it not only steadfastly predicted a Bush victory, but came within 1.1 percentage point of the actual result.

PREDICTION markets aren't just for curious academics anymore; in the last few years, there has been an explosion of interest. A handful of dot-coms, with names like Hollywood Stock Exchange, hedgestreet.com and Newsfutures, offer people the chance to predict everything from how much a new movie will gross in its first month to whether gasoline prices will rise. (Hedgestreet.com's slogan is: "It's your economy. Trade it.")

Some sites plan to permit Wall Street traders to begin trading on such questions, much in the way commodities traders bet on the future price of soybeans. Perhaps most intriguing, companies have begun experimenting with prediction markets as a new kind of forecasting tool. Though it's still a little early to say for sure, prediction markets could wind up changing the ways companies go about making decisions.

There are two recent events that gave rise to this new interest. The first was a quasi-disaster. In 2003, John M. Poindexter, who was heading up a small research group inside the Pentagon, tried to create a public prediction market for geopolitical events, including possible terrorist attacks. But the criticism was swift - Mr. Poindexter was accused of creating "terrorism futurees" - and the program was killed. Still, the publicity got a lot of businesspeople thinking about the utility of prediction markets. "The Pentagon," chuckled Alexander Costakis, who runs the Hollywood Stock Exchange, "put it on the map."

The second event was the publication, in 2004, of "The Wisdom of Crowds," by James Surowiecki, a business columnist with The New Yorker. Mr. Surowiecki's title was purposely meant to echo - and contradict - Charlesles Mackay's famous 1841 work, "Extraordinary Popular Delusions and the Madness of Crowds," which Mr. Surowiecki describes, correctly, as "an endlessly entertaining chronicle of mass manias and collective follies." No one argues that crowds don't sometimes go mad; from Dutch tulips to dot-com bubbles, the evidence is pretty irrefutable. But Mr. Surowiecki set out to show that far more often, the crowd got it right.

His book is filled with examples of the power of "collective intelligence," as he likes to call it. For instance, Mr. Surowiecki describes a case in which a naval officer located a sunken submarine by asking people with bits of specific expertise to take their best guess. Their collective answer turned out to be within a few hundred yards of the submarine's location.

Another example came on Jan. 28, 1986, the day the Challenger shuttle exploded. All the stocks of rocket makers were down that day, of course, but by far the hardest hit was Morton Thiokol - which made the faulty part that was responsible for the disaster. The stock market is nothing if not a prediction market writ large. But how could the market possibly have known about Morton Thiokol's culpability on the first day? No information had been revealed. There was no excessive insider selling. There wasn't even any media speculation as to the culprit. Yet, somehow the market knew.

Two of the people who read Mr. Surowiecki's book were Eric E. Schmidt, the chief executive of Google, and Bo Cowgill, a young Google executive. Being a forward-thinking geek, Mr. Cowgill decided to set up some prediction markets at Google, and management gave him the go-ahead. He's been doing it for a year now.

The Google market is internal only - the voters are all Google emmployees. To avoid insider-trading problems, Mr. Cowgill stays away from any stock price or quarterly earnings questions. And to entice people to participate he gives away T-shirts. And he asks pretty useful questions: "When will a product launch? How much will a particular feature be used? How many full-time people will accept jobs at Google in the next quarter?" he said.

The utility of such forecasting is clear: it is important for companies to know when a product will be introduced, and tapping into the collective intelligence of employees to predict the date is at least as good a way to find out as talking to the project manager. In fact, it is probably a better way; at a recent conference, Mr. Cowgill presented a series of slides showing that Google's internal market was amazingly accurate, especially in markets in which lots of people participated.

Google is hardly the only company doing this. Eli Lilly has used internal markets to forecast drug development. Hewlett-Packard has become a fan. Emile Servan-Schreiber, the chief executive of Newsfutures, sells software to companies that helps them set up prediction markets. His first customer, in 2003, was Eli Lilly. Now, he says, he is working with 12 to 15 companies.

Yahoo has gone about it a different way, setting up a public market called Tech Buzz. It allows people to make bets on which technologies will be popular in the future; the point, for Yahoo, is to get out in front of popular coming search terms. "The more we know about trends, the better we can adapt our search services," said David Pennock, who runs the predictive markets for Yahoo.

Still, it's hard to know for sure whether predictive markets will turn out to be the next big thing. "There is a lot of interest, but we're still missing the big success story," said Robin D. Hanson, a professor at George Mason University and a reigning expert in the field. "People are trying it on small, cute things. We haven't seen a company that's made $1 million using this stuff."

Indeed, Mr. Cowgill told me that he really had no idea how Google's management used the forecasts generated by the company's prediction markets. Mr. Servan-Schreiber said the companies that were showing interest were "early adopters." One of his clients, Corning, has set up a prediction market to gauge demand for L.C.D. screens, which it manufactures. "Corning has billions on the line, and it needs to be able to assess demand," Mr. Servan-Schreiber said. Maybe that experiment will turn out to be the success story Mr. Hanson is looking for. (Corning, alas, declined to comment.)

The real problem for prediction markets is that the notion is so counterintuitive. We're used to the idea that experts, full of specialized knowledge, will get it right far more often than a crowd of people with no special insight. The thought that the reverse is actually closer to the truth is hard to get one's head around.

"I'm not sure why it works," Mr. Mauboussin said. But then he offered a theory. "All of us walk around with a little information and a substantial error term. And when we aggregate our results, the errors tend to cancel each other out and what is distilled is pure information."

Well, maybe. Myself, I'm going with the "Shakespeare in Love" theory. You recall, surely, the producer played by Geoffrey Rush. Whenever he was asked how it was that a play that seemed to be such a hopeless muddle during rehearsal was transformed into a gem on opening night, he had a stock reply, one that seems to describe prediction markets perfectly:

"It's a miracle."