There has been a lot of hand wringing the past couple days about how the prediction markets "got it wrong" when it came to the Democratic primary in New Hampshire. Since prediction markets are all about probabilities and not absolutes, using "right" vs. "wrong" in any dialogue and dismissing results as "BS" shows a lack of understanding about how to interpret results.
For example, in the CNN marketplace, Hillary Clinton's closing position in the New Hampshire primary market was $16.40. Barack Obama's closing price was $82.60. The easiest way to interpret these results is to say:
"If we flipped a coin 100 times, Hillary would win 16 times. Barack would win 83 times." So yes, traders were saying there was a high probability Obama would win, but it was never an absolute. Thinking about prediction market results in the context of coin flips is a useful way of staying grounded when interpreting what the market is saying.
If Obama had won we would never be having this dialogue. That said, it's probably useful to come to the forefront now as the awareness of prediction markets widens and more and more businesses consider using prediction markets internally and externally. Unfortunately in the media, talking about chance and probability is a more nuanced discussion than "right" vs. "wrong" which means the discussion is not likely to get the treatment it deserves. But as we tell our existing and potential clients who are considering Inkling for business purposes, the most important role of prediction markets is to give insights in to what people are thinking. Their perceived "accuracy" or ability to predict events occurring in the future is often a tangible benefit proven countless times, but should never be over-promised as a magic crystal ball.
If you do want to judge an Inkling marketplace on how well it's "performing," you can examine all the cases where the marketplace said there was an 80-90% probability of an event occurring, then look at how many times it actually did occur. If the event did occur a vast majority of the time, the marketplace is performing well. Similarly, if traders say there is only a 10-20% chance an event will occur, and indeed those answers are occurring infrequently, the marketplace should be considered successful.
We're in the process of compiling some data and will have a subsequent post shortly looking at these exact figures across all Inkling marketplaces.
As for the New Hampshire primaries, this is the best article we've seen so far about the role of prediction markets:
http://www.alleyinsider.com/2008/01/prediction-markets-blow-it-nh-primaries-confound-expectations.html
Also, the last time this happened on a much smaller scale was the Democratic takeover of Congress in 2006. Bo Cowgill and Cass Sunstein wrote an excellent article in the New Republic addressing certain people's misinterpretation about the market results (TNR took the article down but it's available here):
http://groups.google.com/group/Prediction-Markets/msg/3514dbddf76636f6
1 comment:
I'm more upset about getting it wrong on the "who will win more 'primaries' on super tuesday" question which originally paid out to Obama (who won in 13 states) but then was changed and paid out to Clinton, presumably because some of the contests were caucuses. You could argue that "primary" is a term that refers to caucuses, conventions, direct voting, rock-paper-scissors, or any other electoral method used by a party to select a party's nominee. It would have been better of you to pay out bets on both sides, particularly since there is no real dollar cost to you. Anyway, with out that, I'm annoyed that I need to pull out a dictionary each time a question is asked...not worth doing for me.
Post a Comment