Catching the Crazies

My Sister a couple years ago introduced this great phrase to me which I try to use whenever I can. When she thinks I or someone else is being ridiculous she says: "Adam, you've caught the crazies!"

It seems every few months when something unexpected happens, journalists and pundits who monitor prediction markets "catch the crazies."

The latest example is the round of Nobel prizes. Today, Elinor Ostrom from Indiana University and Oliver Williamson from Berekely were awarded the prize in Economics. Conventional wisdom said Williamson was an outside shot and Ostrom wasn't even on anyone's radar screen. On our public marketplace, Williamson consistently had a 3 or 4 percent chance of being the winner. Did this mean Inkling "had it wrong?" No, it meant Williamson was perceived to have an outside shot based on the information available to people interested in these things.

Why is it consistently so surprising then to the media that the prediction market prices reflected this? It's not like they were reporting anything different. Prediction market prices are simply an aggregation of information available to people and their reaction to that information in the form of buying or selling shares. Markets about Nobel prize winners and the like are great fun but with a lack of information available to traders because of the secretive process, they're not going to be very useful.

Contrast this with a question asked internal to a company where a great deal of information is available, the traders are typically influencing the outcome or they know the actors who are, and there is usually historical precedence. That is an appropriate use case for judging prediction markets on their ability to not only match forecasted probabilities with reality, but provide tangible business value in the form of cost savings and risk awareness and avoidance.

The name "prediction market" is one I've never liked because it oozes having "crystal ball" powers which means there is inevitably going to be a backlash by the media and bloggers when public prediction markets fail to look like they had the inside scoop. Unfortunately I don't think many writers will suddenly explore the nuance of what a prediction market price means or a good question vs. bad as it's certainly easier to just say "they got it wrong" or "they got it right." It's a design challenge we continue to work on to see how we can continue to simplify the output of a prediction market and make it as digestible as possible. We also try to help people understand what prediction market results mean and when a question is going to garner good feedback vs. unreliable feedback, but it's clearly a road we still have a ways to go on.