When Intuition 'Trumps' Analysis

In growing my Inkling score from five thousand to ten million Inkles, one of the most important questions was related to the number of points each team would score in the most recent NBA season.  The question asked about the difference between each team's points and the average of all teams.  As I watched the question play out, I noticed that a couple power users were projecting that teams close to the current average would remain close (their values were low), but there also seemed to be a lot of volatility--teams' projections were constantly changing.  As a result I started forecasting upwards any time a team approached the average.  I didn't know what the forecast should be, but I felt pretty good saying it should be higher than where it was.  Eventually, I created a little Excel file to try to quantify my forecasts, assuming some volatility in their scoring, but mostly I was forecasting based on observation and hunch.

Note the shift in approach between intuition (fast, emotional, human) and analysis (rigorous, logical, quantitative).  Both intuition and analysis are really valuable when forecasting in prediction markets, and one of the key advantages of implementing a prediction market is that it leverages both types of thinking.  We tend to think of analytical approaches as being more accurate, especially when making important decisions, and in many cases they are.  I felt more confident in my Excel projections than my initial hunch.  But the accuracy gain was minimal and it took much longer to generate.  There was a lot of predictive value in my initial hunch, and ultimately the forecasts I made before doing any analysis did more for my score those I made after.

In other cases, analysis may actually lead you astray.  For instance, consider Donald Trump's presidential campaign.  When forecasting primary elections, the best analyses typically rely on candidates' polling average, and Trump is currently leading the polls.  Does that make The Donald The Front-Runner?  Prediction Markets say no, both ours and others, and I'm pretty confident they're right.  I also don't think it's rigorous analysis as much as people's intuition guiding the markets.

As a forecaster, it's important to learn when to rely on intuition versus analysis.  But as a prediction market administrator / consumer, what matters is that the market incorporates both intuitive and analytical approaches.  As a result forecasts and be fast, emotional, and human, while also being rigorous, logical, and quantitative.  Most importantly, they're highly accurate.

Ben Golden is a software developer, economist, and evangelist for Inkling Markets.  You can find him on Inkling Markets as benthinkin, and on Twitter, @BenGoldN.  Email him: ben at inklingmarkets dotcom

Pundits And Prediction Markets

As I've become more involved with prediction markets, I've grown increasingly frustrated with journalists who make predictions (aka pundits) without linking to prediction market questions.  This is, in my opinion, lousy journalism, and insulting to readers.

Linking to a prediction market question has a number benefits, including:

• requiring questions to be resolvable, which forces pundits to clarify what they're actually predicting
• tracking performance, revealing over time whether a pundit's claims are insightful or nonsense
• allowing readers to respond by forecasting on the prediction market, creating a more engaging user experience

Anyone who believes what they're saying should be willing to stake their reputation on it by creating/linking to a prediction market and disclosing their forecasts.  It also creates a richer audience response; forecasting in prediction markets involves clear expressions of individuals' views, which many readers might prefer over the chaos of comment forums.  My hope is that eventually, articles that contain predictions will always link to a prediction market--or embed it within the media platform itself--similar to how finance articles display tickers for any stocks they mention.

In the meantime, readers can also create prediction markets.  Lately, whenever I come across something that looks like a prediction, I'll create a market myself--here are some recent examples.  Creating new questions on Inkling is easy, and anyone who signs up can do so.  (note: we do require approval, but we almost never reject a question.)  It typically takes just a few days for our users to start moving markets to a reasonable forecast.

The media's job is to provide readers with accurate information, and when it comes to forecasting, prediction markets are an invaluable source of information.

Ben Golden is a software developer, economist, and evangelist for Inkling Markets.  You can find him on Inkling Markets as benthinkin, and on Twitter, @BenGoldN.  Email him: ben at inklingmarkets dotcom

Enterprise Crowdsouring: A Primer

When my grandmother immigrated to the United States, she couldn't afford to call her family on the telephone.  That was about 70 years ago.  Today, I have a friend whose brother moved to Sri Lanka to become a Buddhist monk and literally lives in a cave.  He and his family Skype.  This is the power of the Internet--for a significant portion of the planet, it's now possible for any two people to communicate from anywhere, in real-time, basically for free.

This technology has improved our lives in many ways, but it's also brought a lot of unexpected change, and will continue to do so.  Almost every human institution that exists today--companies, governments, academic institutions, systems of government and economics--were created at times when free real-time communication was unfathomable.  When you hear technology companies talk about 'disruption', they're usually talking about changing an institution whose methods assume that people can't communicate in real-time for free.

Enterprise Crowdsourcing isn't about changing a specific industry, but rather the very nature of how businesses and other enterprises operate.  We're challenging the command-and-control model of businesses, which says that tasks should be done by specialized departments, with information flowing up to decision-makers at appropriate time intervals and then back to departments at the discretion of their managers.  Command-and-control actually makes a lot of sense when communication is expensive, but it's becoming increasingly inappropriate as a way of doing business now that communication is free.

When you need to forecast your company's sales for the next quarter, the old model said you should hire an analyst or consultant and assign them the task of sales forecasting.  The new model says you should ask everyone at your company, perhaps using a prediction market.  This communication is reasonably cheap (compared to paying analysts / consultants) and your employees have access to the insight needed to make highly accurate forecasts.

Forecasting is one example of a business function where the crowd outperforms an individual/department, but there are many others, some of which I'll outline in upcoming posts.

Ben Golden is a software developer, economist, and evangelist for Inkling Markets.  You can find him on Inkling Markets as benthinkin, and on Twitter, @BenGoldN.

The Best Is Not Enough

Barry Ritholtz has written a curious column titled The 'Wisdom of Crowds' Is Not That Wise for Bloomberg View, which criticizes prediction markets.  This is not a new view for Ritholtz, as he reminds us by linking to six blog posts critical of prediction markets each written by...Barry Ritholtz.  Indeed Ritholtz has made it his mission to find instances of prediction markets 'failing', and has found six of them.  These include:

• For about two months in 2003, a prediction market expressed there was a 50%-75% that Howard Dean would win the Iowa Caucus...Dean did not win the nomination.
• For most of 2007, a prediction market thought there was a 10%-30% Barack Obama would win the New Hampshire Primary.  Then for about a week before the election, it thought the chance was in the 50%-95% range...Obama lost the New Hampshire Primary.
• In 2005, a prediction market did a poor job forecasting the result of the Michael Jackson trial.  (And so did everyone else.)

Ritholtz's expectation of prediction markets seems to be that they should be perfectly accurate, and when they fail to meet that standard, he dismisses prediction markets as unwise.  But prediction markets aren't trying to be perfect; they generate probabilistic forecasts, meaning they expect to be 'wrong' sometimes.  When a prediction market assigns an 80% likelihoods to events, it expects that roughly one in five won't happen.  So over time, of course there will be some instances where a market leans in one direction and the opposite result occurs.  If you can only find six of these cases over a twelve-year timeframe, you're not looking very hard.

The value of prediction markets is that they're more accurate than other forecasting methodologies, including surveys, data modeling, and gut instinct.  (They're also often cheaper to implement, too.)  Ritholtz utterly fails to demonstrate that any alternative approach is better.  He concludes the following:

I remain unconvinced you can call prices "wise," no matter what market sets them. Perhaps the most constructive comment one can make about the crowd in market prices is that there are no better alternatives yet invented for determining the price of any item to be bought or sold. "The best we've got" hardly rises to the level of "wisdom." (emphasis added)

Here Ritholtz admits that prediction markets are the best tool for forecasting, but nonetheless argues semantics over whether markets are 'wise'.  In my book, using the best tool available is a wise thing to do, so even if the markets aren't wise themselves, people who use them to generate accurate forecasts are.

Ben Golden is a software developer, economist, and evangelist for Inkling Markets.  You can find him on Inkling Markets as benthinkin, and on Twitter, @BenGoldN.

I, Benthinkin

When I joined Inkling Markets in Sep 2014, I started forecasting on our public-facing forecasting site.  I already had some experience using prediction markets, having been actively involved with SciCast and its predecessor DAGGRE, and was determined to show my new employer that in addition to being a pretty decent software developer, I could forecast with the best of them. 

The challenge was daunting—Inkling users start with five thousand Inkles (our nominal currency) while the top forecasters have accrued hundreds of millions (in one case billions).  To reach the top ten, I would need to double my score more than thirteen times.  It was time to get to work.

One big difference between Inkling and my previous experiences is that I had some relevant domain experience.  Whereas SciCast focused on science and technology—areas where I have only limited knowledge—Inkling includes questions about sports and politics, where I’m much more comfortable.  Rather than relying entirely on technical forecasting tricks I’d developed and original research and analysis, I could often confidently place bets about future events by drawing insights from my understanding of how sports and politics work.

In future posts I’ll provide a detailed look at how I designed, implemented and refined my strategy.  I’ll share tips and tricks, and more importantly how I think about prediction market questions.  I’ll talk about why I love prediction markets, what I get from them, and why you should get involved.  I’ll explore industries / businesses that benefit from greater use of prediction markets (spoiler alert: all of them).  And I’ll discuss the administration of prediction markets: how to pose questions to yield the best possible results, how to keep users happy, etc.

As for my progress forecasting on Inkling, I haven’t reached the top ten (yet), but I am in the top twenty.  In ten months, I've doubled my score only eleven times, for a score of roughly 11 million Inkles.  This is, I believe, a record for Inkling; I can’t find another user who reached 10 million Inkles within a year of joining the site, and only two cracked 5 million.  As I continue to forecast on Inkling Markets, and work with prediction markets, I look forward to writing about my adventures.

Ben Golden is a software developer, economist, and evangelist for Inkling Markets.  You can find him on Inkling Markets as benthinkin, and on Twitter, @BenGoldN.