Crowdsourced Academy Awards Predictions

This Sunday evening the stars will all come together in Hollywood as they always do once a year for the 2013 Academy Awards.

Every year in our public prediction market someone publishes questions about practically every category rewarded, and this year was no different.

We decided to have a little fun this year though and create a separate site just to showcase Academy Awards predictions. You can see it here:

http://academyawardspredictions.com

After working up the design, building the site was a breeze. We used widgets from the public site to both display the real-time predictions and to allow visitors to make their own predictions. The widgets are also built to allow you to customize their design via CSS so they can match whatever site you're displaying them at.

Because all of the interaction and calculations are handled on the Inkling side, we then just needed to build a "static" site with some HTML, CSS, and Javascript. Instead of hosting it ourselves however, we decided to just host the site at Amazon S3. A recently introduced domain service called Route 53 made this even easier.

This was a fun project to work on and I suspect we'll be doing more of these in the future. March Madness anyone?

Intrade's Unfortunate Encounter With the CFTC

Eric Zitzewitz, an Associate Professor at Dartmouth and expert in prediction markets wrote a great opinion piece for Bloomberg about why Intrade should not be sued by the CFTC and subsequently why U.S. citizens should still have access to participate in Intrade. You know, just like the rest of the world does.

http://blogs.reuters.com/great-debate/2012/11/27/the-folly-of-making-political-prediction-markets-like-intrade-illegal/

Inkling Predictions iOS Update

Version 1.0.1 of Inkling Predictions is now available in the app store.

We fixed several bugs and made the app compatible with the iPhone 5.

As always if you have any feedback or feature suggestions for us, be sure to let us know.

Inkling Predictions, our iOS app is now available

After getting through a couple review hurdles with Apple, our iOS app for Inkling is now available!

"Inkling Predictions" can be downloaded to your iOS device from here:

The app can be used to access our public site at http://home.inklingmarkets.com or to access any corporate site you're already a member of.

You can make predictions, review your dashboard, view and make comments for any question, and see your balances.

We look forward to continuing to develop more features and see our app as overcoming a big hurdle in making it easier for anyone to make predictions.

Front loaded incentives using fantasy currency

As a software product considered "optional" to use in most settings, we have to really worry about incentives for people to participate, both as part of our software (game mechanics) and what we advise our clients to do outside the software, i.e. recognition for participation, prizes, etc.

With that in mind I've been thinking a lot about an article that appeared on TechCrunch a couple weeks ago about a study done at the University of Chicago regarding the level of performance of public school teachers under two different incentive programs.

In the first group, teachers were rewarded at the end of the year based on their student's performance on a standardized test. For every percent improvement over their school district's average, they would receive up to $4,000.

In the second group, teachers were told they had been given $4,000 at the beginning of the year and that number would be reduced based on how their students did. For every percentage point improvement over the average, that would be the amount of money they could keep.

As you may have guessed from the title of this blog post, the teachers who had something to lose performed better. Their students were on average 10% better than the district average. The teachers who were given a bonus offer at the end of the year showed no improvement.

Here was the money quote (no pun intended):

"The results of our experiment are consistent with over 30 years of psychological and economic research on the power of loss aversion to motivate behavior: Students whose teachers in the 'loss' treatment of the experiment showed large and significant gains in their math test scores," said List, the Homer J. Livingston Professor in Economics at UChicago.

Current "best practices" about incentives in software usage suggest a diverse approach of rewards, multiple leader boards, a certain rate and style of marketing and status communication, and development of community and interaction.

While there are many things we could improve in Inkling, we're already doing a lot of this with mixed results. These types of incentives seem to appeal to a certain psychological profile, but not to a majority, so application providers, including us, are left with just trying to get a maximum number of registrants so we can get a respectable number of active users. The research about teacher's performance is encouraging because they were surely a very diverse pool of people psychologically, yet this one basic "carrot" seemed to work incredibly well.

So how can those learnings be applied to Inkling and perhaps more broadly to application development in general?

Here are some ideas that have been rattling around:

• Since everyone starts out with 5,000 inkles, we could introduce a "tax" that charges you on a monthly basis based on your usage of the software. If you've made X number of predictions, you get a "tax exemption" but if you don't, you start to lose your inkles and have less predictive power in the application.
• We could generalize this behavior for whatever behavior we're trying to promote: logging in, making comments, sharing the site with a friend, etc.
• Perhaps "tax" has too negative of a connotation. We could replicate what the University of Chicago did with the teachers and give people a bonus at the beginning of each month that they lose unless they make predictions, comments, etc.

We would have to be careful though that we're not incentivizing behavior we don't want, i.e. people just going in and making "garbage" predictions just to avoid paying the tax or losing some of their bonus. Which means a tax or bonus structure would have to be based on people's performance. But conveniently, performance can't be evaluated unless they exhibit the behaviors you want them to anyway.

More generally, perhaps the next generation of incentives in software applications will introduce their own currency specifically for this purpose. For example, applications are always bugging you to complete your profile or to do Facebook Connect and usually just show a status bar or reminder text that you "haven't completed 100% of your profile." What if instead when you signed up you were given 1,000 of fantasy money that you begin to lose if you don't do these things within a certain period of time. And if you do do them, the fantasy money can be cashed out for stuff: a waiver of AirBnB fees on a rental, an extra InMail in LinkedIn, an extra 1GB of space in Dropbox.

Continuing with the profile example, I'm sure these companies have quantified what it means to have someone have a complete profile in real dollar value because they can be more effectively marketed to. There would be a nice business case therefore to do this if it means 10% more profile completions assuming the economics work.

Companies like Kiip are kind of already doing this, but with earning "recognition" as you achieve things in applications. The fantasy currency enables the more effective reverse approach of front-loading the incentive and it's yours to lose. Currency is also much more flexible because you can begin to use it in other parts of your application - "earn X by doing Y."

Endorsement for the Founder Institute

A few months ago I was asked to be a mentor in the Founder Institute program here in Chicago. I was delighted to see a profile of the program in the New York Times recently as I've become a big fan since being involved with the program.

The Founder Institute, if you haven't heard of them, is a global program that helps entrepreneurs get their businesses off the ground by going through an intensive months-long training program. Entrepreneurs  work through their idea, do a lot of planning, incorporate, and get started on their business all while under the guidance of a large group of local mentors who are at the ready to provide help when called upon.

Unlike YCombinator which we went through, the aspect I like most about the program is it's friendly for people who already have jobs and are trying to start their businesses on the side. The reality is most people can't just take multiple months off to attend an incubator program, usually for financial reasons, but also because of family commitments. Founders Institute is like going to night school for entrepreneurs.

If you're thinking of starting a technology-related business, I'd urge you to check out the Founders Institute as a potential option. The price to join is fairly affordable and the network of peers and mentors you then have access to is extremely helpful. It also provides a badly needed framework of accountability for you - you can literally get kicked out of the program if you don't get your deliverables done or you score particularly low on their evaluations, and as far as I know, there are no refunds!

Inkling's 1%

How do you keep a prediction market interesting for the 1%? And more importantly, if there is truly a 1% in the distribution of wealth, how do you also keep a prediction market interesting for the 99%?

The Inkling public marketplace has been around for over 5 years now. Over that time, 38 people have amassed earnings over 1,000,000 inkles. Most of them are still playing to this day, but it's a challenge to keep them entertained. Many only play in questions which can earn them a significant amount. They submit questions whose starting prices are out of reach for anyone but the most wealthy because the larger trades keep it interesting. Meanwhile people new to the application know it's virtually impossible to crack this exclusive club without making a very significant time commitment and  also being good at predicting things.

I've been looking around and I've yet to find anything written about this specific problem in prediction markets, but there are certainly corollaries to regular gaming. What do you do with the people who can beat the game, or play for so long they're regularly reaching the top levels and may be more skilled than the game creator themselves? Usually that's when v2 of the game comes out with new challenges, better graphics, better teaming features, new weapons, etc. and an entirely new chance at making money from your users.

Unfortunately a complete conceptual revamp isn't really in the cards right now so we have to look for other solutions.

We've had suggestions from people to limit the size of trades that can be made, or simply zero everyone's account and start all over again. But this money has been hard earned and I've always been a big proponent of not "messing around" with people's balances. Thus other suggestions of real-world style approaches such as taxes and redistribution of wealth are also off the table.

I think ultimately the right answer will be to deemphasize money earned as the primary focus of the site and create "local" competition among people with comparable balances or other comparable demographics. The guy in position 18,345 shouldn't be worried about the guy in fifth, but he could be easily focused on moving up to 18,344. This is certainly a popular way of doing "best of " leader boards now.

I could also see moving completely away from "best of" leader boards and starting to create leader boards based on your demographic profile or other attributes. Here's how you're doing against other people in your hometown. Here's how you're doing in your age bracket. Here's how you're doing in a particular category of question. Here's how you're doing against people who started in the same day, month, or year you did. There still might be some whales in these leader boards you need to contend with, but the more leader boards we have, the easier it will be to find a niche for yourself and feel like you should continue to put time in to get to the top.

This post wasn't so bad was it? Find out about the next one by following us on Twitter.

Beta testers wanted for our new iOS app

We’re about a week away from distributing a private build of an Inkling iPhone app.

Functionality is pretty basic right now: simple trades, tracking your positions and performance numbers, listing questions and searching for them, and the leaderboard for highest worth portfolios.

We’re starting off catering to simple users with this app, then we’ll get to power users in future versions. The app will work for both our public site and any of our client sites.

Contact me directly: [email protected] if you’d like to get a beta build and give us feedback. We have the right to say no if we think you're sketchy or one of our competitors. :)

Understanding the impacts of a risk occurring

Recently, we have had several clients inquire about using Inkling to understand not only the likelihood of a risk occurring but the potential impact of that risk.

Prediction markets handle this nicely because of the flexibility you have in asking the question and also the fact they output a quantitative value to your question.

For example, we can ask:

"If risk X occurs, what will happen?"

• Impact A
• Impact B
• Impact C

In this instance, the question type in Inkling would allow for multiple possible right answers so each impact can be judged individually. If the risk does not occur (the impacts cannot be assessed,) each impact would be cashed out at 0.

We can also directly correlate the occurrence of a risk (or any other event) with impacts. To do so, we simply multiply the chances of the impact and the risk to understand the likelihood the impact will occur in association with that risk.

For example, we can ask: "Will risk X occur?" and let's say the current price is $75, representing a 75% chance the risk will occur. In a separate question, we can ask "Will impact X occur if risk X occurs?" and let's say the current price is $25, representing a 25% chance the impact will occur.

Multiplying (.25 * .75) the two, we get ~.19 or a 19% chance Impact X will occur.

Assessing the impacts of risks is important for decision makers to understand as it will directly influence what they may or may not want to do to mitigate a risk. For example if the likelihood of a risk is high but only non-harmful impacts have a high probability of occurring, it may not be necessary to mitigate the risk. But if the reverse is true, mitigating that risk may take on a higher priority.

Congratulations to David Pennock, Prediction Market Extraordinaire

Just a brief note of congratulations to David Pennock who until very recently worked at Yahoo Research on all sorts of projects related to our field. He's now, along with what sounds like his entire group, part of Microsoft Research. David will be managing a new office for them in New York.

http://allthingsd.com/20120502/microsoft-hires-14-yahoo-researchers-to-kickstart-new-nyc-research-lab/

David's incredibly smart, a really nice guy, and has always been supportive of our work. We wish him the best.

The Psychology Behind Redesigns: Are You Just Insecure?

I'm in the process of working on a revised look for our application - probably the 5th or 6th time I've done this in the 6 years the application has been in existence.

As I got to yet another page that needed minor tweaks to fit in with the new look I was giving the application, I begun to wonder why I was even bothering to do this in the first place. No one has been complaining, people still applaud us for the "ease of use" of our application, but here I was burning 40-60 hours to work on this. Hours I could have been using to sell, to track our customers better, blog more, or any number of activities that may "move the needle" on our small business more than an application redesign.

But yet here I was obsessing over the border color around an icon.

My wife works at a design firm and she comes home with tales day after day about companies who are going through elaborate redesigns of their sites and paying hundreds of thousands of dollars in the process, not to mention the hundreds of hours their own employees are spending facilitating their work.

Thinking about this a bit more, there seem to be two prevailing reasons why people do redesigns. The first are simply table stakes. You need to have a decent looking site or app to be credible in the marketplace or people are actually going to discount you. Good design brings a certain amount of credibility.

But how about when you have that already? Why do people get so obsessed about redesigning their site when clearly it's going to contribute little or nothing to the bottom line?

I've come to call this the "Front Lawn" theory. There's a certain satisfaction, a certain security in knowing your front lawn is green and mowed. That the front of your house looks presentable to passer-bys. That your front lawn, as an extension of you and what you represent in the neighborhood, is tidy and well organized. I've visited plenty of homes where the outside is immaculate and the inside is a dump.

Ultimately what I think drives a lot of redesigns is basic insecurity, cost and effort be damned. Business sense is thrown out the window for a purely passion driven, irrational decision. When the moment comes for the executive to send the URL to a buddy, or present their company in some fashion, can they personally be proud about what is being shown? Is their front lawn immaculate?

Couch time, here I come. :)

Wisconsin Business School student researchers mentioned in slashdot for their prediction market

Nice job to the Unviersity of Wisconsin students who are doing some research on prediction markets in their business school. They made it to slashdot. Always a worthwhile accomplishment:

http://politics.slashdot.org/story/12/04/03/2121206/healthcare-reform-act-prediction-market

A Ratings Agency for Startups

One of the cooler efforts we've been associated with in awhile, Cdling (pronounced "seedling") is a ratings agency like S&P and FICO that measures risk for local and global investors, experts and startups.

Here's an explanation of what they do:

Low cost startups are unleashing millions of inexperienced investors and hundreds of incubators globally. Driven by an urgent search for economic growth, governments are creating angel investor incentives, lower barriers to foreign capital and reducing regulation. With legal crowd funding outlets in the US like Kickstarter becoming more commonplace, a lot of people will get hurt unless ratings like Cdling’s are established.

FICO scores, S&P rating reports and stock markets have long been used to measure and reduce risk and make commerce faster and smoother. Now Cdling is doing the same thing for the startup community.

Cdling rates people in terms of their real world experience at selecting which companies will succeed by analyzing who has invested with who and in what founding teams. Cdling also scores companies by predicting their likelihood to meet their milestones (this is where Inkling comes in.)

Investors can then use this insight to monitor more, smaller deals, catch the best breaking deals when they are ready to be financed, and as an early warning system to make sure they are spending their limited time and attention where it will earn the most return.

Cdling is currently closed to the general public, but you can get added to the pilot and track their progress by following this link. When you do get in, you'll have an extra CD$5,000 to spend on predictions.

Inkling's Oscar Prediction Guide 2012

Someone didn't ask about every category for tonight's Academy Awards on our public site, but here's a prediction guide for a lot of the categories as of 4:15 ET/1:15 PT. The higher the chance, the more likely the participants thought that was going to be the winner.

Category	Predicted Winner	Inkling's Prediction
Original Screenplay	Midnight in Paris	62% chance
Adapted Screenplay	The Descendants	78% chance
Visual Effects	Rise of the Planet of the Apes	69% chance
Sound Mixing	Hugo	45% chance
Sound Editing	Hugo	64% chance
Best Short Film (Live Action)	The Shore	41% chance
Best Short Film (Animated)	The Fantastic Flying Books of Mr. Morris	46% chance
Best Original Song	Man or Muppet	68% chance
Best Original Score	The Artist	83% chance
Makeup	The Iron Lady	74% chance
Foreign Language Film	A Separation	71% chance
Best Film Editing	The Artist	48% chance
Best Documentary (Short Subject)	Saving Face	39% chance
Best Documentary (Feature)	Paradise Lost 3: Purgatory	33% chance
Best Directing	The Artist	73% chance
Costume Design	The Artist	44% chance
Art Cinematography	The Tree of Life	66% chance
Art Direction	Hugo	68% chance
Animated Feature Film	Rango	88% chance
Best Supporting Actress	Octavia Spencer	82% chance
Best Actress	Viola Davis	76% chance
Best Supporting Actor	Christopher Plummer	80% chance
Best Actor	George Clooney	54% chance
Best Picture	The Artist	80% chance

A dashboard two ways

One of the perpetual issues we face as a company that sells prediction markets is reconciling two primary kinds of users: the first are people who have never heard of a prediction market or aren't familiar with stock market concepts. This is the vast majority of our users. They are usually being asked by someone inside their company to participate in an internal prediction market and the experience is completely new to them.

The second type of users are familiar with stock market concepts, understand buying and selling shares, and have maybe even participated in a prediction market before like the HSX or Intrade or the Iowa Electronic Markets.

We've always worked hard to try and satisfy both camps in one user experience, usually erring on the side of simplification vs. complexity. But that's a fine line to walk and instead of really making both types of users completely satisfied, we've delivered more of an 80/20 experience.

Last year we began the split of the two user groups with two distinct trading interfaces for "simple" and "advanced" users. In a couple days, we'll be following up this split with the availability of two different dashboards to track predictions. The default will be a "simple" dashboard where you'll track your investments, but will never come in to contact with the concept of "buying shares."

Here's what an entry in the simplified dashboard will look like:



We wanted to make sure the only calculation the user would need to understand is, I spent X to make the crowd prediction Y. We also tell them what their maximum profit or maximum loss would be at any given time. And we let them see the "history" of predictions in any answer, describing in plain language exactly what they did at any given time:



In contrast, a person could choose to use the "advanced dashboard" which simply reveals more about their predictions. They can see how many shares they bought, the concept of short selling is not hidden but is instead made transparent, and there are more liquidation options.



We've also built a search function for your dashboard, so those who have more than one page of predictions can easily find them by typing in a word or two associated with the question instead of having to page through them or rely only on various sorting options. Someone told us recently they have over 40 pages of predictions on their dashboard. Search should be a huge timesaver.

I'm sure we still haven't gotten either experience perfect, but this should lay the foundation for better satisfying both levels of users.

Expect to see the new dashboard introduced some time Thursday or Friday evening.

Watch out for bullets

I called my cable company recently to see if they had any promotional plans I could switch to to relieve the exorbitant rate I was paying. They said no, but they did offer to swap out my old crappy DVR for a Tivo at no charge.

I drove to the RCN office in Chicago to do the swap. I walked in to find a sparsely furnished lobby and two customer support people sitting behind concrete walls and bullet proof glass.

I know they have a stock of relatively expensive equipment, but that can't be the only reason for the Western Union style lobby.

The experience got me thinking: how bad does your customer experience have to be before you need to start outfitting your support centers with reinforced steel doors and bullet proof glass? Shouldn't this be a signal to the company that something is seriously wrong with customer relations, or is this just what a semi-monopoly always looks like (everyone hates you?)

Over-Optimism in Official Budget Agencies' Forecasts

We're always looking for where applications of prediction markets may make sense to improve a process. It looks like official government forecasting may be one such area to tackle.

Jeffrey Frankel from the Kennedy School of Government at Harvard and Director of the Program in International Finance and Macroeconomics at the National Bureau of Economic Research has written a paper entitled: "Over-Optimism in Forecasts by Official Budget Agencies and Its Implications."

From the digest on the NBER site:

...Overly optimistic official forecasts of future budget balances have facilitated complacency and so have contributed to tax cuts and increases in government spending, and therefore to realized budget deficits, during the last decade.

Analyzing data for 33 countries, Frankel finds that the average upward bias in the official forecast of the budget balance, relative to the realized balance, is 0.2 percent of GDP at the one-year horizon, 0.8 percent at the two-year horizon, and 1.5 percent at the three-year horizon. The longer the horizon, and the more genuine uncertainty there is, the more scope there is for wishful thinking.

A prediction market's biggest advantage would be to allow a more diverse opinion pool that would hopefully eliminate this bias. One could imagine not only involving officials at the OMB but various staff from Congressional offices, experts from various Departments, and even industry experts. If the government won't take this on directly, could there be a "shadow" forecasting process with these same people? What would be their incentive to participate? Hmm...

Your 5 Bullet Points

Last weekend I attended a funeral for a wonderful woman in Bethlehem, Pennsylvania. While funerals are of course sad affairs, when they're for someone who has died of "natural causes" at an old age, I tend to be less sad and more contemplative about how they lived their lives and what I can learn from them.

At the funeral, person after person got up to speak about the generosity of this woman. She was a loving grandmother and great-grandmother. She was a teacher and a librarian. She was an elder in her church and she welcomed new community members with open arms, often being the first "node" they met. She served as a starting point to make them feel more comfortable in their new surroundings.

A 23 year old grandchild got up to speak about how she was his life's exemplar. The career he is pursuing and the values which he lives by are a direct result of his grandmother's influence.

After the graveside service, I was driving with my Dad and we discussed how we've all thought of our own funerals. What will people say about us when we're gone? When the definition of our lives must be condensed down to a 30 minute memorial service, what are our 5 bullet points going to be?

Everyone has heard the adage "no one wishes at the end of their life they had worked more," but what does that phrase really mean? For some it means they wished they had spent more time with their family. For others, more time having fun, or more time with their kids, or giving more of themselves to others.

Among people I know, most earnestly want to achieve a balanced lifestyle (isn't that really what that adage is getting at? A lack of balance?) They want to take care of themselves financially, then they'll begin donating time and money to others. I know until recently this is the attitude I carried, focusing almost exclusively on work and just trying to be a decent friend and family member. After I've "made it" is when I'll start to truly give back.

But then my attitude(?), maturity level(?), begun to change. In my reading I found myself more attracted to the stories about people giving of themselves vs. those whose companies just got bought or earned investments. Sermons reminding me about "giving back" were resonating more.  The plight of others in need begun having a more profound impact on me.

Do I still want to "get mine?" Of course. But I've already begun altering my schedule in fits and starts the last year through volunteering. And I know by doing so I've already had a tangible impact on several people's lives. And those few hours of volunteering felt damn better than any deal we won or compliment we received for our software. Some might say I should therefore find something else to work on. But I think the answer is instead to give even more.

We work. We make salaries. We live in comfort. In turn we have the ability to help others. I realize it's all somewhat symbiotic. But when does one shift the balance? Instead of waiting until a magical time when there are seven figures in my bank account which may never come, why not start spending more of my time giving now? Isn't that simply another way of achieving wealth?

I guess that is why there is so much attention at the end of someone's life about what they gave of themselves. Because despite the incessant focus on money and achievement in our society, in the end, that's how we still judge you. What did you give of yourself?

"Gamifying" everything can backfire

Gamification has been all the rage recently. It's assumed now that any social app and even new business app needs some form of gaming associated with them to incentivize people to do something.

A prediction market, in a sense, is gamification personified. The existence of a virtual currency drives you, just like in real life, to start comparing yourself with others and competing with them. We exacerbate that competition by adding leader boards showing who has made the most money.

Conventional wisdom says you lavish the top money makers with prizes and praise.

Unfortunately prizes and an over-emphasis on certain leader boards can incentivize exactly the type of behavior you don't want. In most applications, leader boards are simply meant to drive action: add or edit content, check in somewhere, or perform some task. But in a prediction market, we want people's actions to be more nuanced. We want them to make predictions, but ideally we want the right mindset behind those predictions.

Too much emphasis on leader boards and the virtual currency and you have someone asking themselves: "what prediction is going to make me the most money?" Instead we want people to be asking: "what prediction is going to be right or wrong?"

It's a fine line to walk which is why we've been removing focus from our leader board showing who the top money earners are and emphasizing other activities more focused on participation like making good comments and submitting new questions to be published.

In fact we've been redesigning our dashboard where people track their portfolio and in its current iteration we've omitted leader board information altogether. Instead we're trying to draw people more to how their individual predictions are doing and if there is new information that would drive them to change their mind.

TL;DR It's easy to jump on the band wagon to "gamify" everything but you may inadvertently be incentivizing exactly the type of behavior you don't want.

What the Bagel Man Saw - An Accidental Glimpse at Human Nature

I saw this old Stephen Dubner article from the New York Times Magazine pop up on Longform this week and thought it was a great story.

A former economist decides to quit his job and start selling bagels on the honor system at a few hundred offices in the Washington, DC area. He leaves a couple dozen bagels and a money box, then comes back at the end of the day to collect.

In the process he learns quite a bit about what kinds of offices are more honest in paying for the bagels than others and what external factors affect payment (weather, holidays, etc.) He collected so much data he can now fairly reliably predict what the payment rate will be for any kind of office he sells his bagels at.