Tuesday, March 31, 2009

The Key to Keeping your Prediction Marketplace Active and Successful

This post is about a key, maybe THE key, to keeping your prediction marketplace active and successful. The short answer?

Create markets more frequently.

Prediction markets are just like any other social media community. For example, the most popular use of social media is blogging. Assuming you already have something interesting to blog about, what is likely the number one way of starting a blog and a community of people who follow and participate in it?

Blog more.

You don't see many vibrant blogs where the author only blogs once a month, or creates 10 blog posts in one day then doesn't touch the blog for a month or two.

Likewise, you can probably expect, that as an administrator of a prediction marketplace, it's important to keep the frequency of posting new markets high and consistent.

One particular client from an analysis we describe below was found to be creating markets about once a week, and they get great participation.

On the other extreme, a very promising marketplace that had a ton of early participation hasn't created any new markets since the second week it launched and participation has flatlined. They want to keep the marketplace going but they're essentially starting over to get people interested again.

So a good rule of thumb to take away from this: put a date on your calendar at least once a week to make sure a new market is created.


When people first explore using prediction markets they still ask: "Do they actually work?" and "Will people understand them?" I think we've done a pretty good job addressing that they do indeed work:


And from our interaction with both users and administrators using the thousands of Inkling powered marketplaces out there, people from all demographics understand and enjoy them. Here's a quote from SwelJoe at news.ycombinator.com on startups that got started with Y Combinator:

"...A couple of others have really impressive reach outside of the usual Web 2.0 set and into the 'real world' (Loopt, in particular, though Inkling Markets is everywhere, too...my dad has used Inkling's product, and he just got broadband for the first time last week)."

But to avoid the problem described above of the marketplace languishing, administrators should perpetually be asking themselves: "How do I keep our prediction marketplace active?"

We've blogged and written about incentive strategies and know that people need SOME form of incentive to participate in a marketplace. And as we've said before, most people we talk to immediately think of awarding prizes to top traders. But we've found the biggest incentive in prediction markets, and more broadly in any social media, is simply to create an interesting place to go. "Interesting" means participants will get something out of it either by communicating with others, learning something new, or feeling good about contributing. It is not just about giving prizes or having a slick interface.


But words are words and proof is proof. We decided to see if we could analyze some Inkling data to see if there is a real correlation between how often markets are made and user participation.

We randomly picked 40 Inkling powered sites to start analyzing.

Since Inkling marketplaces that are largely about elections and sports behave much differently than markets predicting revenues and project deadlines, we picked all private marketplaces run by companies.

We wanted to make sure we had marketplaces that were making a serious effort to invite people and get market questions made. So these sites all had 15 or more users, and 20 or more markets created during their lifespan.

These 40 sites span various industries (energy, consumer products, real estate, gaming, entertainment, etc.) and they were created at various points during Inkling's history. Some of them are currently active and some of them are not.

A further selection criteria was then applied. The average trader had to be at least 90 days old in the Inkling system. We did this because we wanted to look at continued participation in the marketplace 2 months after their first trade and not have this study influenced by the novelty effect, which you'll see below.

Following this filter, 15 sites made the cut. The 25 other sites either didn't run their pilots long enough or were too new to be included.


What's a good variable to determine how active a prediction marketplace is? Well that can be a bit arbitrary. But let's choose something that has a low bar that seems worthy of improving.

How many people in your marketplace make a trade 2 months after their first trade?

This allows the novelty to wear off.

The other variable chosen is the percentage of an Inkling marketplace's "lifetime" spent creating markets.

"Lifetime" is defined as the time in between a marketplace's 5th trade and it's last trade. This was picked to avoid the kicking the tires phase where someone signs up for a pilot and makes a few test trades before taking some time to really launch the marketplace.

We created a scatterplot of these 2 variables.

And it does seem that there is a linear relationship between the two variables with a Pearson correlation coefficient of 0.73.

The correlation coefficient also appears to be statistically relevant. If our null hypothesis is that the two variables are not correlated, we look at our T statistic, and it's 3.84. 3.84 with 13 degrees of freedom (a sample size of 15 sites), has a p value less than 0.01, which is definitely smaller than the popular cutoff of 0.05. So we can reject the null hypothesis and accept the alternative hypothesis that the two variables are indeed correlated.


This exercise was fun and informative, but correlation doesn't mean a cause and effect relationship was found. Both of these variables could be dependent on a third variable that isn't analyzed here.

It could be surmised that both of these variables might be equally influence just by time. For example, both administrators and traders are influenced by a company's holidays and business cycle.

However, we might be able to anecdotally disprove that from our interactions with the sites analyzed here. Most of them represent large international businesses, where the Inkling administrator is not tied to the same projects and timelines as most of the traders.

To prove a true cause and effect relationship here, it would be necessary to perform a well designed experiment, and maybe this correlation could inspire one. Perhaps an experiment analyzing this type of relationship has already been performed, even about other social media systems like blogs.


Prediction markets are like many other systems. You get what you put in. If an administrator puts in the effort making questions, cashing markets out on time, communicating with traders via the discussion boards, announcements, newsletters, etc., the traders of a prediction marketplace return that investment by continuing to come back and offer their wisdom in the form of trading and discussion.

And making more markets doesn't have to mean more work for an administrator. Inkling makes it easy for all participants to create their own markets and we encourage every administrator to keep this feature turned on. And if you're plumb out of ideas for new markets this week, you can always just borrow some from our public site.


Anonymous said...
This comment has been removed by a blog administrator.
Anonymous said...
This comment has been removed by a blog administrator.