The Great Prioritization Debate of 2009 Coming to a Company Near You...

As people have returned from the long Holiday break, we have been hearing more and more from our friends, delivery partners, and clients about the grand "prioritization" process that is happening at their companies. With a gloomy economic outlook the desire is always to look inward: what should we be focusing on? What are our highest priorities? Which projects are worth doing now and which should be shelved? What's a luxury and what's a must-have?

Using a prediction market to ask these questions is valuable for a couple reasons:

• As opposed to the executives hunkering down for several days in a meeting room and arbitrarily deciding what projects should move forward and what should get cut, this is a credible process everyone has been involved in. No employee can say later they weren't asked for their input.
  


• Ad hoc prioritization exercises kill internal morale and productivity, plain and simple. I've personally been the victim of and participated in these meetings. As onlookers we all sat around gossiping about what was going to get cut, what would get saved, and looked for any piece of information that would give us a clue as to what was going on in the meeting room. When one of our bosses would walk out there would be intense, hushed discussion. "Did she look stressed? Pissed?" And participating in these exercises as a manager wasn't much better. Much of the talk was arbitrary horse jockeying and was rarely based on reality. The loudest talker or the person with the highest position always preserved their projects regardless of merit. Not to mention it took days to get through and no one got any work done.

Running a market to go through this exercise is simple. Figure out a verifiable metric you are going to measure projects against and ask about each. For example one of our clients is running a market like this:

Q: Which of these research projects will be included in a product design by the end of 2009?

Possible Answer	Current Probability
Project 1	50% Chance
Project 2	50% Chance
Project 3	50% Chance
Project 4	50% Chance

After a couple weeks they saw something like this in the market:

Possible Answer	Current Probability
Project 1	5% Chance
Project 2	85% Chance
Project 3	40% Chance
Project 4	65% Chance

This was great input to the decision makers. People didn't feel very confident Project 1 was going to make it while people felt very confident Project 2 would. Projects 3 and 4 are close to even odds so maybe they are given additional resources to increase their odds, or shelved. Regardless, the decision points are now being discussed based on how the projects will affect the business, not on whose pet project it is.

Even better, as the year continues this market is continuing to run and managers are watching how probabilities change based on their actions, say if they decided to put more resources in to Project 4, do people now think it has a higher chance of making it in to a product design?

Using Inkling to prioritize projects doesn't remove or hijack the decision making responsibility managers have but it focuses the discussion on business value of projects and makes everyone feel like they have skin in the game about the decisions being made, which is crucial at a time when it's easy to just sit around and talk about how bad everything is and get nothing done in the meantime.

Easier to access market management

A couple months ago we added hover menus to our "management" market listings. Those are the markets listed when you click "manage markets". You see that link when you are an admin or your Inkling marketplace allows users to make their own.

The hover menus allow us to hide some of the repetitiveness of the user interface. There's a lot of things you can do with a market these days (edit/cash out/suspend/trader performance/delete/etc.), and a couple more coming real soon. All of those actions translate into some kind of link or button to make it happen. So instead of having 5 or 6 buttons on every row listing a market, the hover menu keeps each row more concise and easier to read.

Last night we consolidated our market listings. So now if you are an admin managing a bunch of markets or a regular user who created a few markets, whenever you find your market in a list (search, the different sorts, the home tab, etc.), you'll have the manage menu available to you on those markets you are managing.



This gets rid of the unfortunate use case where you need to search for a market and then cash it out. Before you would have needed to search for the market, click the market name to get to it's market page, then click edit to get into the management listings containing the market, and then you'd have had your cash-out button available to you.

With last night's change, it's just search for your market and then right there in those listings are your management functions. Fewer clicks. Less pain.

We've also iterated on the look and feel of the menu. Now it is higher contrast with the rest of the page so it's easier to spot.

It's also inline with the top of the row it belongs to (before it was lined up with the middle of the row). This makes it easier to anchor to a specific market in a list. So if the market is right at the top of the page, now half of the menu isn't going to be hidden by the browser's frame (i.e. outside the viewport).

We also are keenly aware that these management functions also need to be much more integrated with a normal market page. So when you are viewing a market, you can edit and cash out things right from that page rather than having to jump back out to a list of markets to get to this menu. We'll have this improved very soon.

Hope everyone had a relaxing holiday and is already enjoying 2009.

New corporate home page

Last night to kick of the new year, we launched a new version of our corporate home page (http://inklingmarkets.com).  We are very proud of how it turned out.  

Apart from getting an aesthetic redesign there are some elements of the new content that really stand out.  

We've been in business over 3 years now, and have seen millions of trades made, and tens of thousands of markets created in thousands of different marketplaces powered by Inkling.  Through all of this we've seen quite a few different scenarios of how prediction markets are used publicly and privately including what works and what doesn't.  

So we've listed 8 case studies.  These summarize how 8 of our clients are using Inkling, some examples of the questions they are asking, and benefits they've seen.  We hope these help spark even more ideas how Inkling can be used in various industries and for all sorts of different decisions. We'll continue to post these from time to time.

We've also added a "Do prediction markets work?" section that mentions our study on the accuracy of Inkling we did in 2008 and had posted about on our blog.   This is obviously the most important question to ask about Inkling.  Unfortunately, how to measure the accuracy of prediction markets is still something often misunderstood.  So this section of our website discusses that question with evidence drawn from our business.  

Hope you like the new website and are enjoying the holiday and any remaining vacation!

Inkling mention on rubyonrails.com

The Ruby on Rails website received a refresh recently which includes the page on applications currently using Rails.  We are very honored to get a mention. 

Rails has been a major ingredient to our success.  I'd almost like our use of Rails kept quiet, but when all your developer job postings have "RoR" in them, it's not a very good secret.  :) 

We've been very lucky to be able to borrow from a wealth of talented developers, documentation, innovative open source projects, etc. Of course other languages have tremendous communities too, but Rails really fits us well. Rails and Ruby have gone a long way in helping us try to create simple, fast, and stable solutions to our customers' needs. 

And of course as Rails evolves, they help us evolve. We've got some really cool stuff coming in 2009 that is made so much easier to get done because of this framework. 

Hope everyone has a wonderful holiday and Happy New Year!

Discussion board comments now on Facebook Connect

This weekend we turned on the ability to publish your discussion board comments to your Facebook Connect feed.  

A huge thanks goes out to Javan for getting the bulk of our Facebook Connect work done. Facebook is a wonderful piece of engineering and design.  Sometimes their documentation however on things like Connect can be a bit daunting.  Perhaps it's the wiki style approach, but Javan was able to make sense of it and got us rolling.  

Single Sign On with SAML tested against Ping Federate

We just recently finished interoperability testing with Ping Identity's product Ping Federate and our SAML implementation is doing great.  

Inkling on Facebook Connect

This is a teaser no more!  Today anyone who's a user on http://home.inklingmarkets.com can now connect Inkling to their Facebook account.  

If you are logged into home.inklingmarkets.com, you can go to the settings link in the top left.  There you will find a "Connect to Facebook" button.  Clicking that will do it's magic.  After that whenever you trade you'll see an option to publish to Facebook if you want.  A popup of what you can publish to Facebook than shows up and you can choose if you want to publish a little one liner about your trade or more details about the market in general.  

I hope you like it, it's been a lot of fun to use so far.  And we plan on adding more to it: to let people publish the new markets they create, comments they make, etc.  Please let us know how it works out for you. 

Here's an example of some trades I made, that shows up on my Wall right now. 

Inkling with SAML 2.0 Single Sign On

We've been working on quite an interesting and difficult project here at Inkling, single sign on using SAML 2.0.  We have been beta testing it with a client of ours using a Siteminder identity server and have been pleased so far with the results.  

Here is a 30 second video of Inkling integrated with OpenSSO (Sun's open source identity server).  Please forgive the music; single sign on can be kind of boring to watch.  Also this was a big accomplishment for our development team, so it needed a bit of flare. :) 

 

Please let us know if you also have a need for single sign on with SAML 2.0 soon and would like to start beta testing with it.  

For any developers reading this, this work was powered by saml2ruby, which Todd Saxton and Pat Patterson contributed.  saml2ruby was created largely as a proof of concept, but over the last couple months, we've made improvements to it.  For example we fixed a security flaw it had, similar to the one Google had in their SAML 2.0 implementation in September of this year.  We also found the XML cannonicalization library used for this wasn't doing a good enough job to verify XML digital signatures properly for identity servers other than OpenSSO at the time.  

The current plan we have with Pat's blessing is to move saml2ruby over to github, where hopefully it can get even more attention and love from the rubyists there.  We'll also provide our patches as well as soon share some lessons learned with this whole experience of SAML, service providers, identity servers, etc.  

Getting Grilled by MBA Students

I had the pleasure today to present at Northwestern's Kellogg School of Management in Professor Marco Ottaviani's class on Strategy and Organization. Readers of our blog who are also hardcore prediction market aficionados may recognize Professor Ottaviani's name. He published, along with many other prominent economists, an article in the journal Science called "The Promise of Prediction Markets."

Speaking in his class I was once again reminded of a phenomena I often experience when presenting to MBA classes: questions overall are probing and insightful, but without fail the first question I always receive is about market manipulation, the possibility of collusion, or project managers hijacking their own projects to increase the value of their own long or short positions. I wonder why this is? Why are the initial questions not more positive? Do students automatically try to "go for the jugular" of their guest speaker? :)

Defining success of a pilot

A couple weeks ago we ran a one-day workshop for a new client of ours that is about to begin a 6-month pilot for 250 employees.

A common exercise to conduct before embarking on any new project is deciding what the success factors will be. How will we know at the end of the project if we've been successful? What will our metrics be?

When it comes to prediction markets, one obvious success factor is how "accurate" the markets have been. And by accuracy we of course mean the prices of stocks matching the probability of events occurring over time (if markets are regularly saying a particular answer is going to happen 80% of the time, it should actually happen 80% of the time!)

But there are other factors we discussed that should also be used to judge the success of pilots. Among them:

• Good insights from trading data e.g. “executives tend to be consistently overly optimistic in their trading behavior” or "we discovered multiple traders behaving as outliers in a heavily traded market about our quarterly forecast that led us to reconsider our models."
  
• Market results are viewed as valuable input to decision making. In other words, questions have been asked where understanding the probabilities of the answers occurring have proven to be valuable in setting direction or have provided an ROI.
  
• People are actively creating new markets or requesting new markets be run, or even complain when markets are not being cashed out right away. All these behaviors show passion and investment in the marketplace.
  
• People citing the prediction market in the daily discourse of business during status meetings, in the cafeteria, and in company newsletters
  

"Method is embarrassed" - interesting notification

Method has an interesting way to notifying users of their website potential bugs they may have encountered. 

Hover over "Method is embarrassed"

and a popup notifies users what the problem is: 

Refresh your positions on your own

The performance of your positions is cached.  The reason this needs to be done is because calculating the true value of each of your positions can take some computing horsepower.  Not a ton, but enough so that if you have a lot of positions, if it was calculated in real time, your dashboard page would take about 12 seconds to load each time.  

We are trying to improve the experience of this area of our application so that even if it's cached, you won't find it frustratingly stale.  One of the UI elements we are experimenting with is giving you the ability to refresh the performance of whichever position you want, whenever you want. 

When you login today, you'll see that on any of your positions on your dashboard you can hover over that row and then click the performance indicator (the up/down images) or the refresh link below it to get the cache refreshed with the real time performance.  For example: 

There are many more improvements we have planned for the dashboard.  Stay tuned.  And have a very Happy Thanksgiving!

Clouded judgment

From an article in the Sunday New York Times:

...Normally, a big bank would never allow the word of just one executive to carry so much weight. Instead, it would have its risk managers aggressively look over any shoulder and guard against trading or lending excesses.

But many Citigroup insiders say the bank’s risk managers never investigated deeply enough. Because of longstanding ties that clouded their judgment, the very people charged with overseeing deal makers eager to increase short-term earnings — and executives’ multimillion-dollar bonuses — failed to rein them in, these insiders say.

We talk a lot with our clients about using prediction markets as early warning indicators to simply begin asking more questions. Longstanding friendships and power centers in organizations are well entrenched and difficult to operate around. The signals from the markets, not only from the stock prices, but also from the data generated by trading, are a credible and efficient way to expose reality so ultimately the decisions made are what is right for the business, not just for the people involved.

Your Inkling Dashboard Now Available as a Google Gadget

You can now add your Inkling Dashboard to your Google homepage. On your Dashboard look for the +Google icon.

After adding it to your Google homepage you should see a minimal version of your Inkling Dashboard: 

This feature is available on all the Inkling marketplaces so Google homepage users now have a way of keeping track of multiple memberships they have across the Inkling universe.  

Interesting marketing lesson from Silk

Seth Godin at Ted on some lessons on marketing. Was listening to it in the background, but perked my ears up when he mentioned that Silk (soy-milk) tripled their sales by putting a product like soy-milk in the refrigerated section at the grocery store when in fact, their soy-milk doesn't have to be refrigerated at all before it's opened. Seems obvious why, but it sounds like this was not obvious at the time.  The silk story is 15 minutes in.

Woah, has there been some crazy stuff going on over here.

There's been a lot of new projects going on here at Inkling and a major catalyst to all this momentum was Rails developer Javan Makhmali (http://javanmakhma.li/) joining our team recently.

Javan is a fantastic Rails developer and has a great eye for simplicity and intuitive design. He's worked on quite a few great projects and one that stood out because it was such a fun coincidence was http://noonebelongsheremorethanyou.com/. I forgot where I had originally found that site, maybe it was Seth Godin a year and a half ago. That was a seriously cool idea, and one that Javan kept running amongst some monstrous traffic when it spread like crazy.

You've been seeing his work all over Inkling Markets lately: the new digested market alerts, the hover menus on managing markets, faster cashouts of the really heavily traded markets, market refunds, and a ton of other things behind the scenes. He's also been cooking up some things that I need to keep a lid on for just a bit longer, but hopefully we can launch soon.

Anyways thanks a ton, Javan. Your work is greatly appreciated by our customers, our users and us.

Deeper Dive: Life Sciences

As Inkling introduces prediction markets to more and more companies, we are beginning to notice some issues common to each industry. This is the first of a series of postings that will discuss how Inkling is being used to address some of these issues. The first example I want to share in detail is in the life sciences industry.

To understand some of the issues life science companies are facing we need to understand some macro trends:

• In 2006, people in the first wave of the baby boomer generation reached their 60's and by 2024, all boomers will be over 60. In 2030, the elderly, defined as over 65 are expected to represent almost 20% of the population versus 13% in 2010.
  
• The biotechnology industry continues to thrive. New medications, devices and other medical advances continue come to market at an ever increasing pace.
  
• With the human genome mapped and a greater understanding regarding the role of genetics in chronic illness, personalized medicine is gaining increasing acceptance. As a result, new diagnostic equipment and tests are being developed to identify a patient's genetic composition and determine if the patient has a greater chance of developing certain diseases.
  

Alongside these macro trends is a great amount of uncertainty:

• Constant potential for new regulations
  
• Modification or even a complete transformation of the U.S. healthcare insurance market
  
• Changes in how the FDA may evaluate and ultimately approve new drugs, with more attention being paid to safety and tolerability of compounds
  
• Greater generic competition
  
• Greater oversight in the direct-to-consumer (DTC) market
  

Knowing how uncertain the future is, how do life science companies create strategies for growth while at the same time, avoiding the paths with pitfalls that can negatively impact their business?

Prediction markets are an excellent way to assess and evaluate the risks that life science companies face.

For example, a pharmaceutical company is currently developing several new molecules many of which have promise. In particular, the company has developed a new compound that treats a major neurological disease. The company knows that its competitors have other products that are further along in the clinical development process. But the company does not think the competing products are as effective as its molecule. To further complicate the picture, recently a new device was approved by the FDA which treats the disease in a completely different manner. However, before most insurance companies will pay for the device, it needs to be approved for reimbursement by Medicare.

Based on the competitive environment, the company faces a key question: Does it spend its resources on the molecule for the neurological disease or, would it be better positioned if it developed one of its other drugs where it may face less competition but also a smaller potential market?

Using Inkling, the company asks several questions which try to better assess the competitive situation: Will a competitor's product be approved by the FDA before the end of the year? Will Medicare agree to cover the costs associated with the new device? If Medicare agrees to cover the device, what will the total out-of-pocket costs to patients be? Before the end of the year, will an international drug company license its product to a U.S. competitor to ultimately gain entry to U.S. market?

Questions that were largely answered by hunches and guesses can now have probability scenarios assigned to each of them.

As these markets are run, senior management now has much more context from which to make the decision regarding the future of their own molecule.

Curious what the new administration will bring?

One of the great things about hosting a public prediction marketplace run by "the people" is the quick response to current events our traders have. With the results of the election now about 12 hours old, traders on our public marketplace have already submitted several markets about what the new Obama administration will be like. Among them:

• When will President-elect Obama appoint his first Cabinet Secretary?
  
• Who will be appointed the next Secretary of the Treasury?
  
• How many Republicans will be included in Obama's cabinet?
  
• Will Colin Powell serve in President Obama's cabinet?
  

Trade away!

Some Recent Email Changes

A couple changes we've made to New Market Alerts sent via email:

First, we started putting the title of the market in the subject of the email.

Second and more importantly, we received complaints every once in awhile from people being inundated by new market alerts. This would happen when someone would launch many markets in rapid succession and all of the sudden user's email boxes are filled with a dozen new market alerts.

To address this we created a "digest" email that monitors new markets created over a period of time. All markets created within a window are included in a single email. So now you may get a New Market Alert with a single market listed or 6 markets listed. We hope this is a good compromise for those who want to be notified of new markets but don't want to be getting too many emails from the marketplace(s) they're participating in.

Pop!Tech Markets Launches

After having attended the Pop!Tech conference a few times ourselves in the past, we were thrilled with the opportunity to work with the folks at Pop!Tech to launch Pop!Tech Markets.

Pop!Tech is a renowned ideas summit and social innovation network dedicated to accelerating the positive impact of world-changing people and ideas. The organization is known for its visionary Pop!Tech conferences, engaging media productions, and the innovative social change programs that it fosters worldwide.

Nate and I will be attending this year's conference in picturesque Camden, Maine, so if there are any Inkling regulars from our public marketplace or corporate users attending, come find us!

Here's a bit more about this year's conference:

Taking place October 22 – 25 in the seaside village of Camden, Maine, this year's event will chart the core scarcities humans and organizations will encounter this century – and how a wealth of new innovations, bottom-up approaches to collaboration and insights into collective wisdom might hold the keys to addressing the challenges that lie ahead. Now in its 12th year, the Pop!Tech conference is a remarkable gathering of more than 600 thought leaders, influencers and social change agents – from the worlds of science, technology, business, social innovation, the arts, philanthropy and more – that each fall explores the new ideas, technologies and forces of change shaping our future.

Pop!Tech '08 will focus on the rapidly-changing 21st century dynamics of systems based on scarcity and abundance. Many different domains will be explored, from the social web to population growth and energy consumption, from truly sustainable business models to politics and peacemaking. Every topic will be examined through an array of lenses – including the new ethics, economic realities, risks and possibilities in a world defined by dramatic scarcities and abundances. An eclectic group of world-class visionaries will help drive the discussion, including:

Malcolm Gladwell – pop sociologist and journalist. Author of #1 best sellers The Tipping Point and Blink, Gladwell will preview and discuss the themes in his forthcoming book The Outliers – an examination of what separates extraordinary and average people.

Clay Shirky - Internet analyst. An adjunct professor in NYU's graduate Interactive Telecommunications Program and author of the recent hit book Here Comes Everybody: The Power of Organizing without Organizations, Shirky will discuss the potential that’s been unlocked by mass adoption of social media.

Gary Slutkin, M.D. – epidemiologist and anti-violence innovator. Dr. Slutkin will share a breakthrough approach he’s developed to help eradicate violence: treating it like an epidemic. Based at the University of Illinois at Chicago, Slutkin is founder and Executive Director of the Chicago Project for Violence Prevention’s anti-violence program, CeaseFire, which has helped reduce shootings in inner city Chicago neighborhoods by as much as 75%. Previously, at the World Health Organization, he helped Uganda become the first country in Africa to reverse its AIDS epidemic.

Laurie Garrett – global health expert. The only journalist to ever win a Peabody, Polk and Pulitzer (while at Newsday and NPR), Garrett is an acknowledged expert in public health, newly emerging and re-emerging diseases – and their effects on foreign policy and national security. At Pop!Tech she will discuss the current state of global public health and why it’s not working.

Stephen Badylak, D.V.M., Ph.D. and M.D. – regenerative medicine expert. Last year Dr. Badylak shocked the medical world when a remarkable powder he helped develop – culled from pig bladders and containing multiple growth factors that are effective in humans – led two patients to entirely re-grow severed fingertips in just six weeks. At Pop!Tech, Dr. Badylak will detail his work to help Iraqi war veterans re-grow body parts lost in battle.

Suzanne Seggerman – serious games expert. President and co-founder of Games 4 Change, a non-profit advancing the use of video games as tools for social change, Seggerman will survey the burgeoning serious games movement – which uses game technology to address the most serious issues of our day: the Israeli-Palestinian conflict, poverty, global warming and the genocide in Darfur.

Bill Bishop – journalist and author. In The Big Sort, Bishop marshals mountains of original data to illustrate how Americans have sorted themselves into homogonous communities nationwide, becoming culturally and politically divided. At Pop!Tech, he will discuss what this polarization means for our economy, culture and democracy moving forward.

Kelly Dobson – roboticist. Dobson is a researcher and Ph.D. candidate at the MIT Media Lab developing a method of personal, societal, and psychoanalytical engagement termed “Machine Therapy.” At Pop!Tech she will examine the changing relationship between people and machines and exhibit robots she’s developed with “expressive engaging behaviors, strength of character, negotiative egos and neurotic propensities.”

Matt Mason – former pirate radio dj. Author of The Pirate’s Dilemma: How Youth Culture Reinvented Capitalism, Mason is a noted London pirate radio and club dj who will address the economic value of digital piracy.

Chandler Burr – The New York Times fragrance critic. A global scent expert, Burr’s recent book The Perfect Scent: A Year Inside the Perfume Industry in Paris & New York juxtaposes a year behind the scenes at Hermes – shadowing the creation of legendary perfumer Jean-Claude Ellena’s new scent – with a year inside Coty, with Sarah Jessica Parker, as she directed the making of her perfume "Lovely." At Pop!Tech, Burr will lead an interactive tutorial, with actual perfumes and scent raw materials, on the science, business, technology and global geography of scent.

Jay Parkinson, M.D. – Web 2.0 personal care physician. Dr. Parkinson is a founder of Hello Health, a groundbreaking, 21st century healthcare practice, and Chief Imagineer of Myca, developer of an online platform enabling doctors to see and receive client updates and communicate via email, IM, video and SMS. At Pop!Tech he will share how he’s helping doctors become entrepreneurs – utilizing today’s most powerful communications tools – to the benefit of both patients and healthcare providers.

Feeds for Discussion Threads

We now have RSS feeds for discussion threads in individual markets. Just click on the RSS icon next to any Discussion header in a market to track all comments made.

Grass Roots Data vs. Formal Data

I had the pleasure of presenting today at the A.C. Nielsen market research conference at the University of Wisconsin in Madison. Along with one of our consumer product company clients, we presented an overview of prediction markets, how they are being used in companies, and the results of some experimentation we did with them. Good questions during our talk, good conversations with academics and market research practitioners. A productive day all around.

What I wanted to focus this post on however was a moment during the first plenary session I found quite interesting:

The session was led by Pete Rose from Yankelovich who reviewed the major findings of their Monitor report, the " longest-running, most in-depth study of consumer value and lifestyle trends." It was fascinating to sit in a room of market research practitioners and students as they clamored for new information and guidance on how they should interpret the latest U.S. financial crunch in the context of how to market their products. In fact at the conclusion of Pete's excellent presentation, the first questioner asked how the research data we had just been presented changes because of the events of the last two weeks on Wall Street and in Congress. In other words, if I'm a market researcher at a luxury hotel or fancy car company, should I be quaking in my boots? And vice versa, if I'm a market researcher at a budget hotel company, should I be ramping up the marketing budget?

In response, Pete, as a market researcher firmly planted in an empirical research methodology, gave a vague answer and told everyone to wait for their next report to come out for more details. No problem, but wearing my prediction market hat I had a simple thought: why not augment this traditional methodology with a constant stream of questions posed in a prediction market? Grass roots data vs. formal data. We talk about this all the time in other contexts, but for market researchers, is such an approach blasphemy?

Exploring this thought further, let's say we did work for a budget hotel company. We could first identify what the levers are that drive our business. Business travel patterns. Gas prices. Stock market performance. Competitive activity. Then we could recruit people who have insight in to these factors and ask them to participate in an ongoing prediction marketplace. These aren't necessarily just hotel goers, they could be economists, wall street traders, bankers, and professionals from the energy industry. We could run a survey with this same group of people, but how would we know to run it until after the fact? We would have to be conducting surveys on a weekly basis - an expensive proposition. In contrast, a prediction market could be running 24/7 and serve as the early warning indicator that allows our hotel company to be more proactive in light of the changing levers that positively or negatively affect our business. We could even ask this group about the performance metrics we care about directly: "What will our occupancy rate be in Q3?" Even if the market prices didn't provide a clear signal of change we should be aware of, we may see outliers among the traders or be able to gather qualitative data in the discussions accompanying the questions, leading us to follow up with more detailed research or additional questions in the marketplace.

Formal market research reports will always have their value, but why not supplement them with ongoing mechanisms like prediction markets. They may be unconventional but could also serve as a more timely indicator of what's coming down the road, giving us precious time to act.

My first of many blog posts

As recently announced on our blog, I recently joined Inkling as Director of Business Development. I am excited about my new position and look forward to helping the company continue to grow, with the goal of making prediction markets more of a mainstream business intelligence tool in corporate America.

My interest in prediction markets has been growing over the years. I first became aware of them from a web site that was using markets to predict changes in technology trends. In addition, like Inkling's founders Adam and Nate, I have worked for a variety of large companies and many times during my 20 year career, I was extremely frustrated when decisions were made by a select number of senior managers, without taking advantage of all the knowledge around them. In my personal experience, decisions made by senior management were often altered or even reversed when it became apparent that they were wrong; it was only then that less senior employees were consulted and their insights considered. Not the most productive way to run a company…

Not long before I was introduced to Inkling, I had the opportunity to meet James Surowiecki, the author of “Wisdom of the Crowds” at a business meeting in Miami. I am an incredibly loyal reader of Surowiecki’s New Yorker column, but I had never read his book. After, Suroweicki’s talk, I was given a signed copy of “Wisdom of the Crowds”; I read the entire book on the plane ride home to Chicago. Then, just a few months later, I met the Inkling founders and knew this was the opportunity for me.

As part of my role I am interested in getting feedback from traders regarding how to improve the Inkling platform. I know a lot of Inkling’s traders provide comments and suggestions, and I encourage you to keep sending them.

Finally, during my first weeks at Inkling, I created a number of markets on our public marketplace and am anxious to see how they turn out. Some of the markets I have created include:

• What will the voter turnout be for the presidential election in November? At this point, there is no clear consensus on the question so getting more insights would be great.

  
  

• For 2008, how much will renewable energy represent of the nation’s total energy consumption? With so many variables to consider as it relates to this question – gas prices, regulation, energy demand – it will be interesting to see the ultimate answer.

  
  

• What will the market share for Google's new browser (Chrome) be at 12/31/2008? Considering all that Google has accomplished in its short ten year life, it easy to imagine that Chrome will ultimately be a winner. But how long will it take?
  

I look forward to hearing from you in the future.

New Graphs

As many have already noticed and sent us comments about, we have introduced new graphing capabilities in our markets. These graphs are much more dynamic than our previous graphs. They allow you to "grab" the graph and move it back and forth along the timeline, zoom in or out by time period, and enter a span of dates to view. We've also addressed the problem of having a gaggle of stocks on a graph by allowing you to select and de-select the specific stocks that are shown so you can easily compare stocks or remove others that are visually in the way.

What we're most excited about is the ability to plot stories from the market's RSS feed directly on the graph so you can see the potential correlation between news and trading activity. In conjunction, we've made it easy to include a Yahoo or Twitter RSS feed with any market in one click so more people hopefully take advantage of this graphing feature.



Additional graph-related widgets and exporting of images (vs. Flash) are coming soon. If you have other feedback, let us know.

Please Give a Warm Welcome...

Please give a warm welcome to Heidi Levin who is the newest member of the Inkling team.

Heidi joins us from Northern Trust Global Advisors here in Chicago where she served as vice president of the private equity group and managed a diversified private equity fund of funds. She has a wealth of experience in business development and forming and managing strategic partnerships that should serve her well in her new role as Director of Business Development here at Inkling.

We're extremely excited to be working with Heidi and know she's going to do a fantastic job.

We've invited her to post her thoughts from time to time on this blog so you should be hearing from her directly soon.

Facts are Better than Dreams

In the book Good to Great, Jim Collins looks to understand what makes a good company "great." Why do some companies stagnate or grow anemically while others grow explosively and achieve long term success?

In re-reading his book recently, I was struck by a section called "Facts are Better than Dreams." From the book:

"...This, of course, begs a question. Are we merely studying a set of companies that just happened by luck to stumble into the right set of decision? Or was there something distinctive about their process that dramatically increased the likelihood of being right? The answer, it turns out, is that there was something quite distinctive about their process.

The good-to-great companies displayed two distinctive forms of disciplined thought. The first...is that they infused the entire process with the brutal facts of reality. And the second...is that they developed a simple, yet deeply insightful, frame of reference for all decisions. When, as in the Kroger case, you start with an honest and diligent effort to determine the truth of the situation, the right decisions often become self-evident. Not always, of course, but often. And even if all decisions do not become self-evident, one thing is certain: You absolutely cannot make a series of good decisions without first confronting the brutal facts. The good-to-great companies operated in accordance with this principle, and the comparison companies generally did not."

A topic we regularly discuss with prospective clients is their desire not to "embarrass" anyone with questions that are asked in the marketplace. For example, asking about a project milestone or key performance metric and potentially exposing bad news would be embarrassing for managers and so the initial reaction is the question should not be asked at all. Sometimes companies get beyond this initial fear and find the insight is actually worth the risk of embarrassment and other times we've been told a company simply could not stomach doing something that made information flow so transparent.

Looking back at my notes of conversations with companies over the past several months and plotting out an "organizational acceptance" graph, it was interesting to note the companies who had a primary concern about exposure of reality were typically those that were either struggling in their industry space or were already known as bastions of bureaucracy.

Those companies who were willing to introduce prediction markets were either companies who have always been innovators in how they run their business, or companies where a leader or maverick manager had made a commitment to make information flow more transparent, egg on face be damned.

The Olympics

We don't talk much about our public marketplace but it continues to thrive with new markets being created and hundreds of trades being transacted every day. We're regularly amazed at the diversity of markets people run, and with the entire world focused on Beijing for the 2008 Summer Olympics, it was inevitable many markets would be created asking about the results of individual events, China's performance as a host country, and overall medal standings at the end of competition.

Cue the John Williams Olympic Fanfare music and see how good you are at predicting the outcome of anything and everything Summer Olympics 2008.

Why did you trade that way?

In any Inkling market, traders express their opinions through their trades and in the discussion threads associated with each market.

Now there is an opportunity to capture feedback (optional of course) about why someone traded the way they did for each specific trade. This input is especially helpful to those who are trying to use the prediction market outcome as input to resource allocation decisions, understanding risk probabilities, or simply as input towards strategic decisions.

This new source of free-form information may also help to more easily identify outliers - now both the trade data and anonymous explanations are at a marketplace owner's disposal. Let's say a question is asked about the probability of a quality assurance issue arising in a product about to go on the market. If the market says the probability is low but there are a few people giving specific explanations as to why they feel the opposite, minimally the market owner can begin asking questions and gather more information.

In capturing this information we've tried not to interrupt the simple trading experience. It's just an optional field on the final confirmation screen (click to enlarge):

Better onsite alerts

We've put in a new system to show better onsite notifications and errors.  We used to show them as color bars in the bottom of the browser window, but people commented they didn't always notice them. Now you'll see them in the top right of your browser window. These notifications alert you to successful actions like completing a trade:

or warnings that something didn't go quite right, like an unsuccessful login:

We hope they'll be easier for people to spot and at the same time not be intrusive. They were of course inspired by the wonderful Growl notifications software for the Mac and were made possible by Stan Lemon's excellent jGrowl library, a javascript implementation of Growl.

Startup Ideas yCombinator is Looking to Fund

We're an early graduate of yCombinator, the seed fund started by Paul Graham. Today Paul outlined 30 areas they would like to invest in.

If you're looking to start a company, take a look - the breadth of the ideas is far and wide, including an area near and dear to our own hearts, Enterprise 2.0.

Judging a prediction market's accuracy

Recently we wrote a blog post about the accuracy of prediction markets. As we promised, we follow that up now with more specific data.

Two years ago, Google wrote a blog post about how their internal prediction markets were working. It was an inspiring picture and one that got many people excited about using prediction markets. Now that we've been hosting prediction markets for 2.5 years, we have quite a bit of our own data. Looking at well over 2 million trades and thousands of markets across hundreds of marketplaces, we can easily say Google's impressive results weren't a fluke.

The most popular type of market our market makers create determines the probability of an event happening: Will David Cook win American Idol?, Will Miller Chill still be sold in 6 months, Will a new aircraft design be delivered on time?

To beat the proverbial departed equine, you can't just look at a single outcome of one prediction market question to determine how "accurate" your marketplace is.

When Mrs. Burnette told us in 4th grade that flipping a coin had .50 probability of coming up heads, she didn't stop there. She made us measure it to prove that this was in fact true by measuring the relative frequency of heads coming up. For homework we had to flip the coin over and over and write down the outcome. Flipping a nickel at home I just got: heads tails heads heads tails heads tails tails tails tails heads heads tails. Using our very new skills with fractions we could then measure the probability by computing 6 heads/13 trials = 0.46, pretty close to 0.5.

Mrs. Burnette appeared to know what she was talking about.

So just like flipping a coin, if Inkling told you something has a 15% probability of coming true, you can't just look at one outcome (i.e. one coin flip). You need to look at multiple scenarios where Inkling said something would happen 15% of the time. If those things actually come true, 15% of the time, Inkling is doing well at this.

We plotted a graph a lot like Google showed 2 years ago. Count the number of markets that predicted an event would occur 5% of the time, and see how many of those occurred: almost 5% of them. Count the number of markets that predicted an event would occur 15% of the time, and sure enough 15% of them ended up occurring. And so on. Until we got the graph below.



The green line is what we’d look like if we were perfect: things predicted to happen 15% of the time happen 15% of the time, things predicted to happen 65% of the time happen 65% of the time, etc. Inkling is the black line hugging pretty close to perfect.

Another type of prediction market is one that predicts the numerical outcome of something: What will the population of New York City be in 2010, How many utility patent applications will be filed in the US in 2014?

In this case we’d like to see a plot of what is the value we predicted with what actually happened. We plotted hundred of these markets in the graph below.



The green line is perfect again. We’d be perfect if Inkling said you’d sell 100 units of something, and you sold 100 units. If Inkling said you’d sell 1000 units, you sold 1000 units. The red line is a line of best fit through the data. Not too shabby.

We've discussed several times on this blog and elsewhere the accuracy of a marketplace. There are significant misconceptions about what the results of a prediction market actually mean, especially in the media. Hopefully these graphs reinforce what a prediction market is revealing as this is the first step in using the new information as input to strategic decision making, etc.