Wednesday, April 02, 2014

New Job Available - Rails Developer

We are hiring a new Rails Developer. Information about the position is posted here:

If you're interested in applying, we'd love to hear from you. If you know anyone that hates their current job and is looking for something way better, we would appreciate you passing this along to them. :)

So You Think You're Smarter Than a CIA Agent...

A government funded project we're participating in received a nice write up and audio story on NPR this morning.

Read the article and hear NPR's broadcast of the story here.

The Good Judgment Project originated out of IARPA's ACE program which is an attempt to push the state of the art on crowdsourced forecasting techniques and also see if "crowds" without access to classified information would be better forecasters than those inside the intelligence apparatus.

From the article:
According to one report, the predictions made by the Good Judgment Project are often better even than intelligence analysts with access to classified information, and many of the people involved in the project have been astonished by its success at making accurate predictions.
But there are caveats to this as the IARPA project sponsor Jason Matheny points out:
Matheny doesn't think there's any risk that it will replace intelligence services as they exist. 
"I think it's a complement to methods rather than a substitute," he said. 
Matheny said that though Good Judgment predictions have been extremely accurate on the questions they've asked so far, it's not clear that this process will work in every situation. 
"There are likely to be other types of questions for which open source information isn't likely to be enough," he added.

Monday, March 03, 2014

2014 Oscars - How did Inkling do?

The Oscars are over, the Hollywood elite are home in their mansions nursing their hangovers and taking an in-house spa day, and the rest of us are left to gossip about the highlights and lowlights of the show last night.

With that in mind, let's see how Inkling did in its predictions.

Of the 18 categories we reported on last night before the show started, Inkling correctly predicted 17 of them for a 94% hit rate. The one miss? Original Song. Damn.

Kudos to all those who participated and made predictions. Given these are crowdsourced predictions, we're nothing without you. :)

Sunday, March 02, 2014

2014 Oscar Prediction Guide

The Oscars begin shortly and here's Inkling's predictions. Each winner Inkling predicted has the highest chance of winning among its fellow nominees in the category. Remember, we deal in probabilities, so all we're telling you is the chance a nominee will win, like when the weather man tells you there's a 70% chance of rain, there's a 30% chance it won't rain!

Read on after the predictions for a little more about how to interpret these results and how we calculate them.

Alright, on to the predictions:

Which film will win the most Oscars?
Gravity (75% chance)

Best Picture
12 Years a Slave (76% chance)

Best Actress 
Cate Blanchett (82% chance)

Best Actor
Matthew McConaughey (55% chance)

Supporting Actress
Lupita Nyong'o (68% chance)

Supporting Actor
Jared Leto (76% chance)

Original Screenplay
Her (48% chance)

Adapted Screenplay
12 Years a Slave (80% chance)

Visual Effects
Gravity (80% chance)

Sound Mixing
Gravity (39% chance)

Sound Editing
Gravity (38% chance)

Original Song
Happy (53% chance)

Original Score
Gravity (31% chance)

Makeup and Hair Styling
Dallas Buyers Club (51% chance)

Foreign Language Film
The Great Beauty (57% chance)

Best Director
Gravity (53% chance)

Costume Design
The Great Gatsby (34% chance)

Gravity (63% chance)

Animated Feature Film
Frozen (92% chance)

To see details about each prediction, like what chances the other nominees have, go here:

As you can see, sometimes there is a strong signal from people making predictions when they think they know who the winner will be, i.e. Animated Feature Film (Frozen 92%) or Best Actress (Cate Blanchett 82%) - but other times there isn't much of a signal at all (costume design, original score, sound editing, etc.) This lack of a signal can be for two reasons. First, sometimes there just aren't that many predictions made so the chances don't move very much from where they started (evenly split up among however many nominees there are, i.e. 5 nominees = 20% chance each). But other time no strong signal can mean disagreement among people about what the outcome will be. For Original Screenplay, for example, there's a 48% chance "Her" will win. But this was after 36 predictions. In contrast, there were only 18 predictions made in the Adapted Screenplay category, but those 18 predictions amounted to an 80% chance "12 Years a Slave" would win, meaning there was almost unanimous opinion among those who made the predictions. "Her" was a more controversial pick in that category.

If you have more questions about how this all works, find us on twitter at @inklinghq.

Enjoy the show tonight!

Tuesday, January 21, 2014

A little "less" in "relentless"

As someone running a small company with a handful of employees and several contractors, I've had to learn the limits of how hard I can push people and what I can ask them to do. I suspect other founders or managers in larger companies could stand to examine this aspect of their leadership style as well.

A few months ago we took on a very large consulting project helping George Mason University and IARPA launch and grow a new prediction market focused on Science and Technology forecasting called SciCast (it's launched, you should check it out!). While the project has gone well thus far, it has also presented our small company with a significant set of challenges: namely that we still have product development, customer support, consulting, and sales activities for our own product line, Inkling Markets! 

Where as before our interaction with our clients was controlled and routine, the SciCast project is a traditional consulting engagement with deliverables, timelines, a demanding client, multiple collaborators, and metrics we've been asked to meet. As anyone who has been a consultant knows, this level of accountability can be a serious amount of work. And with that amount of work inevitably comes adversity and additional pressure to succeed.

Unfortunately, this is where things tend to get dangerous. 

Doing well on this project, trying to grow our revenue, building our company for long term success - these are all goals that are of primary importance to me as the founder, but only secondary importance to any employee. An employee may partially share my goals for the business and see its growth as beneficial to their sustained employment, but they have other incentives at work as well. They want to make a good salary, they want to maximize bonuses, and they want to build experience for gaining more responsibility in the company or for their next job outside the company.  For us to have the most productive working relationship, we must reach a balance. That balance is lost if I'm making excess demands on people's time and well-being.

Before working at Inkling I worked at a large consulting firm for many years. Consulting firms are famous for the "grind" - working 6 or 7 days a week on a client project, excessive travel, 16+ hour days. If you talk to anyone in the middle of one of those projects, you hear words like "death march," "fire drill," and "insanity." Managers are hard charging and tend to focus more on making sure short-term tasks are getting done on time vs. the long view of burning people out. One not need to look far then to understand why attrition rates at consulting firms are high and why a disproportionate number of people leave after only a few years. 

That model may work (despite itself) when a large business can afford to have a human resources machine drumming up interest at universities around the world for new employees to indoctrinate, but it doesn't work so well for a small business whose switching cost at losing even a single employee can be quite burdensome.

With that in mind, I've tried to catalog some warning signs that you may be pushing your team a little too hard and need to ease up:
  • People start becoming unresponsive or orders of magnitude slower responding to requests;
  • Quality of work diminishes - just doing tasks for the sake of doing them;
  • People are quiet in discussions and don't proactively offer up opinions or solutions; and most obviously
  • Openly complaining about the amount of work or the tedium of completing it.

"But we still have to meet our deadlines and how do we do it without working people hard," you ask?

Sometimes the problem may not be quantity, but quality. Is everyone working as effectively as they can? Is someone spinning their wheels on something unnecessarily? Sometimes you may just have to bite the bullet and tell your client you simply cannot get something done on time. My experience more often than not is if you are transparent about the likelihood of missing a deadline as early as possible and give valid reasons for why it can't get done, people are understanding and will work with you to narrow the scope of your work or agree to an extension. If there is no other option left than to put yourself through a "fire drill," try to keep the length as short as possible and ease off when it's over for a few days. I strongly believe in the concept of vacationing while not actually on vacation.

As a founder, no one is going to be as passionate about your business as you will be. If you want your team to work hard, they need to share in some of that passion and be incentivized correctly. If they are not, their shelf life will be short. Eventually, people break and simply decide it's not worth it anymore. When that happens, you're on your own with no team at all, and that's the worst outcome of all.

Like this post and want to know when I write others every once in a great while? Follow me on Twitter

Wednesday, December 04, 2013

Inkling Predictions iOS app updates (v1.2)

The latest version of the Inkling iOS app is now available on the app store. New features include:

  • Native alerts for new questions that have been published and new comments made in questions you're participating in
  • Support for our "advanced" trading interface
  • Lots of bug fixes and layout improvements
Please be sure to update and download the latest version and as always let us know if you have any feedback, suggestions for new features or issues with something not working.

Wednesday, October 30, 2013

Inkling Predictions for iOS 1.1

We were late to the party in updating our app for iOS7 but finally have a version out that is compatible and doesn't crash. Anyone who has a current version and has upgraded to iOS7 should run an update and everything will work again.

We're also close to doing another release that will add a bunch of new features including advanced trading, native alerts, setting your own price alerts, and more. It should be an nice upgrade and will be available in about a month.

Tuesday, September 10, 2013

Realtime dashboard, stats, and an evolving look

A few application improvements we've made recently:

  • On your dashboard the data was always stale. It was automatically refreshed every hour, but this could get confusing in active questions where your profit/loss had gone up or down quickly, but you couldn't tell unless you manually clicked "refresh" for that position. Now all your data on the dashboard is updated in real time.
  • We're taking a look at each email sent by Inkling to make sure it has valuable information in it and re-building them if they don't. For example we just re-did the emails you get when a question ends. Now it should be much clearer exactly how you did and what happened overall in the question. 
  • Stats are updating now much more frequently
  • We're slowly migrating to a new look for the application. And if you haven't noticed, we updated our logo too.
  • We've made a ton of "under the hood" improvements which should make the application feel snappier. 
For our enterprise customers, we've introduced several new capabilities including:

  • Enhanced reporting capabilities allowing you to fine tune what data you want to pull down from what time periods.
  • In addition to collecting people's predictions, you can now ask for a specific probability value as another input.
  • Lots of enhancements to the API. 

Tuesday, July 02, 2013

Psychology Influences Markets, Research Confirms

When it comes to economics versus psychology, score one for psychology. Economists argue that markets usually reflect rational behavior—that is, the dominant players in a market, such as the hedge-fund managers who make billions of dollars' worth of trades, almost always make well-informed and objective decisions. Psychologists, on the other hand, say that markets are not immune from human irrationality, whether that irrationality is due to optimism, fear, greed, or other forces.
Now, a new analysis published in the XX issue of the Proceedings of the National Academy of Sciences (PNAS) supports the latter case, showing that markets are indeed susceptible to .

Thursday, June 20, 2013

Acknowledging Everyone's Contributions Through Revenue Sharing

When I was a senior in college I worked in the kitchen at a pizza place called Lennie's in Bloomington, Indiana to make a little extra money. I made pizzas, sandwiches, pasta, and salads. I washed dishes, I coordinated the food going out to the tables, and I helped clean up at the end of the night. Typically I worked with 3 or 4 other people in the kitchen, along with 4 or 5 wait staff and a manager. Friday and Saturday nights during the school year were always incredibly hectic as Lennie's was, and still is, one of the more popular destinations in town. Needless to say, everyone was bone tired by the end of the night.

One of the lessons you learn working in the kitchen of a restaurant is the symbiotic relationship between the kitchen and the front of the house. Unless you've actually worked at a restaurant, I don't think you can appreciate the level of team work, dependency, and coordination that are involved in delivering a quality experience to the customer.

Management at Lennie's seemed to understand this and structured payment to their workers accordingly. On top of everyone's base hourly wage, the wait staff earned tips. But at the end of the night they left a percentage of their tips to the kitchen, who then divided them evenly. Sometimes if the kitchen had bailed a waitperson out in some way, say they had mis-entered a ticket and we had to remake a customer's food, that waitperson would give us a little extra to say thanks.

Fast forward 5 years to when I became a manager at a global consulting firm. They had a profit sharing plan too, but the profit target was arbitrarily set by the CFO and no one really knew how close or far away we were from hitting that target. It also felt very abstract to the point that if I saw some of that money at the end of the year, great, but I certainly wasn't banking on it. More troubling to me however was when I began selling work on behalf of the company. I wasn't a "salesman" per se, but in the course of doing business with a client, I would certainly be responsible for "inside sales" and grow the account over time. Yet I continued to make my monthly salary. I sold projects worth 6 and 7 figures, but saw no direct benefit monetarily.

The natural answer most companies go to then is a commission model. But commissions are usually reserved for sales people. Sales people are on the front lines doing the work to find new business and closing deals and they should be rewarded for that. But they would have nothing to sell if it weren't for the hard work of the teams behind the scenes. Stock options are also a tool companies use to reward employees and do technically give them ownership, but at startups and smaller companies like ours, those are far riskier than options in publicly traded companies. Options could indeed end up being worth a lot of money, or more likely, end up being worth very little.

At Inkling, we've been doing well enough that in the past few months we've hired 3 new people to join our team. In doing so, we had to think about new compensation packages and what they should entail. One of the things we're going to try is a revenue sharing and commission structure that applies to anyone in the company. Any revenue we bring in in the future, each employee gets a cut. If they are instrumental in a new sale, they get an additional cut on top of that. These cuts will be paid out the month the sale happens, not at the end of the year.

Sales matter and keep the company in existence. But sales wouldn't occur without a team behind the scenes working on a quality product. We're going to try and openly acknowledge this relationship just like it existed at the pizza place. As the guy primarily responsible for selling however, I'll try not to screw up too many orders.

Tuesday, June 11, 2013

Inkling Demoing at WorldFuture 2013 in Chicago on July 19th

The World Future Society's annual conference, WorldFuture 2013: Exploring the Next Horizon brings together the world’s premier minds to discuss the long-range future of science, technology, humanity, government, religion and many other topics. Sometimes called a “World’s Fair of Ideas,” WorldFuture 2013 will feature MIT Media Lab founder Nicholas Negroponte, visionary author Ramez Naam, Ford futurist Sheryl Connelly, and geosecurity expert John Watts.

The conference starts July 19th.

Being local folk, we've been asked to be part of the BetaLaunch event the night the conference starts, right after the Negroponte keynote. If you're attending the event, come by and say hello. It looks like it's going to be a fun conference!

Sunday, June 02, 2013

Risk Exposure as a Risk Assessment Metric using Probability Markets

Adam demonstrated how Inkling can be used as a risk assessment tool to estimate the probability of an impact occurring. This impact probability equation can be formulated as:

impact likelihood = probability of risk * impact likelihood given risk has occurred

Prediction markets are exceptionally well-suited for doing these kinds of impact assessments, especially in public policy concerns where risk probabilities are not easily derived from historical data or where public participation is an essential part of the deliberation process. You can take an impact assessment one step further by measuring the magnitude of a given potential impact with regard to its likelihood, commonly referred to as risk exposure. Risk exposure is the risk adjusted value of the consequences should a risk become realized.

Once you've derived a likelihood of an impact occurring, you can then quantify your risk exposure by multiplying the probability of the event occurring by the value of the total loss of risk.

risk exposure = probability of risk * total loss of risk

... or to calculate the risk exposure of an impact:

risk exposure = (probability of risk * probability of impact) * total loss of risk

The total loss of risk can be thought of in terms of the value of an asset, shift or loss in demand, number of people affected, value of ecosystem services, or any other quantifiable amount of utility that may be lost if a risk is realized. Total loss of risk may be something you already know, such as the value of your real estate assets or it could be a value derived from a third prediction market. For example, asking a question in a prediction market such as "how many barrels of oil could flow through phase IV of the Keystone pipeline in 2014?" will give you the total loss of risk of barrels of oil coming in from Canada via the Keystone pipeline. Multiplying that number by the probability that the segment is not built because of environmental policy outcomes will give you your risk exposure which can be weighed vs your investment. 

As a risk manager, an important thing to remember when dealing with total loss of risk exposures is that a low probability of a high loss of risk may be equivalent to a high probability of a low loss of risk. Therefore, many risk managers will construct risk matrices when evaluating their portfolio's risk exposure in order to see a full range of their risk decisions. Loss of life, for example should not be boiled down to risk exposure as a low probability of many lives lost is not equivalent to a high probability of a few lives lost. They are just not comparable.

As you can see, building a resilient portfolio is dependent upon minimizing your risk exposure. This can be done by: 1) setting aside enough resources to cover your risk exposure; 2) reducing the amount of total loss of risk you are taking on; 3) reducing your risk probability; or 4) mitigating the potential impacts should a risk occur. Finding the right balance between these 4 variables is the art and science of risk management.

We feel that prediction markets naturally lend themselves towards robust risk assessment tools. If you have used Inkling for risk management purposes in your organization please write us, we'd love to hear about it.

Pat Carolan
Inkling Markets

Friday, May 17, 2013

The Road to Making - From Starter League to Developer

In college I studied economics. I recommend it to anyone who feels comfortable with the idea of knowing a lot about the world's woes and having very little ability to do anything about it. When I graduated, applying for jobs introduced me to a harsh reality, I had a lot of ideas but my tradecraft was lacking. I'd been huffing the sweet ethers of theory but yearned for the sobering oxygen of practice.

After college I caught a break and landed in the field of business intelligence (BI) before it really had a name other than "IT". Everyone has a different definition of BI, but here's mine: BI is the art and craft of persuasion using data. That's it. However you do it, be it graphs, regressions, tables, art, words, it doesn't really matter. It's the message that's the thing not the medium. Your job as a BI analyst is to gather and present the best version of the truth that you possibly can given an imperfect model of what you're trying to represent. You solve puzzles using the scientific method. It's a discipline where you strive for objectivity asymptotically.

I designed models that priced risk, but I also spec'd out the feeds that transmitted data and drew the diagrams showing how data would be related, stored and retrieved. Only, I didn't really build any of it. I was an analyst. It was my job to derive the logic, design the spec, write the requirements. We called ourselves designers and architects because that's a pretty good analogy, but our jobs stopped at the blueprint and the craftsmen took over from there. Analysts wrote the spec, developers built the product. If you're good, you know how to find the sweet spot between too much and not enough detail and you know the dimensions of a 2x4. But at the end of the day, I was still the guy who took the requirements from the customer to the developers, a people person dammit.

But I wanted to make things. Knowing how to design something that works isn't exactly the same as knowing how to build it. After designing the inputs and outputs of BI systems for a while you develop theories about how it could be done better. You think to yourself that given a chance to build it yourself, you can do it fitter, faster, more productive. This is the worst sort of hubris of course and the gods are laughing at you while you put your boots on but I'll come back to that. So after being an analyst and researcher for almost 10 years I decided I had to learn some tradecraft. I had picked up SQL and R along the way... how much harder could app development be i asked myself? My mom and friends already thought I was building apps like Zuck. How hard could it be? Damned hard, it turns out.

I enrolled in the second class of the Starter League during the winter of 2012. Starter League is the world's best learn-to-code school and is located in Chicago. It teaches web app development in 12 weeks. Starter League recently partnered with 37signals and is changing lives, I've seen it. All that said, everyone in my class completely sucked at web app development when we started. With no exception, we all struggled to grasp the fundamentals. Even the CS graduates didn't have a professional quality app ready by graduation. There were no "naturals". With time, many of us got better and went on to good jobs. But it was easily the hardest thing I've ever done. What programmers forget, but what is obvious to someone coming from functionally siloed enterprise work is that these "easy" web development frameworks are polyglots of various languages, DSLs, domains, competing theories and disciplines. You have to know a ton of stuff to build anything useful. Here's a short (incomplete) list of what you need to know to launch the simplest rails app:

- Ruby (and its gems, many of which are are themselves domain specific languages)
- Rails
- Javascript
- Unix
- Database Administration
- Dev Ops
- Application Architecture
- QA methodology

So after the Starter League, I was still just getting started. I spent a year after that working on projects with friends while working full-time in BI. During that time my wife and I had a child and I did my coding during my spare time. I spent 10-20 hours per week at night and on weekends building apps and "rm -rf"ing them just as quickly. The funny thing about building these toy apps is that I learned to love doing it just because. For the same reason my dad takes apart lawnmowers and rebuilds chainsaws, programming is absolutely wonderful work.

Then, after practicing for a year, Adam Siegel, Inkling Markets' CEO, sent me an email. He was looking for a BI guy with rails experience! I was honest and up front about my abilities. I told him that I was not a pro. He was willing to hire me anyways because he believed I could learn and because he wanted to build out his reporting and analytics. He also wanted someone that would be interested in prediction markets. Perhaps an econ background has some use after all! It sounded like a perfect fit. I believe the cliche that luck happens when preparation meets opportunity, so I did not hesitate to take the job.

So here I am finally making stuff. Practicing my new tradecraft, learning the superpowers of the programming elite and earning a good living. They even call me padawan. Now back to the part of the story where the gods laugh at my foolish pride. Working at a software shop is sacred. I wear my hoody with respect because you wield the power to create, destroy and hurt yourself. It is as intense as being handed the keys to an F-14 and told, "don't crash it kid", and it is challenging. Everyday more challenging than the last. Since joining Inkling, I have written code I am proud of but the learning curve has been steep. My predecessors are YC grads, 37signals employees and Accenture Labs guys. The code is dense and deep. I came in wanting to design the ultimate end-to-end real-time BI solution in D3 and Ember but have spent much of my time preparing for that day by writing tests, migrating reports to email and climbing the very steep D3 learning curve. Another cliche I believe in is crawl-walk-run. After a couple of months, I now think I'm starting to walk and the road is high. All-in-all though, it has been worth it every step of the way.

If there's anything I can pass on to would-be-makers that I've learned, it's this:

Doing analysis, reading hacker news and conducting research is good at teaching you where you're headed before you get there but it's not the real thing. If you're a good developer, respect what analysts bring to the table; if you're an analyst trying to become a developer, try to unlearn what you've learned (not that one is better, it's just a different mindset).

Don't write any code you don't understand (you'll get less done, but you will learn better)... unless you're using a gem... then that's half the point. You'll get faster.

Don't overshoot your abilities too much in job interviews. The demand for developers is good enough that if you work hard to learn your craft, you'll find a job eventually. A good employer has the long view and will hire you because they believe your good ideas will soon be met by your ability to realize them.

Pat Carolan
Engineer and Data Analyst
Inkling Markets

Thursday, May 02, 2013

New Report Features

Today we made some updates to the ‘All Trades’ and the ‘Price Changes’ reports which site administrators can begin using right away. Here is a list of the changes:

All Trades Report:

   added: stock status
   added: market status

Price Changes Report:

   added: price status
   added: stock status
   added: market status
   added: market type
   added: price id

The ‘All Trades’ report now has two new columns: stock status and market status. The ‘stock status’ column will show you the current status of the stock (also known as your possible answer) for every trade. When an answer is resolved, it will flag all trades for that market in the report as either ‘closed’ or ‘open’ depending on whether or not the question has resolved.This may be different than the status of the market (the question you’re asking) in the case where multiple answers may be correct, or if the possible answers are independent of each other (for example, in the question “What will the closing price be on these days in 2013 for Google?”, the stock for the answer last day in March (0LDIM) may cease trading at the end of march, while June, September and December stocks continue active trading).

The ‘market status’ column, similarly, will tell you the status of the market (also known as your question). We had users coming to us requesting these new columns so they could better understand what was happening with trading in their active markets or when they needed to run historical analyses only on their closed markets.

We also added five new columns to the ‘Price Changes’ report. The price changes report shows every prediction on a site and is often used to answer time series questions about Inkling data. The new columns are: price status, stock status, market status, market type, and price id. The price status column, tells you how the price was generated in the market. When you create a new market an initial price is set by the user who generates the question. This is the question originator’s prediction and will be flagged with a status of ‘initial’. This status may be useful if you do not wish to include the starting price in your analysis. A status of ‘market’ indicates that the price was derived via a trade in an active market and a price status of ‘final’ tells you the price set when the answer (stock) was resolved. Final prices are set to either 100 or 0 and are often removed when analyzing price movements. A status of ‘final: no trades’ is set when the answer was resolved without any trading having occurred for that stock.

Because different market types have very different characteristics, as a site administrator you may want to perform your analysis on only certain kinds of markets. The market status column will give you what type of market the prediction occurred in. For most of Inkling’s markets, this will be one of the following: Binary (yes or no answer), a DateMarket (what specific day will something happen), a DateRange (between what range of days will an event occur), Futures (the answer is a number), or Options (multiple choice).

Finally, we added the price id to give you a unique identifier for every prediction on your site, this is typically useful when you are doing analysis where you need to drill down on a single prediction or join data sets together relationally.

These changes lay some of the groundwork for some exciting features on the horizon in our graphing and reporting capabilities. Check back often to see what we’re doing to try to make Inkling’s data more accessible and insightful.

Pat Carolan
BI Specialist
Inkling Markets

Monday, April 22, 2013

Water Policy Markets Launched

A few months ago, Rod Smith from Stratecon Inc. a boutique consulting firm that serves the water industry out of Claremont, CA, reached out to us with an interesting idea: create an expert network of water industry professionals and use an Inkling prediction market to ask them questions about the industry, then analyze the data and apply his firm's expertise to create an entirely new form of industry analysis. Stratecon also plans to offer private prediction markets to its clients that will be for employees of that organization only.

A couple weeks ago, Stratecon soft launched its offering here. And here is a press release about the launch.

We're excited to be working with Stratecon and think the idea of using prediction markets in conjunction with expert networks is a promising idea we'll be exploring a lot more in the coming months.

Friday, February 22, 2013

Crowdsourced Academy Awards Predictions

This Sunday evening the stars will all come together in Hollywood as they always do once a year for the 2013 Academy Awards.

Every year in our public prediction market someone publishes questions about practically every category rewarded, and this year was no different.

We decided to have a little fun this year though and create a separate site just to showcase Academy Awards predictions. You can see it here:

After working up the design, building the site was a breeze. We used widgets from the public site to both display the real-time predictions and to allow visitors to make their own predictions. The widgets are also built to allow you to customize their design via CSS so they can match whatever site you're displaying them at.

Because all of the interaction and calculations are handled on the Inkling side, we then just needed to build a "static" site with some HTML, CSS, and Javascript. Instead of hosting it ourselves however, we decided to just host the site at Amazon S3. A recently introduced domain service called Route 53 made this even easier.

This was a fun project to work on and I suspect we'll be doing more of these in the future. March Madness anyone?

Wednesday, November 28, 2012

Intrade's Unfortunate Encounter With the CFTC

Eric Zitzewitz, an Associate Professor at Dartmouth and expert in prediction markets wrote a great opinion piece for Bloomberg about why Intrade should not be sued by the CFTC and subsequently why U.S. citizens should still have access to participate in Intrade. You know, just like the rest of the world does.

Sunday, November 04, 2012

Inkling Predictions iOS Update

Version 1.0.1 of Inkling Predictions is now available in the app store.

We fixed several bugs and made the app compatible with the iPhone 5.

As always if you have any feedback or feature suggestions for us, be sure to let us know.

Wednesday, September 12, 2012

Inkling Predictions, our iOS app is now available

After getting through a couple review hurdles with Apple, our iOS app for Inkling is now available!

"Inkling Predictions" can be downloaded to your iOS device from here:

The app can be used to access our public site at or to access any corporate site you're already a member of.

You can make predictions, review your dashboard, view and make comments for any question, and see your balances.

We look forward to continuing to develop more features and see our app as overcoming a big hurdle in making it easier for anyone to make predictions.

Thursday, August 30, 2012

Front loaded incentives using fantasy currency

As a software product considered "optional" to use in most settings, we have to really worry about incentives for people to participate, both as part of our software (game mechanics) and what we advise our clients to do outside the software, i.e. recognition for participation, prizes, etc.

With that in mind I've been thinking a lot about an article that appeared on TechCrunch a couple weeks ago about a study done at the University of Chicago regarding the level of performance of public school teachers under two different incentive programs.

In the first group, teachers were rewarded at the end of the year based on their student's performance on a standardized test. For every percent improvement over their school district's average, they would receive up to $4,000.

In the second group, teachers were told they had been given $4,000 at the beginning of the year and that number would be reduced based on how their students did. For every percentage point improvement over the average, that would be the amount of money they could keep.

As you may have guessed from the title of this blog post, the teachers who had something to lose performed better. Their students were on average 10% better than the district average. The teachers who were given a bonus offer at the end of the year showed no improvement.

Here was the money quote (no pun intended):

"The results of our experiment are consistent with over 30 years of psychological and economic research on the power of loss aversion to motivate behavior: Students whose teachers in the 'loss' treatment of the experiment showed large and significant gains in their math test scores," said List, the Homer J. Livingston Professor in Economics at UChicago.

Current "best practices" about incentives in software usage suggest a diverse approach of rewards, multiple leader boards, a certain rate and style of marketing and status communication, and development of community and interaction.

While there are many things we could improve in Inkling, we're already doing a lot of this with mixed results. These types of incentives seem to appeal to a certain psychological profile, but not to a majority, so application providers, including us, are left with just trying to get a maximum number of registrants so we can get a respectable number of active users. The research about teacher's performance is encouraging because they were surely a very diverse pool of people psychologically, yet this one basic "carrot" seemed to work incredibly well.

So how can those learnings be applied to Inkling and perhaps more broadly to application development in general?

Here are some ideas that have been rattling around:

  • Since everyone starts out with 5,000 inkles, we could introduce a "tax" that charges you on a monthly basis based on your usage of the software. If you've made X number of predictions, you get a "tax exemption" but if you don't, you start to lose your inkles and have less predictive power in the application.
  • We could generalize this behavior for whatever behavior we're trying to promote: logging in, making comments, sharing the site with a friend, etc.
  • Perhaps "tax" has too negative of a connotation. We could replicate what the University of Chicago did with the teachers and give people a bonus at the beginning of each month that they lose unless they make predictions, comments, etc.

We would have to be careful though that we're not incentivizing behavior we don't want, i.e. people just going in and making "garbage" predictions just to avoid paying the tax or losing some of their bonus. Which means a tax or bonus structure would have to be based on people's performance. But conveniently, performance can't be evaluated unless they exhibit the behaviors you want them to anyway.

More generally, perhaps the next generation of incentives in software applications will introduce their own currency specifically for this purpose. For example, applications are always bugging you to complete your profile or to do Facebook Connect and usually just show a status bar or reminder text that you "haven't completed 100% of your profile." What if instead when you signed up you were given 1,000 of fantasy money that you begin to lose if you don't do these things within a certain period of time. And if you do do them, the fantasy money can be cashed out for stuff: a waiver of AirBnB fees on a rental, an extra InMail in LinkedIn, an extra 1GB of space in Dropbox.

Continuing with the profile example, I'm sure these companies have quantified what it means to have someone have a complete profile in real dollar value because they can be more effectively marketed to. There would be a nice business case therefore to do this if it means 10% more profile completions assuming the economics work.

Companies like Kiip are kind of already doing this, but with earning "recognition" as you achieve things in applications. The fantasy currency enables the more effective reverse approach of front-loading the incentive and it's yours to lose. Currency is also much more flexible because you can begin to use it in other parts of your application - "earn X by doing Y."

Monday, July 30, 2012

Endorsement for the Founder Institute

A few months ago I was asked to be a mentor in the Founder Institute program here in Chicago. I was delighted to see a profile of the program in the New York Times recently as I've become a big fan since being involved with the program.

The Founder Institute, if you haven't heard of them, is a global program that helps entrepreneurs get their businesses off the ground by going through an intensive months-long training program. Entrepreneurs  work through their idea, do a lot of planning, incorporate, and get started on their business all while under the guidance of a large group of local mentors who are at the ready to provide help when called upon.

Unlike YCombinator which we went through, the aspect I like most about the program is it's friendly for people who already have jobs and are trying to start their businesses on the side. The reality is most people can't just take multiple months off to attend an incubator program, usually for financial reasons, but also because of family commitments. Founders Institute is like going to night school for entrepreneurs.

If you're thinking of starting a technology-related business, I'd urge you to check out the Founders Institute as a potential option. The price to join is fairly affordable and the network of peers and mentors you then have access to is extremely helpful. It also provides a badly needed framework of accountability for you - you can literally get kicked out of the program if you don't get your deliverables done or you score particularly low on their evaluations, and as far as I know, there are no refunds!

Tuesday, June 26, 2012

Inkling's 1%

How do you keep a prediction market interesting for the 1%? And more importantly, if there is truly a 1% in the distribution of wealth, how do you also keep a prediction market interesting for the 99%?

The Inkling public marketplace has been around for over 5 years now. Over that time, 38 people have amassed earnings over 1,000,000 inkles. Most of them are still playing to this day, but it's a challenge to keep them entertained. Many only play in questions which can earn them a significant amount. They submit questions whose starting prices are out of reach for anyone but the most wealthy because the larger trades keep it interesting. Meanwhile people new to the application know it's virtually impossible to crack this exclusive club without making a very significant time commitment and  also being good at predicting things.

I've been looking around and I've yet to find anything written about this specific problem in prediction markets, but there are certainly corollaries to regular gaming. What do you do with the people who can beat the game, or play for so long they're regularly reaching the top levels and may be more skilled than the game creator themselves? Usually that's when v2 of the game comes out with new challenges, better graphics, better teaming features, new weapons, etc. and an entirely new chance at making money from your users.

Unfortunately a complete conceptual revamp isn't really in the cards right now so we have to look for other solutions.

We've had suggestions from people to limit the size of trades that can be made, or simply zero everyone's account and start all over again. But this money has been hard earned and I've always been a big proponent of not "messing around" with people's balances. Thus other suggestions of real-world style approaches such as taxes and redistribution of wealth are also off the table.

I think ultimately the right answer will be to deemphasize money earned as the primary focus of the site and create "local" competition among people with comparable balances or other comparable demographics. The guy in position 18,345 shouldn't be worried about the guy in fifth, but he could be easily focused on moving up to 18,344. This is certainly a popular way of doing "best of " leader boards now.

I could also see moving completely away from "best of" leader boards and starting to create leader boards based on your demographic profile or other attributes. Here's how you're doing against other people in your hometown. Here's how you're doing in your age bracket. Here's how you're doing in a particular category of question. Here's how you're doing against people who started in the same day, month, or year you did. There still might be some whales in these leader boards you need to contend with, but the more leader boards we have, the easier it will be to find a niche for yourself and feel like you should continue to put time in to get to the top.

This post wasn't so bad was it? Find out about the next one by following us on Twitter.

Wednesday, May 30, 2012

Beta testers wanted for our new iOS app

We’re about a week away from distributing a private build of an Inkling iPhone app.

Functionality is pretty basic right now: simple trades, tracking your positions and performance numbers, listing questions and searching for them, and the leaderboard for highest worth portfolios.

We’re starting off catering to simple users with this app, then we’ll get to power users in future versions. The app will work for both our public site and any of our client sites.

Contact me directly: if you’d like to get a beta build and give us feedback. We have the right to say no if we think you're sketchy or one of our competitors. :)

Understanding the impacts of a risk occurring

Recently, we have had several clients inquire about using Inkling to understand not only the likelihood of a risk occurring but the potential impact of that risk.

Prediction markets handle this nicely because of the flexibility you have in asking the question and also the fact they output a quantitative value to your question.

For example, we can ask:

"If risk X occurs, what will happen?"
  • Impact A
  • Impact B
  • Impact C
In this instance, the question type in Inkling would allow for multiple possible right answers so each impact can be judged individually. If the risk does not occur (the impacts cannot be assessed,) each impact would be cashed out at 0.

We can also directly correlate the occurrence of a risk (or any other event) with impacts. To do so, we simply multiply the chances of the impact and the risk to understand the likelihood the impact will occur in association with that risk.

For example, we can ask: "Will risk X occur?" and let's say the current price is $75, representing a 75% chance the risk will occur. In a separate question, we can ask "Will impact X occur if risk X occurs?" and let's say the current price is $25, representing a 25% chance the impact will occur.

Multiplying (.25 * .75) the two, we get ~.19 or a 19% chance Impact X will occur.

Assessing the impacts of risks is important for decision makers to understand as it will directly influence what they may or may not want to do to mitigate a risk. For example if the likelihood of a risk is high but only non-harmful impacts have a high probability of occurring, it may not be necessary to mitigate the risk. But if the reverse is true, mitigating that risk may take on a higher priority.

Thursday, May 03, 2012

Congratulations to David Pennock, Prediction Market Extraordinaire

Just a brief note of congratulations to David Pennock who until very recently worked at Yahoo Research on all sorts of projects related to our field. He's now, along with what sounds like his entire group, part of Microsoft Research. David will be managing a new office for them in New York.

David's incredibly smart, a really nice guy, and has always been supportive of our work. We wish him the best.

Monday, April 23, 2012

The Psychology Behind Redesigns: Are You Just Insecure?

I'm in the process of working on a revised look for our application - probably the 5th or 6th time I've done this in the 6 years the application has been in existence.

As I got to yet another page that needed minor tweaks to fit in with the new look I was giving the application, I begun to wonder why I was even bothering to do this in the first place. No one has been complaining, people still applaud us for the "ease of use" of our application, but here I was burning 40-60 hours to work on this. Hours I could have been using to sell, to track our customers better, blog more, or any number of activities that may "move the needle" on our small business more than an application redesign.

But yet here I was obsessing over the border color around an icon.

My wife works at a design firm and she comes home with tales day after day about companies who are going through elaborate redesigns of their sites and paying hundreds of thousands of dollars in the process, not to mention the hundreds of hours their own employees are spending facilitating their work.

Thinking about this a bit more, there seem to be two prevailing reasons why people do redesigns. The first are simply table stakes. You need to have a decent looking site or app to be credible in the marketplace or people are actually going to discount you. Good design brings a certain amount of credibility.

But how about when you have that already? Why do people get so obsessed about redesigning their site when clearly it's going to contribute little or nothing to the bottom line?

I've come to call this the "Front Lawn" theory. There's a certain satisfaction, a certain security in knowing your front lawn is green and mowed. That the front of your house looks presentable to passer-bys. That your front lawn, as an extension of you and what you represent in the neighborhood, is tidy and well organized. I've visited plenty of homes where the outside is immaculate and the inside is a dump.

Ultimately what I think drives a lot of redesigns is basic insecurity, cost and effort be damned. Business sense is thrown out the window for a purely passion driven, irrational decision. When the moment comes for the executive to send the URL to a buddy, or present their company in some fashion, can they personally be proud about what is being shown? Is their front lawn immaculate?

Couch time, here I come. :)

Tuesday, April 03, 2012

Wisconsin Business School student researchers mentioned in slashdot for their prediction market

Nice job to the Unviersity of Wisconsin students who are doing some research on prediction markets in their business school. They made it to slashdot. Always a worthwhile accomplishment:

Monday, March 26, 2012

A Ratings Agency for Startups

One of the cooler efforts we've been associated with in awhile, Cdling (pronounced "seedling") is a ratings agency like S&P and FICO that measures risk for local and global investors, experts and startups.

Here's an explanation of what they do:

Low cost startups are unleashing millions of inexperienced investors and hundreds of incubators globally. Driven by an urgent search for economic growth, governments are creating angel investor incentives, lower barriers to foreign capital and reducing regulation. With legal crowd funding outlets in the US like Kickstarter becoming more commonplace, a lot of people will get hurt unless ratings like Cdling’s are established.

FICO scores, S&P rating reports and stock markets have long been used to measure and reduce risk and make commerce faster and smoother. Now Cdling is doing the same thing for the startup community.

Cdling rates people in terms of their real world experience at selecting which companies will succeed by analyzing who has invested with who and in what founding teams. Cdling also scores companies by predicting their likelihood to meet their milestones (this is where Inkling comes in.)

Investors can then use this insight to monitor more, smaller deals, catch the best breaking deals when they are ready to be financed, and as an early warning system to make sure they are spending their limited time and attention where it will earn the most return.

Cdling is currently closed to the general public, but you can get added to the pilot and track their progress by following this link. When you do get in, you'll have an extra CD$5,000 to spend on predictions.

Sunday, February 26, 2012

Inkling's Oscar Prediction Guide 2012

Someone didn't ask about every category for tonight's Academy Awards on our public site, but here's a prediction guide for a lot of the categories as of 4:15 ET/1:15 PT. The higher the chance, the more likely the participants thought that was going to be the winner.

CategoryPredicted WinnerInkling's Prediction
Original Screenplay Midnight in Paris 62% chance
Adapted Screenplay The Descendants 78% chance
Visual Effects Rise of the Planet of the Apes 69% chance
Sound Mixing Hugo 45% chance
Sound Editing Hugo 64% chance
Best Short Film (Live Action) The Shore 41% chance
Best Short Film (Animated) The Fantastic Flying Books of Mr. Morris 46% chance
Best Original Song Man or Muppet 68% chance
Best Original Score The Artist 83% chance
Makeup The Iron Lady 74% chance
Foreign Language Film A Separation 71% chance
Best Film Editing The Artist 48% chance
Best Documentary (Short Subject) Saving Face 39% chance
Best Documentary (Feature) Paradise Lost 3: Purgatory 33% chance
Best Directing The Artist 73% chance
Costume Design The Artist 44% chance
Art Cinematography The Tree of Life 66% chance
Art Direction Hugo 68% chance
Animated Feature Film Rango 88% chance
Best Supporting Actress Octavia Spencer 82% chance
Best Actress Viola Davis 76% chance
Best Supporting Actor Christopher Plummer 80% chance
Best Actor George Clooney 54% chance
Best Picture The Artist 80% chance

Wednesday, February 22, 2012

A dashboard two ways

One of the perpetual issues we face as a company that sells prediction markets is reconciling two primary kinds of users: the first are people who have never heard of a prediction market or aren't familiar with stock market concepts. This is the vast majority of our users. They are usually being asked by someone inside their company to participate in an internal prediction market and the experience is completely new to them.

The second type of users are familiar with stock market concepts, understand buying and selling shares, and have maybe even participated in a prediction market before like the HSX or Intrade or the Iowa Electronic Markets.

We've always worked hard to try and satisfy both camps in one user experience, usually erring on the side of simplification vs. complexity. But that's a fine line to walk and instead of really making both types of users completely satisfied, we've delivered more of an 80/20 experience.

Last year we began the split of the two user groups with two distinct trading interfaces for "simple" and "advanced" users. In a couple days, we'll be following up this split with the availability of two different dashboards to track predictions. The default will be a "simple" dashboard where you'll track your investments, but will never come in to contact with the concept of "buying shares."

Here's what an entry in the simplified dashboard will look like:

We wanted to make sure the only calculation the user would need to understand is, I spent X to make the crowd prediction Y. We also tell them what their maximum profit or maximum loss would be at any given time. And we let them see the "history" of predictions in any answer, describing in plain language exactly what they did at any given time:

In contrast, a person could choose to use the "advanced dashboard" which simply reveals more about their predictions. They can see how many shares they bought, the concept of short selling is not hidden but is instead made transparent, and there are more liquidation options.

We've also built a search function for your dashboard, so those who have more than one page of predictions can easily find them by typing in a word or two associated with the question instead of having to page through them or rely only on various sorting options. Someone told us recently they have over 40 pages of predictions on their dashboard. Search should be a huge timesaver.

I'm sure we still haven't gotten either experience perfect, but this should lay the foundation for better satisfying both levels of users.

Expect to see the new dashboard introduced some time Thursday or Friday evening.

Watch out for bullets

I called my cable company recently to see if they had any promotional plans I could switch to to relieve the exorbitant rate I was paying. They said no, but they did offer to swap out my old crappy DVR for a Tivo at no charge.

I drove to the RCN office in Chicago to do the swap. I walked in to find a sparsely furnished lobby and two customer support people sitting behind concrete walls and bullet proof glass.

I know they have a stock of relatively expensive equipment, but that can't be the only reason for the Western Union style lobby.

The experience got me thinking: how bad does your customer experience have to be before you need to start outfitting your support centers with reinforced steel doors and bullet proof glass? Shouldn't this be a signal to the company that something is seriously wrong with customer relations, or is this just what a semi-monopoly always looks like (everyone hates you?)

Sunday, January 29, 2012

Over-Optimism in Official Budget Agencies' Forecasts

We're always looking for where applications of prediction markets may make sense to improve a process. It looks like official government forecasting may be one such area to tackle.

Jeffrey Frankel from the Kennedy School of Government at Harvard and Director of the Program in International Finance and Macroeconomics at the National Bureau of Economic Research has written a paper entitled: "Over-Optimism in Forecasts by Official Budget Agencies and Its Implications."

From the digest on the NBER site:

...Overly optimistic official forecasts of future budget balances have facilitated complacency and so have contributed to tax cuts and increases in government spending, and therefore to realized budget deficits, during the last decade.

Analyzing data for 33 countries, Frankel finds that the average upward bias in the official forecast of the budget balance, relative to the realized balance, is 0.2 percent of GDP at the one-year horizon, 0.8 percent at the two-year horizon, and 1.5 percent at the three-year horizon. The longer the horizon, and the more genuine uncertainty there is, the more scope there is for wishful thinking.

A prediction market's biggest advantage would be to allow a more diverse opinion pool that would hopefully eliminate this bias. One could imagine not only involving officials at the OMB but various staff from Congressional offices, experts from various Departments, and even industry experts. If the government won't take this on directly, could there be a "shadow" forecasting process with these same people? What would be their incentive to participate? Hmm...