Wednesday, September 30, 2009
Tuesday, September 29, 2009
Building something people want
Here's the latest article from Joel Spolsky in Inc. magazine.
Probably worth a read for anyone involved in product development.
We had the pleasure of meeting Joel when we first started Inkling 4 years ago in Silicon Valley. Joel spoke to the group of Y Combinator startups at our weekly meeting.
The biggest piece of advice he gave is in this article.
Figure out the root problems that matter.
Too often we tend to think we have this problem and that problem and this problem.
For Joel, and very likely many companies building new products, the most significant problem to work on is building a useful product. It's not worrying about people stealing your ideas or building some difficult to manage affiliate network. It's not the perfect Google Adwords campaign, or the perfect font size.
It's building something people want. And then doing that some more.
(Borrowed from a Y Combinator motto.)
Monday, September 28, 2009
Decision Jujitsu
"Making yourself open to correction and persuasion is a pillar in the strategy of principled negotiation."
-Roger Fisher and William Ury (Getting to Yes)
Decision making often seems to be yet another negotiation we have.
Again and again that negotiation is with ourselves.
Saturday, September 26, 2009
Trading widget mimics the ease of polling online
Snap polls are ubiquitous across practically every content site on the web. "Should Apple launch a tablet PC?" "Who will win the football game?" "Should Kanye be banned from all future awards shows?"
People love these polls because they're quick, easy, and provide instant gratification in feedback to see what other people think.
Prediction market trading has a lot of these same attributes, and in many cases provides much more valuable data than a simple poll, but the act of "trading," whether as a trade or a bet or some other transactional mechanism is also a more complex concept for people to grasp.
In our continuing efforts to 1) keep things simple, and 2) bring trading to the casual/fickle user (we're talking about you, executives,) we've created a new trading widget that behaves exactly like a poll, but in the background, a prediction market trade is being made.
Here's how it works:
Once you've logged in to an Inkling public marketplace (or are signed in as an administrator of a private marketplace) you'll see a link called "Trade from your site:"
When you click on that you'll see a sample of the widget you can put on your own site:
Looks a lot like a plain ol' poll, right? That's the point! But there's a slight twist. When you click on an answer, you have to take one more step - you have to quantify your opinion and this is where the behind the scenes trading comes in. Just like trading in Inkling where we've equated English words with shares being bought or sold, we're doing that here too but not even telling you about it:
Once you select how strongly you feel about your answer, you get to the gratification part and see what everyone else is thinking. If you're logged in to the Inkling site the widget originated from, the trade will be recorded on your dashboard just like any other trade. If you're not logged in or not a member of the marketplace, you will be given an opportunity to login or "claim" your opinion by registering.
We've purposely made the widget "plain" and given you the ability to customize the look and feel by importing your own .css stylesheet to make it look exactly like your own site:
We're really excited about this new widget and are already working with our partners in market research to think about how this could replace certain surveys and polls they run.
We're also working with our existing clients to help them think about how they can put this widget on their internal Sharepoint or Intranet sites to draw more attention to the questions they're asking. It could even be a way for clients to expose questions they're asking externally without having to engage someone in an entire prediction market experience.
People love these polls because they're quick, easy, and provide instant gratification in feedback to see what other people think.
Prediction market trading has a lot of these same attributes, and in many cases provides much more valuable data than a simple poll, but the act of "trading," whether as a trade or a bet or some other transactional mechanism is also a more complex concept for people to grasp.
In our continuing efforts to 1) keep things simple, and 2) bring trading to the casual/fickle user (we're talking about you, executives,) we've created a new trading widget that behaves exactly like a poll, but in the background, a prediction market trade is being made.
Here's how it works:
Once you've logged in to an Inkling public marketplace (or are signed in as an administrator of a private marketplace) you'll see a link called "Trade from your site:"
When you click on that you'll see a sample of the widget you can put on your own site:
Looks a lot like a plain ol' poll, right? That's the point! But there's a slight twist. When you click on an answer, you have to take one more step - you have to quantify your opinion and this is where the behind the scenes trading comes in. Just like trading in Inkling where we've equated English words with shares being bought or sold, we're doing that here too but not even telling you about it:
Once you select how strongly you feel about your answer, you get to the gratification part and see what everyone else is thinking. If you're logged in to the Inkling site the widget originated from, the trade will be recorded on your dashboard just like any other trade. If you're not logged in or not a member of the marketplace, you will be given an opportunity to login or "claim" your opinion by registering.
We've purposely made the widget "plain" and given you the ability to customize the look and feel by importing your own .css stylesheet to make it look exactly like your own site:
We're really excited about this new widget and are already working with our partners in market research to think about how this could replace certain surveys and polls they run.
We're also working with our existing clients to help them think about how they can put this widget on their internal Sharepoint or Intranet sites to draw more attention to the questions they're asking. It could even be a way for clients to expose questions they're asking externally without having to engage someone in an entire prediction market experience.
Friday, September 25, 2009
Ben Franklin on making decisions
Breaking Bad just won an Emmy.
Basically, it's about a high school teacher who makes odd decisions.
Most significantly he decides to start making crystal methamphetamine to make lots of money before he dies of cancer. Not a spoiler. You learn this real quick if you get started with the show.
In the first season, he's also up against a decision to kill a man.
He uses a list of pros and cons to decide.
I'm not going to share anymore of the story since that's best left up to watching this terrific show :)
But I was curious where the pros and cons method came from.
I think I may have learned it originally from my mother or maybe in grade school.
Regardless if you do a tiny bit of research, Ben Franklin comes up a lot in discussions of this method.
It's just that his method is a little more systematic than how we usually go about it. And maybe more useful.
The method he uses is mentioned in a letter you can find all over the internet to a Joseph Priestly. Here's the text of the letter I found here.
-----------------
To Joseph Priestley
London, September 19, 1772
Dear Sir,
In the Affair of so much Importance to you, wherein you ask my Advice, I cannot for want of sufficient Premises, advise you what to determine, but if you please I will tell you how.
When these difficult Cases occur, they are difficult chiefly because while we have them under Consideration all the Reasons pro and con are not present to the Mind at the same time; but sometimes one Set present themselves, and at other times another, the first being out of Sight. Hence the various Purposes or Inclinations that alternately prevail, and the Uncertainty that perplexes us.
To get over this, my Way is, to divide half a Sheet of Paper by a Line into two Columns, writing over the one Pro, and over the other Con. Then during three or four Days Consideration I put down under the different Heads short Hints of the different Motives that at different Times occur to me for or against the Measure. When I have thus got them all together in one View, I endeavour to estimate their respective Weights; and where I find two, one on each side, that seem equal, I strike them both out: If I find a Reason pro equal to some two Reasons con, I strike out the three. If I judge some two Reasons con equal to some three Reasons pro, I strike out the five; and thus proceeding I find at length where the Ballance lies; and if after a Day or two of farther Consideration nothing new that is of Importance occurs on either side, I come to a Determination accordingly.
And tho' the Weight of Reasons cannot be taken with the Precision of Algebraic Quantities, yet when each is thus considered separately and comparatively, and the whole lies before me, I think I can judge better, and am less likely to take a rash Step; and in fact I have found great Advantage from this kind of Equation, in what may be called Moral or Prudential Algebra.
Wishing sincerely that you may determine for the best, I am ever, my dear Friend,
Yours most affectionately
B. Franklin
Source: Mr. Franklin: A Selection from His Personal Letters. Contributors: Whitfield J. Bell Jr., editor, Franklin, author, Leonard W. Labaree, editor. Publisher: Yale University Press: New Haven, CT 1956.
-----------------
So his method of pros and cons brings up points that are a bit different then we typically might use:
1) Spend a few days on creating the list. Don't try and bang it out in one sitting. Sleep on it. This might be the best of both worlds for people who've seen the new research that sleeping on a problem to let it work out unconsciously might not work.
So this is a mix here of conscious analysis AND sleeping on it.
2) It's not just about which side has more pros and cons. Each gets a weight to consider.
One additional thing I've seen introduced to Ben's methodical pros and cons that he doesn't touch on in the letter is how to determine weight.
Typical of most businesses risk or issue logs, you would determine weight by multiplying an issue's probability of happening and its impact.
If you have 2 issues (A and B), sure A might be of more serious impact than the other, but if it's of a very slight probability of occurring, issue B might be of more importance in prioritizing (i.e., might have a higher weight)
In fact, here's a blank sheet using this approach.
From a guy named Fred Nikols at Distance Consulting.
Basically, it's about a high school teacher who makes odd decisions.
Most significantly he decides to start making crystal methamphetamine to make lots of money before he dies of cancer. Not a spoiler. You learn this real quick if you get started with the show.
In the first season, he's also up against a decision to kill a man.
He uses a list of pros and cons to decide.
I'm not going to share anymore of the story since that's best left up to watching this terrific show :)
But I was curious where the pros and cons method came from.
I think I may have learned it originally from my mother or maybe in grade school.
Regardless if you do a tiny bit of research, Ben Franklin comes up a lot in discussions of this method.
It's just that his method is a little more systematic than how we usually go about it. And maybe more useful.
The method he uses is mentioned in a letter you can find all over the internet to a Joseph Priestly. Here's the text of the letter I found here.
-----------------
To Joseph Priestley
London, September 19, 1772
Dear Sir,
In the Affair of so much Importance to you, wherein you ask my Advice, I cannot for want of sufficient Premises, advise you what to determine, but if you please I will tell you how.
When these difficult Cases occur, they are difficult chiefly because while we have them under Consideration all the Reasons pro and con are not present to the Mind at the same time; but sometimes one Set present themselves, and at other times another, the first being out of Sight. Hence the various Purposes or Inclinations that alternately prevail, and the Uncertainty that perplexes us.
To get over this, my Way is, to divide half a Sheet of Paper by a Line into two Columns, writing over the one Pro, and over the other Con. Then during three or four Days Consideration I put down under the different Heads short Hints of the different Motives that at different Times occur to me for or against the Measure. When I have thus got them all together in one View, I endeavour to estimate their respective Weights; and where I find two, one on each side, that seem equal, I strike them both out: If I find a Reason pro equal to some two Reasons con, I strike out the three. If I judge some two Reasons con equal to some three Reasons pro, I strike out the five; and thus proceeding I find at length where the Ballance lies; and if after a Day or two of farther Consideration nothing new that is of Importance occurs on either side, I come to a Determination accordingly.
And tho' the Weight of Reasons cannot be taken with the Precision of Algebraic Quantities, yet when each is thus considered separately and comparatively, and the whole lies before me, I think I can judge better, and am less likely to take a rash Step; and in fact I have found great Advantage from this kind of Equation, in what may be called Moral or Prudential Algebra.
Wishing sincerely that you may determine for the best, I am ever, my dear Friend,
Yours most affectionately
B. Franklin
Source: Mr. Franklin: A Selection from His Personal Letters. Contributors: Whitfield J. Bell Jr., editor, Franklin, author, Leonard W. Labaree, editor. Publisher: Yale University Press: New Haven, CT 1956.
-----------------
So his method of pros and cons brings up points that are a bit different then we typically might use:
1) Spend a few days on creating the list. Don't try and bang it out in one sitting. Sleep on it. This might be the best of both worlds for people who've seen the new research that sleeping on a problem to let it work out unconsciously might not work.
So this is a mix here of conscious analysis AND sleeping on it.
2) It's not just about which side has more pros and cons. Each gets a weight to consider.
One additional thing I've seen introduced to Ben's methodical pros and cons that he doesn't touch on in the letter is how to determine weight.
Typical of most businesses risk or issue logs, you would determine weight by multiplying an issue's probability of happening and its impact.
If you have 2 issues (A and B), sure A might be of more serious impact than the other, but if it's of a very slight probability of occurring, issue B might be of more importance in prioritizing (i.e., might have a higher weight)
In fact, here's a blank sheet using this approach.
From a guy named Fred Nikols at Distance Consulting.
Thursday, September 24, 2009
New Feature: Managing categories is now even easier
Before, if you wanted to manage a set of categories for your prediction markets you had to create them ahead of time from your marketplace settings.
Now, if your an admin of a marketplace, you can manage your categories while you are creating a market. So you don't have to miss a beat.
In you market's "Details" tab, just look for the "Manage categories" link (it's labeled "Add a category" if you don't have any yet).
---------------------------
---------------------------
Click on it and you'll have a simple pop-up to add or delete your categories. Here's what it looks like on our public site.
---------------------------
---------------------------
We hope this small added feature comes in handy.
Thursday, September 10, 2009
Save money on split testing using Amazon’s Mechanical Turk
By now, most of us have heard that we should be testing. A/B testing. Split testing. Multivariate testing.
You should test your ads, test new features, test minimum viable products, test emails, and on and on.
And we totally agree.
But there are problems with it. One big problem is cost.
For example, it quickly gets very costly to split test Google ads if each click is $3.
Or what if you want to take an ad out on a premium ad network like Coudal's The Deck or Fusion Ads.
A Deck ad is $7200.
So this is something that for many small companies isn't something you can whiff at, and test a new ad on the second month to see if it does better.
You'd like to figure out as cheaply as possible and as soon as possible how to put your best foot forward.
I'm not saying we have a solution at all to this.
But we did something we find interesting that maybe we or someone else will find useful down the road.
How successful is testing things like ads, headlines, concepts, etc. with Amazon's Mechanical Turk?
Especially since from what I read, getting someone to do a simple task on the Mechanical Turk is cheap. $0.01 a task for example.
We setup an experiment.
Can the folks on the Turk pick the best converting headline if given a selection of headlines?
(Primarily though, this whole experiment was so that we could learn more about the Turk. So to preempt any, "you should have done it this way" or "why are you using the turk + wufoo" for this, we'd love to hear any input, but this was first and foremost a reason to understand how to use the Turk to do anything. Anything else is just "interesting", probably pre-useful, and we wanted to share it.)
I used a selection of headlines I already knew the results for with a Wufoo form. This was a test 37signals ran for their headline.
(This test isn't approved, endorsed or affiliated with 37signals in anyway.)
For background about their test, they saw statistically significant results showing them that 2 of the headlines were better than the rest. Headline A and Headline B converted (30% and 27% respectively) better than the original headline. As confirmed by Jason Fried, statistically Headline A and Headline B improvements are the same at this point. But they liked one of them better than the other, so they claimed Headline A the winner. Statistically however, Headline B could have been chosen as a winner as well.
We are in the business of crowd wisdom, so I'm already hopeful this experiment would show some wisdom from workers on the Turk. But given the incentive of say $0.01 to just complete the survey no matter what, I figured everyone would just click on something randomly to complete the task without thinking. And not a single ad would provide a strong signal.
If that didn't happen, I figured maybe some of the respondents will have heard of 37signals or knew the answer to this post and they'll pick the headline 37signals claimed as the winner.
Here are the results. Which didn't agree with what I thought I'd see.
----
Headline B (named in the 37signals background above) was the winner (38.89% +- 7.1%).
The next best headline was chosen by 16.67% with a confidence interval of +- 5.4%.
So Headline B was chosen with significance from workers on the Turk to do better. And obviously from a test in the real world from 37signals, they saw great results from that headline.
Surprisingly Headline A though, which performed very well in real life, was just in the middle of the pack.
More experiments need to be run to prove any kind of usefulness here. One reason for the behavior seen could be that Wufoo's randomization of the survey choices is biased. In my opinion, this isn't likely, but we plan on doing more tests to see how they play.
I think though that this provides some encouragment that the Turk could prove itself as a crowd that can be used and tuned to make good predictions when you need one.
There's definitely some more experiments we're doing with the crowd at Amazon's Mechanical Turk coming soon...
You should test your ads, test new features, test minimum viable products, test emails, and on and on.
And we totally agree.
But there are problems with it. One big problem is cost.
For example, it quickly gets very costly to split test Google ads if each click is $3.
Or what if you want to take an ad out on a premium ad network like Coudal's The Deck or Fusion Ads.
A Deck ad is $7200.
So this is something that for many small companies isn't something you can whiff at, and test a new ad on the second month to see if it does better.
You'd like to figure out as cheaply as possible and as soon as possible how to put your best foot forward.
I'm not saying we have a solution at all to this.
But we did something we find interesting that maybe we or someone else will find useful down the road.
How successful is testing things like ads, headlines, concepts, etc. with Amazon's Mechanical Turk?
Especially since from what I read, getting someone to do a simple task on the Mechanical Turk is cheap. $0.01 a task for example.
We setup an experiment.
Can the folks on the Turk pick the best converting headline if given a selection of headlines?
(Primarily though, this whole experiment was so that we could learn more about the Turk. So to preempt any, "you should have done it this way" or "why are you using the turk + wufoo" for this, we'd love to hear any input, but this was first and foremost a reason to understand how to use the Turk to do anything. Anything else is just "interesting", probably pre-useful, and we wanted to share it.)
I used a selection of headlines I already knew the results for with a Wufoo form. This was a test 37signals ran for their headline.
(This test isn't approved, endorsed or affiliated with 37signals in anyway.)
For background about their test, they saw statistically significant results showing them that 2 of the headlines were better than the rest. Headline A and Headline B converted (30% and 27% respectively) better than the original headline. As confirmed by Jason Fried, statistically Headline A and Headline B improvements are the same at this point. But they liked one of them better than the other, so they claimed Headline A the winner. Statistically however, Headline B could have been chosen as a winner as well.
We are in the business of crowd wisdom, so I'm already hopeful this experiment would show some wisdom from workers on the Turk. But given the incentive of say $0.01 to just complete the survey no matter what, I figured everyone would just click on something randomly to complete the task without thinking. And not a single ad would provide a strong signal.
If that didn't happen, I figured maybe some of the respondents will have heard of 37signals or knew the answer to this post and they'll pick the headline 37signals claimed as the winner.
Here are the results. Which didn't agree with what I thought I'd see.
----
----
Headline B (named in the 37signals background above) was the winner (38.89% +- 7.1%).
The next best headline was chosen by 16.67% with a confidence interval of +- 5.4%.
So Headline B was chosen with significance from workers on the Turk to do better. And obviously from a test in the real world from 37signals, they saw great results from that headline.
Surprisingly Headline A though, which performed very well in real life, was just in the middle of the pack.
More experiments need to be run to prove any kind of usefulness here. One reason for the behavior seen could be that Wufoo's randomization of the survey choices is biased. In my opinion, this isn't likely, but we plan on doing more tests to see how they play.
I think though that this provides some encouragment that the Turk could prove itself as a crowd that can be used and tuned to make good predictions when you need one.
There's definitely some more experiments we're doing with the crowd at Amazon's Mechanical Turk coming soon...
Wednesday, September 09, 2009
"On Prototyping: The Simplest Solution Never Comes First"
"But the long journey from visionary idea to intuitive product is a trajectory worth contemplating. In the case of Herman Miller’s new Setu chair, designed by Berlin outfit Studio 7.5, the numbers are telling: 18 months of self-financing, 5+ years of development, and 40 fully functional prototypes."
Tuesday, September 08, 2009
Vitaminwater is crowdsourcing a new flavor using market research from a game
"Vitaminwater, the beverage brand that has become nationally-known this decade through clever marketing efforts, is making another innovative move today — on Facebook. It has launched its own application, called “flavorcreator,” that appears within its Page and intends to get users creating their own virtual vitaminwater beverages."
Friday, September 04, 2009
Flickr stream from Tap the Collective DC event
We just got the photos back from the photographer we hired to shoot our event in Washington. I've uploaded them all to Flickr. Video of the entire event to follow early next week! Have a great Labor Day weekend.
Inkling's Flickr Stream
Inkling's Flickr Stream
Thursday, September 03, 2009
First Tap the Collective in DC a Success
I'll give a more detailed recap in the next couple days, but last night we co-hosted an event in Washington, DC last night called "Tap the Collective" where we brought together the "do'ers" behind some of the more interesting collective intelligence projects going on in Washington. Attendance was fantastic, we got about 20 questions for the q&a, and our speakers, with a 7 minute time limit, each did a great job succinctly explaining their initiatives and some of the cultural battles they're waging internally to increase the likelihood of success.
My favorite quote from the night:
"No one ever retires from NASA; they just die."
- Emma Antunes from NASA explaining what a great place to work it is.
More soon, including video and photos of the evening.
My favorite quote from the night:
"No one ever retires from NASA; they just die."
- Emma Antunes from NASA explaining what a great place to work it is.
More soon, including video and photos of the evening.
Subscribe to:
Posts (Atom)