Recently I have been re-reading one of my favorite software engineering books, Peopleware, by DeMarco and Lister. They set the stage for the rest of the book with an alarming statistic – 15% of software projects, and they have tracked hundreds since the late 1970’s, never see the light of day. The numbers are even worse for the largest projects. For those over 25 work-years, nearly 25% failed. No one publishes the costs of these failures, let alone all of the projects that finish but take longer and cost more than originally planned. The numbers would be staggering.
What I found interesting was the author's explanation for these failures. Where they could find people to talk about these failed projects, the overwhelming cause was politics. DeMarco's and Lister's interpretation of this was that it was not necessarily office politics in the traditional sense, although there was some of that, but what they called sociology - the management of people and groups. Understanding people and their motivations and finding ways to get people and teams to communicate and work together better is hard. Managers are not good at this generally, and software engineers in particular are pretty bad. They typically start their careers as programmers and, let's face it, technical people, myself included, are not very good at figuring out people. It is much easier to figure out why the router is not talking to the switch than why John is not teaming well with Carol.
So what can we do to get better? Lister and DeMarco have a number of excellent suggestions. I heartily recommend Peopleware and their follow-up publications. What struck me though reading the book this time was that politics , or sociology using their terms, was exactly the problem we set out to solve in building Inkling. We had seen over and over in our previous work experiences where communication and information flow on large projects was broken. No one wanted to stand up and tell the boss the project was a bad idea. The true risks and status of a project got watered down as the information made its way up through layers of management. Politics drove decisions, not data. It was so dis-heartening we knew there had to be a better way.
Which is how we got to prediction markets. We thought the ability of the technology to harness the wisdom of the crowd was a great way to free up critical information within organizations. And this is exactly what our customers have found. They also found an additional benefit we did not anticipate. Customers such as Ford tell us the market gets the crowd talking, and that the collaboration the platform fosters across organizational silos is as valuable as the ability to quantify business questions. We have written about this describing how Inkling has been used to address Project Risk Management. The concepts covered there are just as applicable to project management in general, whether it is software or any large, people intensive project.
Of course prediction markets, like any technology, are not a silver bullet. Ultimately it will take individual and organizational commitments to focus on and get better at people management. Prediction markets can be a catalyst for that organizational change: when information is more transparent and people are allowed to communicate without fear of repercussion, that changes the dynamic of how projects and organizations function.
Let us know what you think. Is it politics, or sociology, that bring down large software projects? Or are there other factors that DeMarco and Lister missed?
