This weekend I got an email from X (Chris Verhoef). He noticed I used the research of the Chaos Report and his research in a blog post. Chris and J. Laurenz Eveleens wrote an article in IEEE Software (January and February issue) on the Chaos Report. They stated; Although the Standish Group‚The Chaos reports are often used to indicate problems in application software development project management, the reports contain major flaws. They concluded:
- Misleading definitions: They‚use misleading because they are onlybased on estimation accuracy of cost, time, and functionality. But Standish labels projects as successful or challenged, suggesting much more than deviations from their original estimates.
- Unrealistic Rates: The next issue is whether the Standish estimation accuracy definitions are sound. They are not. The Standish Group‚use measures are one-sided because they neglect underruns for cost and time and overruns for the amount of functionality.
- Perverting Accuracy: The third problem is that steering on the Standish definitions causes large cost and time overestimations (and large functionality underestimations), which perverts rather than improves estimation accuracy.