Software Facts - well, numbers at least

About a year ago I needed some numbers about software development - industry norms really: effectiveness, productivity, bug counts etc. etc. Its actually pretty hard to get these numbers and after hunting around I found myself with a copy of Capers Jones Applied Software Measurement.

Jones, for those who don’t know, has made a career out of analysing software and software teams numbers. Applied Software Measurement is his latest work on the subject - although its nearly three years out of date.

Unfortunately, for me, Jones didn’t have the numbers I wanted - I forget what I wanted now. But he does have lots of interesting facts and numbers. You can dispute some of these numbers, you can argue with him but on the whole he knows more about this subject than you or me so he’s probably right. That said, there are a few points where I disagree with him and I might blog about these in future.

For now, I’d like to share some of the number in Applied Software Measurement. Overwhelmingly Jones researched American companies and American teams. I expect most of these numbers will be broadly the same in Europe and elsewhere. All figures are for 2008 unless specified.

Productivity

• Broadly speaking, in the USA, software productivity rates were the same in 2008 as they were in 1988
• The best US companies in terms of productivity and quality are about three times better than average (US)
• The worst US companies for productivity and quality are about 50% worse than average
• Only about 15% of companies are improving productivity and quality
• Similarly, about 15% of companies are getting worse in these terms
• About 70% of companies are neither improving or getting worse
• Where productivity is improving it is usually teams developing web applications who use Agile methods. Some benefits also accrue from use of Team Software Process and Personal Software Process
• Claims by tool vendors of 10-to-1 or 20-to-1 displacement of human activity are generally not justified. It is counter productivity to invest in tools before resolving organisational and methodology issues. i.e. save the tools until you’ve improved things (John Seddon should be happy with this finding.)
• Office environments can have as much of an impact on productivity as tools and methods
• Looking after employees well increases productivity and reduces turn-over
• Companies providing 10 day or more of training per employee per year have higher productivity rates than similar companies who do not
• Accounting errors (and use of unpaid overtime) can hide the true cost of software production by 100%, i.e. the work costs twice as much as the accountants say
• Formal design reviews and code inspections are effective but can fall into disuse because (new) managers do not understand this. (Tom Gilb is right!)
• “Software does not age gracefully, and tends to become increasingly unstable and difficult to modify safely. The Unites States and indeed much of the world are facing some huge geriatric problems with their inventories of ageing and decaying applications” - maintenance, renovation and enhancement are now the dominant activities in the industry and it is likely to stay that way
• On average military software is 25 times larger than end-user software, and the specifications and other documentation 200 times larger.
• Productivity and quality seem to be better in object oriented languages

Documentation & Bugs

• Producing paper documents for software development is more expensive than producing software itself
• Up to 400 words may be written in specification for every line of code in large systems.
• In a large software project requirements change and increase at the rate of 2% per month
• Repairing defects in software is more expensive than coding and producing documentation
• The finding and fixing software defects (bugs) is historically the most expensive activity and will account for 50% of total costs over the product life time
• Approximately 7% of changes and fixes contain new defects
• Most forms of testing finding fewer than 30% of all bugs

Projects

• Over 50% of all software projects are one-man projects
• Productivity and quality are directly coupled: projects with high quality have high productivity and vice versa
• IBM was the first company to discover the link between quality and productivity. They made the information public in 1975 but it is still not widely know.
• User satisfaction and defect counts are also directly correlated

I’ve mined these numbers out of Applied Software Measurement, I might mine some more and blog again. But really, it isn’t a book of numbers. Its a discussion of how to measure software and software teams.

Finally for now, I’m only at chapter 3, Jones says that currently available data on software production is less than 1% of what is needed. He also says that most databases of such data are privately held and not accessible by most of use. Well, I guess that explains why I couldn’t find the numbers I wanted when I wanted them.