Software quality?

Since publishing my stream of consciousness about software quality on Friday (What is a software quality?) I’ve continued to think about the subject. In part putting the thoughts down started to structure my own thinking, in part a few comments made on Twitter caused me to think more, and, simply having written something caused me to reflect.

There are four main points I’d add to Friday’s entry.

First, what I wanted, what I needed for Xanpan, was (is) a good enough working definition of software quality, I don’t need a perfect definition. But even this turns out to be surprisingly difficult. And as Steve Smith pointed out this is not confined to software quality, defining “quality” at all tends to lead you in the direction of Zen and the Art of Motorcycle Maintenance.

Next, what I really need are two things: A definition of software quality and a definition of a software defect. I think I’m starting to approach workable definitions of both. But, definitions (by definition almost) should be free of assumptions about how you achieve - or don’t achieve - that aim. In other words, the definitions should not imply or assume an Agile, Waterfall or any other approach.

Third, one of the things I was concerned about was gold plating or over engineering. Really this is part of the “how do you achieve your aim.” A definition of software quality - and low defects - need not rule out (or embrace) any particular approach. I need to separate this out. Similarly the knowledge issue, while important, needs to be put on the side here.

Finally, I’m coming to the conclusion that we need two different terms for software defects. The clue was in that Tom Gilb quote really.

• An Absolute Defect is, as Gilb says, free from personal opinion or taste. It is objective. For example, function called “Add Two Numbers” always returns 5 - whether you give it 2 and 2, 2 and 3, or 100 and 200.
• A Common Defect (and I’m prepared to find a better name and I don’t imply any link to Deming) is anything which someone at some time decided to report - or classify - as a defect.

Absolute defects are a perfect subset of common defects, i.e. all absolute defects are also common defects but only some common defects are absolute defects.

I think its important to make this distinction because so much time and energy is spent by people arguing over what is and what is not a “defect”. If we define separate terms then we can at least reason about this situation and the fact that individual, teams and organizations often have incentives for classifying something as a defect whether it is or is not.

Jon Jagger pointed out in his comment on the previous blog that “Jerry Weinberg's definition of quality is "value to someone" or "value to some persons”.” A common defect most definitely fits that description. Someone, somewhere, considers it valuable to call this thing a defect.

But I’m not sure I go along with that statement as a definition of quality as a whole. It is too open and not specific enough for me - or perhaps for the uses I want to put it to!

Some observations about defects to end:

• All defects cost time and probably money: even logging a report of a potential defect costs time (and thus money) and action taken as a result (investigation and potential fixing) costs time. Defects themselves - even when not reported - may well costs money, e.g. overpayment or lost customers. (This also implies an element of the unexpected to defects.)
• Absolute defects cost time and money; is the same true for common defects? Well maybe my opinion of something does result in a different financial outcome than someone else’s but does that make it a defect? Even then it can be surprisingly hard for some organizations to tell what makes, and what looses, money. I’d like this to be the dividing line but I don’t think it is.
• Common defects need to be administered, managed and possibly fixed just the same as absolute defects. In my opinion, if someone reports a defect it should be considered against other defects - common or not - and action (fixing) taken or not. The debate between whether a particular defect is an absolute defect or a common defect isn’t very useful, it is the software equivalent of “my Dad is bigger than your Dad.”
• If there exists some pre-code definition (specification, requirement, formula, etc.) of what the code should do it might be easier to classify something as an absolute defect but this isn’t enough reason to create that definition.
• High quality software has few defects of either sort and is perceived as having few defects.
• Software which has a lot of reported defects doesn’t quality as high quality.