Study on benefits of TDD

OK, this isn’t news, this study came out a couple of years ago and was covered by many people then. But, I find myself regularly referring to it trying to find the link. So I’m going to blog about it then I’ll always be able to find the link.

The study is by Nagappan, Maximilien, Bhat and Williams and is entitled: Realizing quality improvement through test driven

development: results and experiences of four industrial

teams and is freely downloadable from Microsoft.

This is the key findings are summarised in this table:

Team / Metric	IBM drivers	Microsoft Windows	MS MSN	MS Visual Studio
Defect density of comparable team not using TDD	W	X	Y	Z
Defect density of team using TDD	0.61W	0.38X	0.24Y	0.09Z
Increase in time taken because of TDD	15-20%	25-25%	15%	25-20%

The way to read this is: the researches looked at two Microsoft MSN teams, one team did not use TDD and had a defect density of Y. The second MSN team had a defect density less than a quarter of Y but took 15% longer.

To my mind that proves that which was to be proven, i.e. TDD reduced bugs. But, I’m also aware that other writers have disputed this and I’ve heard of studies which disprove it. (Anyone got a link? Thanks)

Most people who I’ve met, and who have practices, or understand TDD agree it is effective. However there are those who don’t believe it. It reminds me of the episode of Black Adder where Rowan Atkinson’s Black Adder hires a ship Captained by Tom Baker. When there is a problem it plays out like this:

Black Adder: “Someone in the crew will know... you do have a crew don’t you?”

Captain: “Arh, opinion is divided on the subject... all the other captains say you need a crew, and I say You Don’t”

At the end of the day Confirmation Bias will probably decide which set of results you choose to believe.

Ed Yourdon on Agile

Ed Yourdon seems to have fallen off my radar so far this century. Last century I read a lot of his stuff and came to respect him as a man who knows what he’s talking about when it comes to IT and software development.

(If you haven’t heard of him, or don’t believe me just look at the list of books he’s written.)

I recently discovered that he has a blog, and from there that he has recently been to the Agile 2010 conference. In fact it appears that the whole Agile thing has, to a large part, passed him by.

The really interesting thing are some of his comments on Agile from this blog posting:

• ‘My overall impression is that “agile” (and its variations, such as scrum and XP) are now entering the “mainstream” of computer systems development’
• ‘there’s still a lot of hype and exaggeration, along with a non-trivial amount of myth and folklore and general silliness. But when you strip away all of this, there is a very solid, and extremely well-documented, core of practical system development guidelines, concepts, strategies, and hard-won lessons that every practitioner in our field needs to know about.’

I’m happy to find myself agreeing with Ed, and even happier that the voice more experience sees it the same way I do!

Objective Agility - what does it take to be an Agile company?

Modern Analyst has published my latest piece about Agile at the company level: Objective Agility - what does it take to be an Agile company?

This is actually a bit of a taster for a presentation of with the same title I’ll be doing at the IIBA Business Analysis conference in a few weeks.

And talking of businesses analysis... I am running an ‘Essential Agile for Business Analysts’ at Skills Matter in a few weeks. Many of these themes come up in that course. I believe there is still space available.

And if you miss all that, I’ll be sticking with this theme for the Agile Cambridge conference in October.

The No Business Case Myth

Once in a while I run across individuals, or even teams, who still think Agile is about just getting on and doing it. Well it is, good for them, but, that doesn’t mean there isn’t a reason for doing it.

There seems to be a myth in some circles that work done using Agile techniques doesn’t require a business case. Lets get this clear: Agile does not excuse you from having a business case for your work.

Of course there are instances were a business case might not exist. Some companies, for better or worse, work without them; those companies aiming at innovation may allow work to proceed to a more advanced stage before asking for some rationale; and products which are in a steady state may just tick over without too much attention to a business case.

But in each one of these cases the lack of a business case has nothing to do with Agile. (Actually, each of the cases does have a business case, its either a tacit business case or a part of a much bigger business case.)

Many development efforts which lack a visible business case can benefit from demanding a business case. If you are not sure why your company is doing some piece of work then maybe the company needs to be clearer about the reasoning.

However, a business case might look a little different on a Agile project. It may well be shorter than a traditional business case, it may lack some of the detail traditionally found in a business case, and it may describe much more a trial-and-error approach.

What Agile projects don’t need are in-depth business plans. Here its a question of detail, a strategic business plan makes plenty of sense. But a business plan which lists many many features to be build and a detailed schedule of when they will be built is little more than an illusion. Such plans give the appearance of certainty but certainly don’t provide it. Indeed, too many plans can actively hinder a project - plans are not benign, they are dangerous. (See An alternative view of design (and planning) for more discussion here.)

That said, there is a very good case for writing a business plan - indeed most forms of plans actually: they are rehearsal time for the real thing. They help you explore and learn about the thing you want to build.

Still, sometime you might skip the business plan. Maybe there isn’t time for a rehearsal, and sometimes learning can be maximised but just getting stuck in .

But, if you do write one, then once that exercise is over, once you start doing it for real, don’t try and hold yourself to a plan. Put the plan in the draw and don’t look at it. The value of planning is not the plan you produce at the end of planning; the value it is the knowledge you acquire in your head.

For that reason you want to maximise the number of people involved in the planning rehearsal so you can maximise the learning.

Off topic moan - Price Promise that isn't

Although I try to keep this blog within its very loose boundary of software and business there are times when you want to say something that doesn’t fit.

I could try and justify the following story on the basis that it concerns my upcoming trip to Agile Eastern Europe. Or I could justify it as an example of poor system thinking, but I think it is really a moan.

It also shows how I was fooled. I’m not as bright as I like to think I am, for once I believed what the corporation told me despite my natural cynicism.

It all began when I set out to book my flights to Agile Eastern Europe in October. As usual I did a quick check on Opodo to see what the market fare was and who had flight. Turns out British Airways had a direct flight, was fairly competitive and since they are my usual airline I decided to book with them.

As usual I then went to the BA website to see if it was any cheaper. It wasn’t indeed it was more expensive, but since I was now logged into the BA system and all my details were on file I decided to finish the booking there.

However, BA was more expensive. That’s OK I thought, BA make a big thing of their “Price Promise.” I’ll just keep a record of what Opodo offer and they’ll give me the money back, about £25.

Wrong.

British Airways won’t pay up. Despite sending them the PDF I took of the Opodo offer they say its a different itineracy. Erh? Same flight out, same slight back. But it seems, Opodo was offering me two single tickets and BA were offering a return ticket.

Wonderful. British Airways are hiding behind word play. I want a flight to Kiev and a flight back. Do I care about the niceties of ticketing? No. The difference is lost on me.

So there you have it. BA have made an extra £25 out of me. Clever, pat yourself on the back guys.

Except, I’ll never book with BA.com direct again. That means 20%-30% of all the ticket revenue will go to Opodo in future.

I’ve been avoiding BA for much of this year because of the strikes. This means I’ve rediscovered Heathrow Terminal 1 and flying with BMI and Lufthansa. Looks like I’ll be using them more.

Advice to companies: if you are going to make these kind of price or quality promises don’t hide behind word play when people try to claim. It annoys them even more.

I have only myself to blame, a quick bit of Googling shows other people with similar complaints - here and here.

Sorry to my regular readers for a well off topic piece.

I'm a BA get me out of here! - the BA role on an Agile Team

In the last couple of years I’ve given my “BA role in Agile” (PDF of the Nottingham version, or Slideshare of the Skills Matter version) talk a number of times at a number of different venues. A couple of months ago I got around to writing up the talk and it has now been published in ACCU Overload.

If you’d like to read this you can read it in Overload 98, or take the PDF from my website, “The BA role on Agile teams”.

Shorter versions of this piece have, and probably will be again, published elsewhere. The Overload piece is the full version. Thats the nice thing about Overload, it has the space to explore a topic more fully.

I’m just now putting the finishing touches to the 3-day training course that grew out of this talk and which will be running in September at Skills Matter.

Conferences, Conferences, Training - a busy autumn

I’m speaking at a bunch of conferences this autumn so if any reader out there would like to hear me speak, or ask a questions you might get yourself a ticket to one of these:

• Jax London / DevCon, 27-29 September, London: Return to Requirements, 27 September
• IRM IIBA Business Analysis Conference, 27-29 September, London: Objective Agility, what does it take to be an Agile Company?, 28 September
• Agile Business Conference, 5-6 October, London: “Quality – How much quality can we afford?”, 6 October
• Agile Eastern Europe, 8-9 October, Kiev, “The role of the BA in Agile teams”, 9 October
• Agile Cambridge, 14-15 October, Cambridge: “What does it take to be an Agile company?”, 14 October

In between times, I’m also delivering several public training courses - not to mention private ones but there is no point in me telling you about those.

What’s particularly interesting here is the new “Agile for BAs” courses that I’ve put together with Chris Matts. The first of these ran last week in association with BA Solutions and was well received. A different, longer, version of this, is running in association with Skills Matter twice this autumn:

• 13-15 September, Skills Matter, Essential Agile for Business Analysis, London
• 18-20 October, DevelopMentor, Foundations of Agile Development using Scrum Training and User Stories, London
• 1-3 November, ProgramUtVikling, Applying Lean Thinking for Software Development, Oslo
• 24-26 November, Skills Matter, Essential Agile for Business Analysis, London

The train metaphor of software development

I’m sitting on a train from York so it seems a good time to share my train-leaving-the-station metaphor with the world. In truth, if you’ve worked with me in the last few years, or heard me speak at a conference I may already have shared it with you. But for the rest of the world, and with full embellishments....

Traditional software projects are like a train leaving the station. There is a big train sitting at Platform 9, we know its due to leave soon, but, well, you know what big long distance trains are like, it may well leave a little bit late. Still, to get a seat we need to be early so we are all rushing to the train.

Since we don’t know when the next train to this destination will go - actually, it is far from certain there will ever be another train - we want to be sure everything we need is on board. So we make sure we put everything we might need on the train. (Think of our requirements document.)

Eventually the train lumbers out of the station, overloaded. Quite possibly it leaves late because we were so busy loading the train, maybe one or two people even argued with the guard to delay departure a little bit so we could get more people and things on the train.

The train - our project - is like one of those trains you see in pictures of the Indian rail network with people crowded on and hanging out of the doors.

We all know the train is overloaded, we all know its going to arrive late, we even think it might miss the destination and arrive someplace else. But nobody wants to admit this.

(Particularly true if you have American management and British engineers because the Yanks view the Brits as being overly negative.)

At some point, beyond the half way mark it can no longer be denied that the train will arrive late. So action is taken. Things are thrown off the train to make it go faster.

By throwing things off the train we hope the train will arrive closer to the scheduled arrival time and the intended destination. However, in so much as there was rationale for putting all the things on the train there is less rationale applied to removing them. Things are thrown off the moving train because we can. (Somethings we can’t throw off because they are too connected to other things.)

In the extreme, those who put things on the train, and really know what is needed for the destination (BAs, Product Managers, etc.) never actually got on the train. Once they put the bits on the train they handed over responsibility to Project Managers to get it delivered. These guys are primarily concerned with dates and since they charged with delivering everything they don’t want to throw anything off.

Eventually, the train lumbers into the final stop - which may, or may not, be the intended destinations - with some random collection of things on board. Everyone gets off and breaths a big sigh of relief. Thank God that is over.

Then the news comes: there will be another train. We begin again.

This time though, we have all been burnt. We are going to make sure we have everything we need on the train, plus all the things we threw off the last train, and some more besides because we now understand the need for bargaining and we need levers.

Thats the traditional view. Now for the Agile view.

Instead of big trains we have a metro system - think of London tube, or better still Glasgow’s Clockwork Orange.

I leave the office at 6pm, go to the tube station and wait for a train. The sign tells me there will be one in 2 minutes, and another 2 minutes after that, and so on.

The train arrives and it is full. I have an option: Do I get on the packed, sweaty train and have an uncomfortable journey? Or do I wait 2 minutes for the next one?

I choose to wait. And while I’m waiting my phone rings. Some friends are in a pub nearby and ask if I would like a drink. I have an option: do I go and have a nice beer (valuable) with friends? Or do I value getting home more?

I go for the beer, something I would never do if there was only one train today. But knowing there will be another train, and another train, and another, means that I can go to the pub safe in the knowledge that I will still get home.

Need I say is? We need software development to be like the metro/tube system and not like the big occasional train.

Slides from Skills Matter - BA Role in Agile Software Development

I presented my “BA Role in Agile Software Development” talk at Skills Matter last night. There was a good turnout and the folks who came to the pub afterwards were all very positive about the talk.

The talk had a few small updates from its previous deliveries so if you’ve heard or seen it before there won’t be anything new for you.

Slides are now online on SlideShare and Skills Matter have a video of the event online too.

The Limits of Agile on InfoQ

There is a piece by me on todaysInfoQ - The Limits of Agile.

I’m interested to here your comments.