How do I make testing faster?

Earlier this week I was the guest of a large bank in the City, OK Canary Wharf actually. They had their own little internal Agile conference. As well as myself some of the usual suspects were on parade as well as some internal speakers. It was very enjoyable and as usual the C-List speakers were some of the most interesting.

Later in the day I found myself in conversation with two people concerned about software testing. They posed the question: “How do we make testing faster?” Specifically, how do we make SIT (“System Integration Testing”) and UAT (“User Acceptance Testing”) faster?

Now of these solutions are far from unique to this particular bank. Indeed after listening to the question and situation I recalled several discussions I’d had, problems I’d seen and solutions I’d suggested before. (OK bad consultant me, I should have enquired into the problem, understood the context fully, coached her to find her own solutions etc. etc. But we only had 30 minutes and I wasn’t being paid for a full consulting session - nor am I charging you for reading this! - so lets not invent the wheel, lets fill in with assumptions and given some general answers.)

The solutions we came up with were:

1. More testing environments: it is shocking how often I hear testers say they do not have enough test environments. They have a limited number of environment and they have to time-share or co-habit. There are two answers: either increase the number of test environments or reduce the number of test and testers. I’m sorry I can’t give you a magic sentence that persuades the money people to buy you more environments. Now we can virtualise machines there is very little excuse for not creating more test environments. The usual one is cost. But even here the costs are not that high. (Unless that is you need to buy a new mainframe.)
2. Testers integrated & co-located with developers: if you want to go faster I’ve yet to find anything that beats face-to-face direct verbal communications. If you want/need people distributed then you should accept you will not be as fast as you could be. I’ve said it before and I’ll keep saying it: Testers are first class citizens, treat them as you would developers.
3. Automate SIT: simple really, automate the hell out of testing. Well easy to say, harder to do but far from impossible. Since most UAT/Beta testing is exploratory in nature it is largely SIT that need automation. And please please please, don’t spend money on tool test tools.
4. Make UAT just UAT: Recognise that some of what passes as UAT is actually SIT and automate it. In large corporates an awful lot of what is called “User Acceptance Testing” isn’t “User Testing” - although it may well be “Acceptance Testing.” Look carefully at what happens in this phase: that which is genuine Acceptance Test can - indeed should - be specified in advance (of the coding), can be moved into the SIT phase and automated. The true User bit is going to be much more explorative in nature.
5. Improve development quality with TDD and do it test first: one reason for long SIT and UAT phases is that they find bugs. One way to significantly reduce the number of bugs is to have developers practice Test Driven Development, i.e. build in quality earlier. This may mean coding takes longer in the first place but it will speed up the wider process. TDD is still quite new to many developers so you can speed up the process by giving them some help, specifically TDD coaching from someone who has done lots of this before. Until developers make the jump from “post coding automated unit tests” to “pre-coding test first development” you will not maximise the benefits of TDD, you will have more code in your system than you need and you will have a poorer overall design, and both of those make for more tests later on and potentially more problems. When tests are written before code TDD becomes a design activity.
6. Reduce batch size: i.e. reduce the size of the release, and probably do more releases. Do less, do a better job, push it out more often, reduce risk. Think dis-economies of scale. Many large corporations use a large batch size simply because any release is so painful. This is completely the wrong approach, rather than running scared of releases take them head on.
7. Done means done and tested in the iteration - at the end of the iteration it is releasable: when you reach the end of an iteration and declare work “Done” that means “Done ready for release (to customers).” Well, that should be your aim anyway and you should be working towards that point. If “Done” currently means “Done development, ready for SIT” then you should be working towards bringing SIT inside the iteration. If that looks hard right now then look again at items 1 to 6 in this list. And when you’ve moved SIT inside the iteration start on UAT. Until both those phases (and any other phases in the tail) are inside the iteration they can throw up problems (bugs) which will disrupt a future iteration which implies the work you said was Done is not Done.

Now the catch. Money.

Some of these things require money to be spent. In a rational world that would not be a problem because there is plenty of evidence that increased software quality (by which I mean specifically few bugs and higher maintainability) leads to shorter schedules and therefore reduced costs. And as a plus you will get more predictability. What’s not to like?

So rationally its a case of spend money to save money. Quality is free as Philip Crosby used to say.

But unfortunately rational engineers often find it difficult to see rationality inside big corporation operating annual budgeting cycles. One of the reoccurring themes this week - although not one I think was given official time - was the constraints of the budgeting cycle. The rational world of the engineer and the rational world of the account and budgeteer seem a long way apart.

There is a solution but it is a big solution which will scare many people. It is Beyond Budgeting, and that is why I was so keen to invite Bjarte Bogsnes - the author of Implementing Beyond Budgetting - to talk about Beyond Budgeting at this years Agile on the Beach conference. And its probably why Bjarte recently tweeted “Invitations to speak about Beyond Budgeting at Agile and HR conferences are now as frequent as Finance conference invitations. Wonderful!”