Les Hatton's site and the importance of letting failures fail

I recently stumbled across Les Hatton's site. I've owned one of Les' books for probably over decade (Safer C, a classic) but this was the first time I've looked at his website. I highly recommend a perusal of the site and reading some of his papers. Apart from anything else he's an even more interesting guy than I thought he was (and I already though he was interesting!).

One set of his papers looks at the power-law nature of software code. I'll admit I couldn't follow all of the academic speak and advanced maths terms - which might disqualify me from reading some papers but it
more a sign that I didn't take the time to work it out in depth. (Most of his papers are written in a very accessible style.) The key point I took away was that he extends James Noble's work to non-OO systems. Hopefully I’ll find time to go back and read more of these papers in depth and make sense of the bits I skimmed this time.

Another paper which stood out was “How to build successful complex systems: lessons from open source” with his suggestions for how to build the NHS IT system. Until I read this I was unaware that the Welsh
Health Service had taken a different route to the English one, had spent far less money and had actually produced a successful system.

Les, like many others, sings the praises of the Open Source movement. He’s right, there are some very successful Open Source products out there, but, as I’ve pointed out before, most Open Source
projects fail. And when I say MOST I mean tens of thousands fail for every success.

Fortunately Les’ suggestions allows for this; he suggests a Darwinian process with seeds 100+ development efforts and over a series of rounds reduces them to one winner.

Although Les suggests this in an Open Source context what he is actually suggestion is also a Venture Capital model. He proposes to give over 100 teams a little bit of money (seed money in effect) to create an initial system. Year on year winners are selected to continue development, these are given a bit more money. That sounds exactly like Angel funding, VC Round A, VC Round B, etc. etc. funding. OK, Les model is “open” because the winning teams get to pick over the remains of the loosing but even that exists in commercial activity: failing companies get bought by the winners who keep the best bits.

Either way the result is the same: rather than one “too big to fail” NHS IT project we have a number of small projects which compete and can fail. If we remove the ability to fail from the process then we have to
pay for failure.

Whether it be evolution, capitalism, IT projects or even banks, if failures are not removed from the system then change is inhibited because the winners cannot really win.

This is the important thing about failure. Forget that myth about learning from failures, the important thing is that it eliminates something that doesn’t work.