Sunday, October 28, 2007

Book review: The Innovators Dilmma

I’m just coming to the end of The Innovator’s Solution (Christensen and Raynor, 2003) - OK, I admit it, I can’t be bothered to read the last couple of chapters. Like so many books the first few chapters are very interesting, then the tend to get a little bit boring. I suppose this is because the first few contain the ideas while the later chapters are more about implementation. Still, I always make a point of reading the last chapter as this usually restores some of the feel of the early chapters.

That said, its a good book and interesting. I suppose more people have heard of Christensen earlier book The Innovators Dilemma. So why did I choose to read this one and not the earlier one? Well I guess because it was a) newer, and b) be seemed to offer a solution where the other offered a problem. Having read this I wonder if I would have been better reading the first book after all. That’s not to say this isn’t a good book.

Clayton Christensen is most famous for giving the world the theory of disruptive technologies. Such technologies are ones which, well, disrupt, the current market. Current market leaders fall, distribution channels change and small companies get to grow fast. All good stuff.

What gets less publicity is the alternative to disruptive technologies, namely sustaining technologies. These are technologies that can be used by the current market incumbents to maintain their market. In this book Christensen and Raynor suggest that in many instances companies can choose to use a new technology as disruptive or sustaining, it depends in part on how you package and market it, what market you are in now and which market you want to be in.

They also suggest that in an established market the incumbents will always win a straight fight with new competition. The only chance a new entrant has is to use their technology as a disruptive influence. Conversely incumbents need to use technology (maybe the same technology) to sustain their position. So if you are a new entrant don’t both trying build a better mouse trap, and if you are an incumbent don’t bother trying to disrupt the market.

What is nice about this book, although it does make it longer in place, is that the authors understand that a different context needs a different solution. Unlike many books which say: faced with problem X you should do Y; these authors say, if you face problem X and you are an incumbent than do Y, if you are a new entrant do Z.

The formula for a new entrant with a disruptive technology seems quite simply. Enter the market at the low end, offer a cheaper product than the current incumbents. Rather than compete head on target people who don’t buy the current products (non-competition) with a cheaper, less functional product. Sell to people who don’t buy the current product or are over served by it. Then as your technology improves expand this base. Quite often the current incumbents will move out of your way because they don’t want the low end of the market - that is if they notice you at all!

Of course I summarise and miss out many of the details but you get the idea. If you want to know more then read the book!

Interestingly much of the book’s key message is shared with Crossing the Chasm, they have the same idea but come at it from different perspectives. The key idea is: start with a small niche and grow it.

The advice in Crossing the Chasm is for small high-tech companies who wish to increase sales of their product. Moore suggests you define your own niche, nominate it then grow the niche. In this case the new technology is assumed to be superior - in some way - to the incumbents.

The advice in Innovator’s Solution is aimed at existing, probably large, corporations who want more innovation. Here the niche is always at the low end of the market, it is always a niche defined by cost and price. In this case the technology is superior through a lower cost structure, therefore price can be lower. Over time the niche is expanded by moving the product up market.

Most of the small software companies I see are so concerned with getting a sale, sometimes at any price, that the idea of niche marketing is foreign to them. Sales (and any marketing) tends to be a shot gun affair where any sale is what counts.

Indeed the whole idea of marketing is pretty foreign. Increasingly I see the existence of well functioning marketing as the key differentiator between successful and not successful technology companies. When I say marketing I mean both outbound marketing - telling people you have a product, advertising, etc. etc. and (the lesser known) inbound marketing - understanding your customers, knowing their problems. For both you need to define your target market.

Trouble is: you don’t need a marketing department to start a high-tech company. You do need techies, scientists and engineers to design and build the product. You do need sales people, someone to get out and sell the thing that has been built. If you lack either of these you don’t have a company. So you always find engineers and sales in any company.

But you don’t need marketing. You can carry on with the sales and engineers for quite a while without marketing. But if you want to grow you either need marketing or you need a lot of luck.

This is a mistake made by many many software companies. Which might just explain why Crossing the Chasm, The Innovators Dilemma and Solution sell so well.

Wednesday, October 24, 2007

Feeling sorry for EDS - business that don't know what they want

It is not often I feel sorry for EDS. Early in my career I had contact with EDS and I was not impressed. Since then nothing I have heard about them has caused me to change my opinion. So it is pretty unusual that I ever give them the benefit of the doubt, let alone feel sorry for them. But this time maybe I do...

Just at the moment EDS are being sued by BSkyB - part of News International - for failure to deliver on a project. In 2000 BSkyB agreed to pay EDS £48m to develop a new ‘state of the art’ customer service system. Unfortunately the project collapsed in 2003, BSkyB completed the project without EDS at a cost of £265m and are now suing them for £709m. The case is being covered by the FT if you want to know more.

Basically EDS claim that BSkyB didn’t know what it wanted. It seems the requirement was simply: a ‘state of the art’ customer service system to provide BSkyB competitive advantage over their competitors. BSkyB respond that EDS didn’t follow any recognisable project planning system.

I don’t know the rights and wrong of the case, what I do know is that businesses frequently don’t know what they want. I’ve written about this several times before (Software requirements and strategy and Requirements: A dialogue not a document for example). This case is about EDS and BSkyB so it is high profile but the same debate is acted out between thousands of IT providers and their customers every week. Because these names are less well know, and because these companies can’t afford to go to court we never heard about them and some agreement is reached.

This happens internally too. Companies with their own IT groups often get into this mess, the business can’t tell the IT people what they want. Perhaps it is because I’m in London but most of the stories I hear involve banks. Sometimes being part of the same company can help, and sometimes it makes things worse.

Rather than ask whose fault is it? we need to ask How can we fix it?

Well there are two parties here and both need to be involved.

The business side needs to set the overall goals - ‘build a state of the art customer service system to produce competitive advantage’. And the IT side - whether in house or outsourced - needs to implement what was asked for an no more. But in between there is a massive, massive, gap.

• What does a state of the art customer service system look like?
• What is competitive advantage in customer service?
• How do you avoid spending so much on ‘state of the art’ that you loose competitive advantage?

These aren’t easy questions to answer. Neither side can answer them alone and neither side can expect the other to answer them. You need co-operation, you need an on going dialogue.

Business needs to be able to articulate what it wants but it is wrong to think it can articulate everything up front. It can set goals but in achieving those goals there are thousands, indeed millions, of options and decisions to be made. The devil really is in the detail. Business and IT need to make choices together.

Making those decisions requires people in the role (business analysts or product managers), it requires IT people who understand business goals, it requires business people who understand how IT is created and how to recognise the benefits, and it requires a process to keep talking and making decisions.

In this case it is wrong to think BSkyB can just throw the problem ‘over the wall’ to EDS. And it is wrong of EDS to expect fully defined requirements documents up front. But, the nature of competitive business encourages people to take this approach.

I don’t know how EDS and BSkyB structured this project but here some advice:

• Start small: IT projects are often far too big, if you can find the real benefit it then stick to delivering just that, keep it small. Inside every large project is a small one struggling to get out.
• Requirements have to be driven by the customer, who needs to set business objectives and vision. However further elaboration needs to come from close collaboration between the business and the supplier.
• If you intend to build a ‘state of the art’ system, and gain competitive advantage then know your enemy, know what state of the art means and know what other systems do.

Finally, make sure you are building your ‘state of the art’ system for the right reason. Will a state of the art system really deliver competitive advantage? Maybe you don’t need ‘state of the art’, maybe you just need to use an ‘average’ system better. Maybe you don’t need to build a system at all, maybe you can just take one off the shelf. This won’t be cheap - especially if you have to customise it - but it will be a lot cheaper than starting from scratch.

Sunday, October 21, 2007

Presentations in Southampton and Cambridge

I should have mentioned this before but my blogging has been erratic at late - blame the ADSL line!

I will be giving presentations to the local ACCU South Coast group this group this week (25 October) and the week after the ACCU Cambridge (1 November). To keep things simple I’m giving the same presentation to both groups.

The presentation is entitled ‘Agile Software Development - Where do I begin?’ As the name suggests it will give some ideas and thoughts on how an organization can start to adopt more Agile practises.

Now for the interesting bit...

I’ve been asked to give a talk to a large media company in London the week after. For this I’m going to give a talk entitled ‘What’s wrong with Agile.’ Talk about contradicting myself!

There is a simple explanation... Agile is good, it describes a better way of developing software, it is a powerful story and introduces some new techniques. But, you shouldn’t just adopt Agile. Ask yourself: what is the problem? Then fix the problem, you may well use some Agile tools but don’t set out to adopt Agile, set out to fix your problem.

VikingPLoP footnote - I won a prize!

A footnote to my comments on VikingPLoP the other week. I forgot to mention: I won a prize!

I was awarded the shepherd of the year prize for helping another author with their paper before the conference. For those of you who don’t know how pattern conferences work a quick word of explanation...

Pattern conferences exist to help pattern authors improve their papers. After a paper is submitted to the conference it is reviewed and, provided it is accepted, the author is assigned a shepherd. The shepherd works with the author for about three months to help improve the paper. At the end of the process it is the revised paper that is submitted to the conference.

At the conference the paper is reviewed in a workshop. The author then uses these comments to produce the third version of the paper. This version is then included in the conference proceedings.

So, shepherding is an important part of the PLoP system and culture. More information on the Hillside website.

It is an honour to receive the VikingPLoP shepherding award, or to give it it’s full title: The Neil Harrison Shepherding Award. Actually, as some readers will know, this is not the first time I’ve won this award. I won the same award at EuroPLoP 2005.

I’d just like to say thank you to the VikingPLoP committee and the author I shepherded this year for awarding me this prize.

New website

A blog is a website but it is not a website. It is a website in the sense that it is a destination on the web but the contents of a blog are by their nature different to a website. Years before I started this blog I set up my own personal website, With the creation of this blog I have neglected my website and it has become old and dated.

So it is with great pleasure I announce the rebirth of I’ve used a new (Mac) package to create a new website which is mainly an archive of my writing projects. Here you will find over five years of articles from ACCU Overload, patterns from seven or more conference, presentations from conferences and elsewhere, and various other stuff.

For the moment I’m keeping my company website, separate. I’ll be looking at this in future and may merge the two. After all, three websites is probably enough for anyone.

Wednesday, October 17, 2007

Something is stirring in the office software market

As a PC user there was only one game in town for office software - Microsoft Office. Yes there was other software but very few people used it. Everyone used Word, Excel, etc. and if you used anything else people thought you were strange.

Since moving to the Mac I’ve discovered the hegemony doesn’t exist here. I’ve discovered people who use NeoOffice and as Mark points out in his comment today there is Apple software too.

Funny thing is I first met Mark when we both worked for a company which produced UNIX based office automation software, Quadratron. In fact the company had made a good job of missing the PC market and realised it needed to catch up. Somehow Mark, Adrian, myself and a couple of others weren’t going to make it happen. Once it became clear the company had no future we jumped ship.

But now it appears the office automation market might be on the move again.

Yesterday I used the Google spreadsheet and word processor for the first time. In fact I was using them in collaboration mode with Till Schuemmer as we started the process or organising EuroPLoP 2008. They worked well - despite my very slow ADSL connection - and I was impressed. However shortly after we finished the ADSL connection collapsed again so while Google Apps might be the future they are not the present.

Also this week I learned that IBM is launching, or maybe re-launching, a office automation suite called Symphony. Lotus Symphony was an integrated office package in the days of 8Mhz 80286 CPUs with 640k RAM. It worked but not well enough too make much of a mark.

Why has IBM done this?
Will Google Apps (or ADSL) ever be stable enough to challenge desktop software?
Will NeoOffice or Apple displace Microsoft on the Mac?

I don’t know the answers to these questions but I do see the start of a change.

Sunday, October 14, 2007

VikingPLoP and Software Engineering Radio

Two weeks ago I was at VikingPLoP in Bergen, Norway. I have never been to Norway before so I enjoyed the trip. Bergen is absolutely beautiful, it has to be one, if not the, most beautiful towns I have ever visited.

Anyway, a bunch of interesting things came up.

Personally I had another paper with business patterns work-shopped - as soon as I’ve done the changes I’ll post it online.

There was a lot of discussion about VikingPLoP 2008. All is not well in the VikingPLoP world. VikingPLoP is a small (less than 20 people) conference and it moves each year. This raises a number of problems and it is not my place to say any more. However as a result of these problems I think BritPLoP has taken a step closer.

BritPLoP is the name of a conference that doesn’t exist. It is an idea Kevlin Henney and myself sometimes discuss - usually after a few beers. If it ever takes place it might be branded as VikingPLoP, it would most probably occur in late September and would be the first true patterns conference to occur in the UK. It won’t happen in 2008 but I think it might well happen in 2009 now.

Also at VikingPLoP I was fortunate enough to meet Bob Hammer. Bob is president of the Hillside Patterns group and a really interesting guy. Like so many people in the patterns community he software engineer with a multitude of interests, one of which is book-binding.

Bob has written a new book entitled Patterns for Fault Tolerant Software which should be available in the next few weeks. I’ve seen some of these patterns before and even used a few. But until now I didn’t know Bob was behind them. I’m sure this book will be a major contribution to software engineering knowledge.

Although I don’t do much coding any more I’m going to buy this book. I know I’m going to learn a lot of new idea and techniques. And if nothing else the book includes one of my favourite patterns of all time: Leaky Bucket.

When we were in Bergen Bob was just waiting on the print run but in the meantime he had printed the final proofs and hand bound them. This makes him the first person I have ever met to have: written a book, printed a book and bound a book!

Bob also told me about Software Engineering Radio. This is an internet radio station from Markus Voelter and a few others. I haven’t had time to check out all the podcasts but the ones I have listened too are impressive. This looks like a great new way to spread software knowledge.

Still no ADSL - the state of the software industry

We are now in week 4 without ADSL. Things have improved a little, we have a 192kbp (yes kilo, not mega) link most of the time but it can disappear at any time. If I unplug the router I probably won’t see it again for several days. If we stretch the bandwidth too much it will probably collapse and it will be absent for some days.

The combination of BT, or rather their wholesale unit BT OpenReach, my ISP (Eclipse) and some unknown supplier who sits between them seems to make resolution impossible. As far as I can tell the fault is just shuffled between them. Eclipse would like to get the fault fixed I’m sure but they are, as far as I can tell, quite powerless.

I phone Eclipse just about every usually there is no progress. Sometime I have to reboot the router or switch it off altogether for a day. The it takes about 48 hours before BT do the next thing. Then we’re back to square one.

Turns out BT offer no service level agreements for broadband. It is provided on a ‘best efforts basis.’ For something that has become so essential this isn’t really acceptable.

It seems to be another sign that for all our technical progress the reliability of services is decline. Take digital television, my Philips digital box regularly crashes and requires a reboot. At other times it just goes unbelievable slow.

Fixed line phones are still as reliable as ever but mobile phones can still be poor quality, lines can get dropped and messages lost. While Skype offers superior sound quality its reliability is even worse.

Increasingly we (the population as a whole) seem prepared to trade reliability and quality of service for new features.

Unfortunately I take this as a failure of my own industry, software development. Our ability to developer fancy software is far greater than our ability to produce reliable and bug free software. Our solution to this dilemma seems to be to change the entire world, instead of fixing our quality problem we will just persuade people that this is as good as it gets and they can live with the bugs.

That is what I find so very very sad.

I’m doing what I can. Fortunately in my area I have a choice. To my disbelief I’ve now contracted with NTL to install cable broadband and telephone to replace BT and Eclipse. NTL are now branded Virgin Media but they are traditionally a by-word for poor customer service.

If it works - and it is two weeks before they can install - then I’ll get a faster broadband connection and pay less. Hard to believe, I hope it works. For once the devil I don’t know seems preferable to the devil I know.

Wednesday, October 03, 2007

Agile failure modes - again

Back in April and May I discussed failure modes in Agile software development. Well I’ve found another. That is to say I’ve found another way in which an Agile development practise can go wrong. I’ve seen this before but I didn’t recognise it as a failure mode, I saw it as a little local difficulty now I’ve seen it a second time I understand it a bit better. The trouble with this failure is that at first it looks a lot like what should be happening.

One of the core ideas behind Lean and Agile is improving quality of code. Improving code quality enabled several other benefits. Improved quality means fewer bugs, so future development is less disrupted because there is less rework. Improve quality means less time spent in test at the end of a development. Thus short iterative cycles become possible. And improved quality means you can deliver (or just demo) any time. So quality improvement is important.

One way in which we seek to improve quality is through automated unit testing. This is usually undertaken as Test Driven Development or Test First Development. Now there is a whole host of reasons why it can be difficult but that is another blog. With automated unit tests fine grained functionality can be tested very rapidly which provides for rapid regression testing and increased confidence in new code.

This is where the failure comes in.

We actually want the tests to fail if we change something because they prevents errors slipping into the system. So we add a test for new code, write the new code and by the time we check the code it is finished and the test pass. But, if later something changes which will break the code then we want the test to fail in order to catch the error that has been introduced. In this case we want the test to be fragile so that it catches anything which is now wrong.

The problem occurs when the test is fragile in the wrong way. In the case I’ve just observed the test was fragile when data in the database changed. Previously when I saw this the tests were fragile around COM. The test was/is not sufficiently isolated from its environment.

True dyed-in-the-wool Agile/TDD practitioners will immediately jump to say: you shouldn’t do it that way. And they are right. These fragility points should be addressed. This might be by stubbing code out, using mock objects or by tearing-down and restoring some element - such as a database.

As a consequence instead of the tests helping you validate code and move faster they become a hindrance. The need to update and change the tests regularly adds to the work load and complexity. Instead of the test showing everything is good they add to the maintenance burden, the exact opposite of what was desired.

The problem is that those trying to take the Agile/TDD approach have good intentions by they don’t have the experience, but how do you get the experience without doing it?

Well there are three answers here: first the organization should have invested in training for these people. Second the organization could have hired someone with experience to help them, perhaps as a part-time coach or perhaps as a full time co-worker. And third, the management could have helped the teams learn faster and better and encouraged them to reflect more on what they were doing.

Trouble is Agile development is often introduced bottom up, so people do just jump in. And even where management introduce it then such training and coaching can be expensive - that is if you know who to call and where to buy it in the first place. (If you don’t know who to call then let me know, I know people who do this stuff.)

So although it is well intentioned and the ‘just jump in’ / ‘just do it’ approach can get you started but it can also start building up bigger problems further down the line. If you know what to look for you can spot the problems and - at least try to - take corrective action. If you don’t then it is quite likely you’ll give the whole Agile experiment a bad name,