Agile creates jobs

Regular readers will now that over the last two years I’ve done a lot of work in Cornwall, I jokingly refer to “the Cornish Software Mines” (A case study and Light Touch Coaching to name two blog entries.) To date the most visible result of this work was the Agile on the Beach conference which is running again next week.

There is now a report out which has looked at the programme overall and concluded that the Agile Cornwall Programme created 50 jobs. That might not be a big deal for London but in Cornwall that is a big deal. Particularly since half the jobs were high paying.

So far I haven’t actually seen the report so I just have this report on the report to go on. I’m trying to get a copy as we speak.

I can’t take all the credit, I had help. The thanks need to go to:

• Oxford Innovation for creating and running the programme
• Michael Barritt who was the driving force while of Oxford Innovation and has since moved to Sullivan Cuff Software (one of the programme’s success stories)
• Jon Jagger who provided most of the technical coaching
• Benjamin Mitchell who coached several of the clients when I was fully stretched and gave intensive help to one of the more difficult clients
• Plus Kevllin Henney and Sebastian Bergmann who provided additional training, and Nancy Van Schooenderwoert who provided specialist embedded consulting

This report also looks it should help providing some evidence for Agile.

There is still time to book tickets for Agile on the Beach if you want to meet all these people and some of the people who benefitted.

Version 2 "The Rewrite" - don't do it! - never ever rewrite you system

“The second is the most dangerous system a man ever designs” Fred Brooks, 1975 & 1995

Brooks was talking about software designers, architects, but I think the statement holds true not only for all software developers but for the business people who commission replacement systems.

From time to time I come across a team who want to rewrite their system. Usually this is the technology team who are saying “The code is too complex, the thing was written by a jerk, She cannae take it Captain”.

Sometimes is the business side realising that their system looks out of date - Visual Basic 6 controls on Windows Vista, or turn of the century web style.

Either way there is a “Stop the world I want to rewrite” mentality. Sometimes the powers that be buy the argument, sometimes they bless a rewrite or, as one team called it a “Technical Improvement Programme” - TIP for short.

Lets me come to the point: this is always and everywhere a mistake. You should never, ever, rewrite a system just because the technology is old.

This is a mistake because customers don’t want to change. The may think that VB6 controls look odd but they know how to use it. Even if change would benefit them they are scared of it. Plus, they have their own business to run - or life to live - and changing systems is only going to get in the way.

Second, this is an utter waste. Statistics regularly show that as much as 80% of features in a typical software system are not used. So why implement five times more functionality than you need?

One company I worked with commission an cheap Indian company to rewrite an old PowerBuilder application in Java. They were told to make it identical. So they did, they spent a lot of time making Java look like PowerBuilder. They copy every feature, even the ones not used. It was buggy as hell and the users didn’t want it.

Third: the “just a rewrite” mentality leads people to under estimate the complexity involved and to overlook the opportunities for improvement. Why produce a second system which is identical to the first?

Finally, and worst of all: the world doesn’t stop, you can’t afford to stop and rewrite, nor can you afford to waste the opportunity to produce a really good system, one that is better than competitors.

Yes, I agree your current system might be at the end of its life and you might need to throw it away but the solution is not a rewrite. When this happens you have two options.

The first won’t satisfy every problem but is the least risk: refactor your way out of the problem.

Actively encourage programmers to refactor code, of course you need to embraced Test Driven Development at both the development and testing level first of all and start putting tests in. Thats not to say you cover it in tests first and then refactor, rather you start putting the tests in as you go. This is a bit of high-wire act without a safety net at first so I strongly recommend you get some help if you’ve not done it before.

This is real engineering, this is changing the engine oil while the engine is running.

That is the easy answer but it won’t work every time. Sometimes the changes are too much, say from VB6 to C#.

The second answer to develop a new product, a competitor product. One for which your current system/product is the competition. This development effort needs to be market lead and focus on what customers/users want the the current system doesn’t provide.

I’ll say it again: Market led. The marketing department, the product managers and/or business analysts should decide what goes in. Not the developers.

When companies rewrite they jump to the assumption: “The developers know what the current system does so they just need to copy it.” That is the biggest mistake.

Next, get into the market fast, release early release often. Test what you are building in the market, show your company real, usable progress. Remember, your competitor is the current system, if you can’t do better than it you want to know fast.

The objective is to build a system that your current customers want to use, one for which they are prepared to give up the system they have at the moment. Consider this a variation on Same Customer, Different Product pattern (available online or in the Business Patterns book).

Emphasis the things the new system does that the old one doesn’t. Don’t worry about all the features the old one has that you think you need to launch. You might need them in the end but nobody is going to buy your software because it does the same thing. Add them as you need them, you’ll find you need to bring across far fewer than you initially think.

Now I apologise to all the developers out there who are begging their managers to let them rewrite the system. Your code base may be terrible, the technology may be out of date but a development led rewrite is not the answer. Few companies can survive such a thing.

Selling scopeless contracts

The last two blog entries (“Scopeless contracts: the problem” and “Scopeless contracts: the solution”) have been working up to this one. Actually this is the blog entry I set out to write a few weeks ago but felt I had to explain why.

Even if you accept that leaving scope out of a contract for work is the right thing to do you still need to sell it to your customers. This is, perhaps, the real problem. However this is not always the case. Some clients will immediately recognise the problem, or might not wish to sign a big up front deal.

So, faced with a customer who wants to know “How much will it cost to do X ?” what should you say? Or rather, what can you emphasis to persuade them that you have a better idea?

(Note, on anything other than a trivial piece of work X will not be one thing but many, call them X1, X2, X3, … Xn.)

1. Flexibility: scope less contracts build in flexibility, if the client wants Y instead of, or in addition to X then thats OK. No need to scrap this contract and start again.
2. Work can go away: Suppose X3 become pointless, then it can just go away without any trouble. As opposed to the UK Government Connecting for Health project [[link]] were actually cancelling the project is more expensive than letting it finish.
3. Room for innovation: if you sign a big contract and later think of a bright idea then something has to give. Scopeless contracts allow for this because feature negotiation is ongoing.
4. Bounded by money: Rather than being bounded by scope contracts are bounded by the money available, or sometimes the time available. This forces the creators to work within a very real boundary rather than poorly defined scope and time estimates.
5. Governance control: bounding the contract in money provides for real governance rather than the governance by proxy that scope, features and line items provide. When the money runs out the work stops. (This of course assumes the team are making regular deliverables, if they aren’t you have even bigger problems.)
6. Inventive to reduce work: putting 2, 4 and 5 together you get an incentive to reduce the work to be done rather than inflate. In traditional work scope is problem because removing it creates contract problems and loss of face, but more importantly: there is little incentive to remove work until difficulties hit.
7. Reduce delay & waste, keep options open: All the time spent writing and negotiating scope delays the time when you actually start delivering something. If things change, and the longer you spend writing the scope the more likely things will change, then the more of that work which will be wasted. And the more you think you know what you want the more you constrain the options for producing the optimal solution. Selling without scope reduces this work because scope is derived as you go within the process and only enough is produced as is needed.

You can, and people do, do all these things within traditional scope based contracts but there is additional work to do to change the contract. And, more importantly, it starts the contract with an illusion, a myth: the idea that one day things might be done. Which gives us:

1. Traditional work relies on a scope myth “scope won’t change”. It does but we pretend it was because something was wrong. The result is we spend time and money managing the myth: managers need to revise contracts, staff change boards, write complex governance reports, etc. etc. In other words: because managing scope is doomed to failure the thing we call “scope” is a chimera which we waste time and money on “managing”.

Despite three blog entries I still don’t feel I’ve said what I wanted to say. This is a topic I need to return to….

Scopeless contract: the solution

In my last blog entry, “Scopeless contracts: the problem” I rehashed all the reasons why including scope in a contract for delivery is a bad idea. So having rubbished the most common way of working I had better explain what I suggest you do….

You have two choices.

Option 1: Set up a ‘Business as Usual’ contact.

The client nominates work to be done and the development team do it. Simple. This is the way most organisations work “business as usual” but once the idea of a project enters the scene people expect the project to have an end date, in order to have an end date they imagine that some amount of work needs to be done.

You can still have an end date but instead of measuring progress against “work to be done” (he scope, the backlog) measure progress against the value added.

After all, software is never finished. If you ever run out of requests for a software product it means nobody is using it. However it might not make sense to do all the work that is requested.

How do you size the team? You don’t, you start small. You start with just the money you can afford to loose. After a while, say three months you review what is bring done and decide whether you wish to go faster (expand) or slower (contract), or perhaps stop altogether.

(Another reason to start small is to reduce the effect of Conway’s Law: if you start big you will get a big programme of work, if you start small you might just come up with a small solution.)

Option 2: Set a Goal, work to the goal

If option 1 isn’t governed enough for you then set an overarching goal. Every three months (or what ever time you decide) ask yourself: are we making progress towards the goal?

You might even want some people on the team measure progress towards the goal (these would normally be Business Analysts). All the work you do should be tested against the question “Does this move us towards our goal?”

If you want more structure use my 10 Step Requirements model.

Yes, it really is that simple.

Drop the backlog. Set a goal. Work business as usual. 1, 2, 3.

The thing is, when you contract with someone to build you software its not like asking them to build you a house. You are buying a service not a thing. If you could buy a thing you probably would have done already. Once you make that mindset shift it all makes sense.

The only complication is, well there are two really: first you have to accept the project model is broken so you need to make more strategic decisions. If you need to keep projects for accounting purposes but projects should have nothing to do with development.

Second you need to do this on a rolling basis, review it every three months.

For most organisations the biggest problem is that they have come to believe the project myth, and they believe it all the way up the hierarchy. Until the people at the top really understand that IT doesn’t fit into the project model its difficult to change.

Scopeless contacts: the problem

I’ve discussed Agile contract models before and I’ve agued against points based contracts (‘Points based contracts? Just say No’) but the more I think about client-supplier software development contracts the more I think including any scope in the contract is asking for trouble.

Before I explain let me say: Yes I know this isn’t going to be possible either with your clients or your own sales folk, I’ll follow up this blog entry with another setting out what you should sell instead.

Now I’m using the word scope, although I hate the word it is the word most people recognise. I mean: “the work which will be done.” Call it a requirements document, a PRD, a product backlog or anything else. Its a shopping list of things you think you want.

Right now, here is why I think fixing scope inside a contract is a bad idea:

Scope is always unknown: you may think you know it but you don’t because things change, learning occurs (on all sides), when someone see something they change their mind about what it is they want, or what they ask for, or what they want next.

I’ve been over this ground again and again, first in Why Do Requirements Change (2004), again in Changing Software Development (2008) and more recently in Dear Customer (2012).

And the bigger your project is the more your requirements will change and grow. Capers Jones gives the industry average as 2% per calendar month for large projects. If you’ve got a project/programme that is up and running you can go and measure it yourself. One team I know saw their product backlog increase by 50% between January and July.

Someone said to me recently: “Surely the team has got better at this now, they’ve learnt.” Well yes, they will have got better, that might take the edge off, might rule out some human factors but the world hasn’t stopped.

The World is Changing: lets suppose for a moment you are running a big project, say its been running 2 years and you think it has another 2 years to run. Cast your mind back 2 years, what was the world like in July 2010 ?

First some context: David Cameron has been British Prime Minister for 2 months, Osama bin Laden was alive and well, few people outside Japan had heard of Fukushima, Deepwater Horizon had just been capped and oil was under $50 a barrel.

• The first generation iPad had been on sale for 3 months, it was far from obvious that Apps were about to take over the world
• Nokia were still pushing Symbian phones and were still the biggest mobile phone seller and Android was in version 2.2 (Froyo)
• JP Morgan was fined £33m for failing to protect client money
• Greek debt was downgraded to junk in April
• Barclays were fixing Libor

Go back a little further: Windows 7 was release in October 2009. Further still: the iPhone was released in June 2007 and the AppStore in July 2008. Not that long ago.

Did any of those events effect your business?

Did any of those events, or countless others I haven’t mentioned effect what your business wanted?

Now ask yourself: what could happen in the next two years, to 2014, that would change what your business, your customers want?

President Romney? Grexit? Finexit?

Scope should reduce as well as increase: when you fix scope you also make it difficult for scope to reduce. Requirements should be reducing as much as increasing.

Marty Cagan recently blogged about “The Opportunity Backlog”. Your backlog is not a list of things that will be done, it is a list of things which could be done, and you should do as little of it as possible (its expensive), instead focus on the opportunities which deliver the most.

You human, you make mistakes: well maybe not you personally but the people who work for you.

There are mistakes in what you ask for, mistakes in what you build and mistakes in how people use it. Sometimes fixing a mistake is more important than doing something new.

Second system effect: there is a blog entry waiting to be written about version 2 projects, for now lets stick with what Fred Brooks said in 1975: “The second is the most dangerous system a man ever designs. ... The general tendency is to over-design the second system, using all the ideas and frills that were cautiously side-tracked on the first one.”

Whether you are designing from a technical, user or business perspective you suffer from second system effect.

OK, am making my point?

Since requirements can and will change there is not point in writing them into a contract.

This point is doubly dangerous if you attached estimates and timelines to the contract because you will be wrong again. Human’s Can’t Estimate.

This is bad for the supplier who will sooner or later have to explain what is going on to the customer and its bad for customers who don’t get systems that work.