What is the right size for a User Story?

A question that comes up again and again from teams is “What is the right size for a User Story?” - a day? a week? three weeks? - 1 point? 5 points? 10 points? 37? 100? 5 words? 20 words?

Short answer: there isn’t a universal right size for a User Story, it depends on your team, their skills and their level of domain knowledge.

Long answer: for some teams a User Story is Big, several days or weeks of work. On other teams they are small - maybe a day’s work. Both are possible, both are the right answer. Although, as a general rule smaller is better.

For me there are criteria that a User Story should meet:

• It should be small enough for the technical team to understand it and create in in a short time period
• It is small enough to move on the board sometime soon
• It should be big enough to represent business value in its own right - it might build on something that has been done before (e.g. a lower fidelity version of the same story, the new one increasing fidelity)
• It is big enough to be deliverable in its own right - you might not want to do so but if you needed to you could

In an ideal environment: the story would be developed in a few days, would be released on its own right and the business value measured directly. (For those who know about Minimally Marketable Features (MMFs), also sometimes called Business Value Increments (BVI) or Quantum of Value (QoV), I casually consider a User Story to be a MMF.)

When a team are dedicated to a particular product, and have worked on it for several years, and have learned about the domain - as I would expect in for in-house development operations - I expect to see larger stories. Conversely, when a team don’t know the domain - as is common with outsourced development - I expect to see small stories, and more knowledge pull from the client.

Taking a step back, a few basics:

• A User Story is not in and of itself a requirement; I like to call them “Tokens for Work to be Done”, others like to say they are “A placeholder for a conversation” both are true
• Don’t feel that you must use User Stories: if the format doesn’t fit your need then don’t use it! You can still have a “token for work to be done” and a “placeholder for a conversation” using any words you like, they don’t have to be “As a… I can … So that…”
• The User Story format does capture three very important elements: Who, What, Why so they make a good starting point and are easy to understand but if they don’t work for a particular need then don’t force it into the format

Small stories are better - they can be developed more quickly, delivered more quickly and return value more quickly. But there comes a point were stories become too small to return any business value at which point they, well, loose value. Plus, when you have a lot of small stories they become a headache to manage.

One common problem I see is teams who create really small stories in order to get them to fit within one sprint. These means large stories don’t get scheduled or teams split large stories into many small ones which have no business value themselves.

People say this is a Scrum thing, I’m not sure. Scrum doesn’t mandate User Stories, the story format came along after Scrum. True the format is widely used they has become part of Common Scrum.

What Hard Core Scrum does say is that all the work the team commit to should be done by the end of the Sprint. Which would imply a story has to be small enough to fit I the Sprint. The problem then comes: what else do you put in the Sprint? If one story is going to take more than half the Sprint you need one or more stories to use the rest of it up. Plus you hit the commitment problem - developers have an incentive to under commit and the business/custom has an incentive to over demand.

My solution - one which I’m baking into my description of Xanpan - is:

• Stories are split into Tasks during the planning meeting
• Tasks do not follow any particular format, nor do tasks have any business value
• Tasks are for the team - the developers, testers, analysts and anyone else
• Tasks should be small, a day or two at most.
• A story is not done until all the tasks are done. Stories can therefore flow across iterations, the chances of tasks flowing across iterations is low (because they are small) but they can and do.

However - and this is the key point - in counting the points (velocity) of a team only the completed Tasks are counted. Partially done tasks are not. The only states that matter are Done and Not Done.

As I’ve written before, the breakdown of stories to tasks fills (at least) three purposes:

• It is superficially an estimation exercise
• It is also a requirements elicitation process (I like the Requirements Engineer, BA, Product Owner/Manager to be present)
• It is a design activity.

You know a task is too big because it is difficult to finish. If it is difficult to finish it needed more thought up front, to define and understand it, to bound it, to understand it. (See the mini-series on Story Points earlier this year, and specifically the Story Breakdown entry.)

So there you have it: there is no right size of a User Story.

That said, they may still be “too big” or “too small”. But what constitutes right it dependent on you, your team and how you play the game.

Downside of Dialogue Sheets

Something I’m very conscious of is that I don’t hear much about the downside of Dialogue Sheet Retrospectives. As much as I’d like to believe that this is because there is no downside I can’t honestly say I think that is the case.

There must be downsides.

I have had several suggestions for improvement and some of these have been incorporated into some sheets. There are also a couple of pending improvements which I’ve yet to action. Because there are several sheets and because not all of them are for retrospectives some improvements get incorporated into some sheets but not others. In part I’m happy with this because it increases the differences between sheets.

At XP Day this week I had several conversations about the sheets and I now think its worth listing some reasons people choose not to use the sheets or negatives that emerge. Some of these I’ve known about for a while and may have mentioned before, some are new.

In no particular order, and in my words summarise what I’ve heard….

“The team looked at the sheet and don’t see how they would be an improvement on the current retrospective format”

This reason has been given by several people/teams and while I respect it as a team’s decision to make I can’t help feeling that it pre-judges the sheets. Surely the “Agile” way of doing it would be to try a sheet and see what happens. After all, what have you got to loose? One or two hours, one retrospective.

“We need a facilitator”

Almost a continuation of the previous one another reoccurring comment is from people who can’t imagine not having a facilitator. Again I would suggest try it and see.

I have observed many retrospective dialogue sheet sessions and I find them very interesting. If a facilitator wants a role I have several suggestions:

• Take off the facilitator hat and join the retrospective, you will get insights for the next time you facilitator
• Observe silently from the sidelines: you will learn things just from observing
• Use the sheets from one retrospective as input to the next, go back and pick up items, delve deeper into issues
• Use the sheets as first exercise in a longer retrospective; do a sheet then go back over issues that arise

You might not need a facilitator but that does not mean you must not have one.

“Facilitator would go deeper”

I am convinced there are issues and times where having a facilitator will produce a better outcome. For example, some issues benefit from talking about in depth. That is why I always thing the sheets are one tool amount many. However I don’t think you always need to go deep. As with the first two items in this list i encourage you to try it and see.

Cross training retrospective facilitators

This might be a rare reason for not using the sheets but is quite valid. One company wanted to train a number of people to under take retrospectives so each retrospective was used as a training exercise for a facilitator.

“We have a company mandated retrospective format and we cannot change it”

This has to be the worst reason for not trying any new technique in a retrospective. If a team cannot change the retrospective format what chance have they got of acting on any findings from the retrospective? Actually I think this suggests a bigger problem.

Mechanical sheet filling

I only have a second hand account of this and I’d like to know more. It appears a team at a large media company in Salford do not engage in conversation in doing the sheet. It appears filling in the sheet has become something of a chore and they simply “do it”.

With out knowing more about this team (anyone know them?) it is hard to say what is going on. It sound like the sheet has been imposed on the team and they are completing it like another piece of administrative paperwork.

Fear of team dynamics

Some people are worried that personality issues on a team will disrupt the dialogue sheet process - conflicts, arguments, etc. I think this is a valid concern. Although I then immediately think of two things.

First: this is an example of a bigger issue - if these people can’t spend an hour together on a collaborative exercise then working together must be very difficult.

Second could this type of exercise actually help with those dynamics? I would guess the answer is sometimes Yes and sometimes No.

Bored by repetition

There are a few teams that have tried the sheets used the them regularly and then they have fallen into disuse. I suspect this is true of any retrospective format that is used too much. The trick is to find whats to keep the retrospectives fresh and keep them generating new insights.

If you know of any downsides or problems with dialogue sheet retrospective then please let me know. I might be able to correct it in a sheet, if I can’t then we can at least understand their use better.

10 pieces of advice for teams

I have started to write up a more useful description of Xanpan. In doing so two things have happened. First I’ve realised that I have an awful lot I would like to say about it, I’m terrified it will become another book. Second, I’m becoming more and more aware of how Xanpan differs from Scrum, XP and Kanban.

As a result I expect this blog will get a little quieter. But before the end of the year I’d like to get a few entries finished and published which have been languishing for a while.

So without further ado here are 10 tips for teams and those who manage, administer or simply organise teams. Of course, if you are a self-managing team you should all read this list.

1. Don’t load overload teams - consider 76% utilisation about the max
2. Keep teams together - bring the work to the teams, don’t form teams around pieces of work and then break them up when it is finished (Corporate Psychopathy)
3. Minimise disruption to the team, if the team needs to be regularly interruptible and responsive then set processes and expectations accordingly
4. The more self-standing, free of dependencies, the team the more effective they will be
5. The team should own the process and methods of working as well as the things they work on. By “own” I mean: they are allowed to change the process and methods.
6. Make the boundaries and responsibilities of the team clear
7. The team needs a full skills set to fill 4,5 and 6; and those with the skills need to be on the team full time
8. Aim for an MVT (Minimally Viable Team) so put slightly fewer people and skills on than you initially think, expand the team as they show success but always keep them minimal
9. Aim to recruit slowly and regularly, avoid foie gras recruitment, i.e. stuffing the team full of new people in the hope of increasing quantity in the near future (you won’t, you’ll cripple yourself in the short term)
10. Give the team as much authority and power as possible to do their work, the more decentralised the organisation the more effective this will be

Looking at that list two things strike me. First is how many of my older blog entries are linked. Second about three of those items are things not to do. Simply not doing bad/mad/stupid things can deliver benefits.

Hard-Core Scrum, Common Scrum and Scrum (continued)

Since I published Scrum, Scrum and Scrum on this blog last week the piece has received a lot of attention and I’ve had a lot of feedback: comments on the blog, comments on Twitter and comments to me in person at Oredev last week.

On the whole these comments have been in agreement. My impression is that people find the posting a useful distinction between Synonym-Scrum, Hard-Core-Scrum (or ScrumTM as I’ve also called it) and Scrum-Lite (or Common-Scrum as it might better be called). In this blog entry I’d like to clear a few things up and reply to one particular comment from Kurt Häusler which I think deserves more attention.

Just to be clear: I’m not attacking Scrum here. I’m discussing difference in how it is perceived, what people call Scrum, and where I think “Scrum” is inconsistent (OK, that might be criticism.)

First terminology.

In using the term ScrumTM I am being sarcastic. Nobody owns a trade-mark on Scrum, anyone can use the term Scrum and the ideas in it. So lets stick with Hard-Core Scrum.

Scrum-Lite also seems to upset people, this name was meant to designate a form of Scrum which is not Hard-Core-Scrum. Lets use the term Common-Scrum because this is the way I most typically see the Scrum ideas implemented.

Next the issue of Project Managers, and to a lesser degree Architects and other specific roles on Scrum. Kurt wrote:

‘Anyway, anyone saying project managers and architects are not allowed in Scrum is simply wrong. Are they really saying that though? And are people saying that really representing a different meaning of the term "Scrum”?’

Taking it in reverse order: I am the one who says there are two-forms of Scrum. I think this makes a useful distinction which helps me, and I think others, understand the world we live in.

Kurt’s first question is the more interesting one, if the answer was “No”, if the world did have a homogenous view of what Scrum is the second question, and the distinction would be unnecessary.

So Kurt, the answer to the first question is a most definite Yes. I think the point is well made by Christin Gorman in her 10 minute video on Vimeo from the Roots conference - “Putting an architect in a Scrum team is like putting mayonnaise in a cake.”

Let me say straight away I’m not picking on Christin here, I actually agree with much of what she says and she is not alone in this view. There are other - better known - names who say the same thing:

• In The Scrum Primer Deemer, Benefield and Larman say “there is no role of project manager in Scrum.” These words are in version 1.1, there is a Version 2.0 on the Scrum Foundation website which adds Bas Vodde to the author list and contains the same words. I haven’t read v2 in detail so maybe in contains more advice for project managers.
• The Scrum Handbook on Jeff Sutherland’s website contains the same words - in fact the biggest difference to The Scrum Primer seems to be the author list.
• I personally heard, several years ago, Craig Larman answer the question “What does a project manager do in Scrum?” by saying “If you have a project manager in a Scrum Team you are not doing Scrum.”

The important point is: Christin understands Scrum to mean “No project managers” and Kurt understands Scrum to allow project managers.

I sympathise deeply Christin’s point at the end of the video, if I may paraphrase: “lets save Scrum to mean what it was meant to mean”. I just happen to think this particular cat is out of the bag. Rather than trying to “save” or “reclaim” the name “Scrum” I prefer to differentiate.

Interestingly, as I have commented before (“When did Scrum start loving project managers?”) Ken Schwaber hasn’t always shared this point of view:

• In Agile Software Development with Scrum Schwaber says “The Scrum Master is a new management role introduced by Scrum” and a little later “The team leader, project leader or project manager often assume the Scrum Master role.”
• In Agile Project Management with Scrum Schwaber says “The Scrum Master fills the position normally occupied by the project manager. I’ve taken the liberty of redefining the role.”

Now perhaps I’m guilty of not checking my facts. I haven’t e-mailed any of these people to ask where they stand on Project Managers and perhaps I should have. (If any of you are reading this please feel free to comment.)

However, I don’t believe a definitive statement would make much difference. What if I had a statement from one of these people saying “Project Managers are in” (or out) ? There would still be people out there interpreting Scrum the other way. It is too late to change this now.

Personally I believe that overtime those who define what Scrum is - those named here plus a wider community - have changed their position on Project Managers. I suspect that in the beginning there was no anti-Project Manager bias. Then is crept in, and now it is less, or perhaps gone altogether.

The points I want to make are:

• there is are different understandings of what Scrum is
• I, and I think others people, find it useful to make a distinction

I really don’t care if you want to say “Common Scrum is not Scrum because…” or “Hard-Core Scrum because…” is not Scrum, because I’m not describing what should be, or not be. What I am describing here is how I find the world.

Comments please?

Scrum, Scrum & Scrum

I’ve come to believe there are three different meanings of the term “Scrum” - well, three inside the software development community at least, if we consider sport we can probably add a fourth.

Even if nobody else does I differentiate between:

Scrum as a synonym of Agile

“Scrum” means “Agile” like “Hoover” means “Vaccum Cleaner” in English English, or like “QTips” means “Cotton Bugs” in American English, or increasingly “Google” means “Internet Search” whether you are using Google, Bing, Yahoo or some other search engine.

This is the way I believe a lot of software people use the term Scrum. Although Agile is more than Scrum alone often (mostly?) when people say Scrum they mean some a general form of Agile, something with iterations, stand-up meetings, perhaps User Stories, perhaps some technical practices, etc. etc.

Hard Core Scrum, ScrumTM

This is Scrum without Project Manager or Architects, maybe even without Testers. This is Scrum of Commitment and Abnormal Termination of Sprints.

I once heard Craig Larman say “If you have a project manager on a Scrum team you aren’t doing Scrum.” Only last week Christin Gorman Tweeted saying: “If you want to use managers and architects, you won't be doing scrum. And that's ok.”

I’ve written before about how this hard-core Scrum might actually conflict with Scrum (When did Scrum start loving Project Managers) and XP (Two ways to fill an iteration).

This view sees Scrum as a kind of Communist Manifesto but with Managers as the Bourgeoisie. It would be funny if this weren’t so serious. I believe that if you take this interpretation you will soon run into opposition from a layer of managers. The paradox is that Scrum is as popular as it is because those manager don’t like Extreme Programming and saw Scrum as the management friendly alternative (The Three Advantages of Scrum).

Common-Scrum or Scrum-Lite

This is Scrum but without the anti-manager zeal. Of course for many people this strips Scrum of its primary asset - and they might be right. It might be that if you neuter Scrum you get a less effective result.

In this form of Scrum Project Managers still exist - assuming you had them to start with, a lot of places don’t. However they have been renamed Scrum Masters - see my example from 2008.

In Scrum-lite the word commitment really means “agreement” because the team are probably using velocity to judge how much work they can do. Architects and other ranks, sorry, roles, still exists. And companies still kill teams - corporate psychiatry is rampant.

In other words: it is full of compromises. It works for some companies, it doesn’t work for others and ScrumTM purists will hate it.

Anyway, I find it useful to distinguish between these three uses of the term “Scrum”. If you know any more please add them to the list.

Unspoken Cultural differences in Agile & Scrum

For a while now I’ve been convinced that a lot of “Agile” is about cultural differences. In particular I believe the canonical version of Scrum, which I often refer to as Hard Core Scrum or Scrum™ is rooted in 1990’s American software management culture.

Unfortunately the role of culture behind many Agile techniques and methods isn’t really stated. This make it even more important to work out what Agile means to you and which tools work in your environment and culture.

I first started paying attention to the cultural differences around teams and Agile after Steve Freeman said something along the lines of “Scandinavian teams just do this, they don’t see much new.” The teams I’ve seen in Scandinavia, and to a lesser degree Holland and the UK, don’t need big lectures in self-organization, left to themselves they do most of it.

I’ve pondered on this for some time and at Agile Cambridge last month I got the chance to talk a little about this with Dave Snowden. We only had a few minutes to talk about this - I was about to present - but it was clear we could have talked for hours.

Take stand-up meetings for example.

When I worked in Silicon Valley I worked in a cube. I hated it. I didn’t even know if my neighbours were in the office, let alone what they were working on or if they had achieved anything that say.

Working in the UK, in open plan offices, usually sitting opposite my co-workers we would certainly know who was in, most likely they would tell me when they had done something.

I believe that stand-up meetings are a good thing in all teams. However I believe they make a much bigger difference in cube and office dwelling environments than in open plan offices. In other words: stand-up meetings have greater benefit in US offices than they do in European offices.

In the original book of Extreme Programming Kent Beck talked of a “sustainable pace” and “40 hour work week”. Beck talks about his experience in Switzerland and contrasts it with the US norm. Although European workers - particularly in the UK - frequently work more than the 40, 37.5 or 35 hours they are contracted for they work hours don’t get excessive for too long. I’m not sure that is the same in the US. Sustainable pace means different things.

Personally I’ve always found “sustainable pace” does not fit well with the Scrum idea of “commitment”. I’ve written before - Two Ways to Fill an Iteration - about the contractions I see. Culturally it isn’t hard to see how “sustainable pace” is easier to do in Europe than the US.

(By the way, I’m limiting myself to the US and European, specifically UK, cultures because these are the ones I have some experience of.)

Now lets talk about the big one: Self-Organizing teams and evil managers.

(Apologies if this comes across as Scrum bashing, I believe Scrum works, or at least Scrum-lite works, and I believe self-organization is best. I just don’t believe hype.)

Some Scrum courses and advocates make a big thing out of self-organization. Personally I don’t. In my courses I I talk about it a little, more if people want to, but I focus on giving people experience of how it works. My style of Agile - yes I’m slowly writing up Xanpan - believes that self-organization is the result of the right approach not a stating point.

Self-organization goes hand-in-hand with an attack on Managers, and in particular Project Managers. Again this theme runs through all Agile but in Scrum is particularly strong. Hard-core Scrum really has it in for Project Managers but then replaces them with Scrum Masters which most companies think is just the same job with a different name. I’ve written about Scrum’s contradictions with Project Managers before so I won’t repeat myself, When did Scrum start loving project managers?

But Scrum’s dislike of managers only goes so far. After all, Scrum is the management friendly version of XP - see Scrum has 3 advantages over XP.

Now I’m not a big fan of Project Managers, in my programming days I too suffered a few managers who tried to hand out tasks and impose their way of doing thing. And I’ve been told No when I want to spend a little money - I once threatened to leave the company if they didn’t buy more RAM for my development box.

When I look back over my career I have to say most of the (project) managers who behaved like this made themselves irrelevant. We worked around them. Certainly removing them would have made us more effective.

But, most of my experience is that managers, even project managers, didn’t behave like this. They may not have been the most empowering of people but generally they left me, and programmers I worked with, to get on and do it. We worked out what needed doing, we divided the work up amount ourselves, we scheduled at the micro-level.

In other words: we did self-organization on a day-to-day level.

Again I think there is a cultural difference here. In my experience America is a more hierarchical culture than Europe (with the possible exception of Germany). Please, I’m not talking class systems here, I’m talking command and control.

I think Europeans are more likely to question their managers to their faces - perhaps it was the 1914-18 experience - or just plain ignore them.

National characteristics are not the only basis for culture. Industries have cultures too. Here again I’ve written before: Agile will never work in investment banking - one reason being that investment banks are very hierarchical.

Most of the managers, even project managers, I meet when delivering Agile training courses and consulting, like the Agile approach and want to work with it for the benefit of their teams. Few seems to be the whip-cracking project managers of folk lore.

Jon Jagger tells a story about one of his clients in Norway. The company was bought by an American competitor. When the engineers started to work together they would convene a teleconference. The American programmers would arrive with their managers while the Norwegians would arrive by themselves. The American’s would ask “Where is your manager?” The idea that the Norwegian could talk and make decisions by themselves was a new idea.

I had a manager in California - Irish as it happened - who used to practice “management by walking around.” Every morning he would appear in my cube and chat. Took me months to realise he was my manager and when I did I wished I had been more guarded with some of my remarks.

This all begs the question, why would American management be such a hinderance to software development? (Note I’m not saying European management is better, that has its own faults, I’m just exploring different cultures.)

For many years the USA was the world foremost manufacturing nature. Part of this success was the superior management practise implemented by the Americans. Such practices made a difference on the factory floor but even here they have been .

I can, and should, write more about managers and their role in the software team. Suffice to say for now: there is an awful lot of bad management out there. I believe good managers, good management practices, can be help companies massively. But on the whole, in the IT industry, there is a lot of poor management.

Bringing this back to Agile.

I think Agile’s anti-management slant is really a reaction to bad management. Hard-core Scrum is the Extreme in this respect. Scrum-Lite, as practiced by most teams, is less so.

Agile, and particularly Scrum, is the product of 1990’s culture, and particularly American work/management culture. Since American culture spreads the fact that Europe never suffered from quite these ills may down to the fact that we were not that good at copying American ways.

There may be a silver lining here: if America adopt Agile, Scrum, self-organization, etc. then we should see it spread out to the rest of the world.

If you disagree with me I’d love to have your comments on this blog. And if you agree I’d love to hear your stories to. Please, leave a comment.

Dialogue Sheets feedback and stories

Retrospective Dialogue Sheets continue to be a popular download, and I’m off to Sweden in a couple of weeks to present the sheets at Oredev. Unfortunately I haven’t had time to review the downloads to extract any useful information from the several thousand downloader details but I do continue to get some interesting feedback.

A few months ago Gail from Siemens Healthcare e-mailed me to tell me how she had used the sheets to conduct a distributed retrospective. The team in India had one sheet and conducted their retrospective in their work hours. Several hours later when the US team arrived at work conducted their own retrospective using another sheet. Gail then gathered the findings and reviewed the two sheets and conclusions which were complementary.

More recently I heard from a Scrum Master at Stattnett in Norway about their use of the sheets. What follows is a (slightly edited) account which nicely describes why many people like the sheets:

“thank you for the sheets! We've now used them in two retrospectives, and we will continue to use this technique for the future.

My main reasons for applauding their use:

• The sheet stimulates equality among team members during the discussions, by the fact that all must be seated around a table (no one standing up and facilitating the discussions).
• The sheet rules demand at least some contribution to the process from each team member during the session, by reading out questions loud - and also decide for themselves what is needed to read (and not to read).
• The sheet is very physical and with few/no barriers for participants to express themselves on the sheet for everyone to see - even if it is only by drawing "stick men" on the edge (next time, maybe even the stick-man-painter will write some words on the sheet).
• The sheet and rules (when followed) prevent the Scrum Master from falling into a "team leaderish" role when the rest of the team gets lazy and wont't bother to involve in the process - they all have to take their part of a common responsibility for getting through the questions and to the end of the "game".

...and there are more.

These characteristics make our retrospective sessions less bureaucratic and more inspiring than with our old technique. We have been using the more traditional technique for several years, with each team member preparing notes before the meeting, writing successes and suggested improvements (often as complaints...) on green and pink paper memos.

The notes were then collected on a whiteboard and discussed and voted on at the end - with the Scrum Master (me) as facilitator. This preparation of notes [was rushed] and no one really likes to do this. Going through each and every note was time-consuming duty with debatable value. [In the end] the retrospective meetings became quite uninspiring form for all of us.

But with the dialog sheet, we now have a better, more inclusive and more constructive climate in the meetings. We spend our time on issues of real value to the team instead of ceremony parts no one really likes, and more in line with the original intentions for this session.

The sheet can be put directly on a wall after the meeting, for everyone to see. I don't have to make any kind of additional summaries anymore (as I did, even if core scrum doesn't prescribe this). I can instead take a photo of the sheet for the record, if needed.”

I regularly hear that of teams who hang the completed sheet on the wall as a reminder of the discussion and conclusions. I have also heard of a team who hung a blank sheet on the wall at the start of an iteration and team members filled in the timeline as they did the work.

Finally, I recently discovered a blog posting from University College Dublin which used the Dialogue Sheets as part of student retrospective exercise. I don’t believe these are the first College to do so, University of Central Lancashire might have that honour.

Since I learned about the sheets from Cass Business School in London I like the idea that the sheets are going back to college!

Correction: Business Analysts & C3 project

In a couple of blog post, and on other occasions in other places I have talked about there being a Business Analyst working on the original Extreme Programming (XP) Chrysler C3 project.

I’m not sure where I picked this piece of information up but I was wrong. One, Chet Hendrickson of the original team read my post and the team actually had direct access to the payroll team to understand what was needed, agree acceptance criteria, etc.

The Wikipedia entry on C3 discusses a “customer representative” on C3 and indeed Extreme Programme suggests you have a onsite “customer”. Somewhere along the line I read, was told, or just got the impression that this person was, on the C3 project a Business Analyst. Now I see I was wrong.

Having set that, I would still suggest that a Business Analyst can fill the role of customer representative and often have experience and training in doing this. I will accept that sometimes a BA doing this might become a block and it would be better for the team to talk directly to the customers.

Why try Kanbnan?

In my last post “Scrum doesn’t work for us; should we try Kanban?” I gave a warning about adopting Kanban because Scrum doesn’t seem to work. Having given advise on when not to use Kanban I feel I need to suggest when choosing Kanban is the right decision.

Of course list that follow is only broad guidance, each and every team needs to make their own decision based on many factors I cannot even guess at here. Nor is this an extensive list, I’m sure Karl Scotland, David Anderson, Benjamin Mitchell and others would add more to the list but its a good start.

Here goes….

Use Kanban for maintenance: teams who fix bugs, especially teams who fix bugs in live system. Such teams work respond rapidly to unpredictable work, bugs are difficult to estimate and can come up at any time.

Use Kanban when predicting future work is difficult: this is a generalisation of the previous case but expanded. It is not just bugs which are unpredictable, sometimes the work which needs to be done is unpredictable.

Note: some teams fall into this position not because the work they do is really difficult to predict but because there is no effort put into predicting the work, usually because nobody is tasked with looking ahead or the person who is asked to do so isn’t allow enough time to do so. The easy way to spot this is the absence of someone with a title like: Product Owner, Business Analyst, Product Manager or similar.

Use Kanban when the team find Scrum style routines stop them achieving flow: for effective teams who focus on rapidly turning new requirements into working software Scrum routines such as planning meetings can get in the way. Nor is there a lot of point to running a product backlog if items are being delivered rapidly or not done.

Use Kanban when the system is too complex to model with Scrum(XP): the plan-it, do-it, deliver-it model which underpins Scrum isn’t too helpful when a process has a lot of wait states or dependencies. The standard Scrum answer is probably to remove these but that isn’t always possible (e.g. you can’t force a customer to respond).

Use Kanban when learning is a higher priority than delivering: to be effective Kanban requires to learn from what the system is telling you and adjust. If you can’t put the learning into effect Kanban will not be effective. However, Scrum’s pre-packaged solutions means that many teams seem to adopt Scrum and forget about learning (e.g. retrospectives are forgotten.) Worse still, because Scrum (claims) to fix so many problems “out of the box” teams may develop a learn dependency.

Use Kanban when you need to introduce the Agile approach more gradually: strict Scrum mandates a big-bang approach, sometimes this isn’t possible. Kanban provides a foot in the door which becomes a lever for opening it more. (This point flows directly from the previous when you consider change to be the result of learning.) However this also comes with a warning: change can quickly stall if people do not act on what they see and learn.

In the past there has been debate about which style (Scrum or Kanban) a team should start with and which is the advanced style. Personally I find it easier to start with Kanban although many people feel the natural order is the opposite: start with Scrum, get good and advance to flow and Kanban. Although it is easier - in my mind - to start with Kanban it is also a lot, lot, easier to get it wrong.

There is a paradox there: Scrum mandates the role of Scrum Master, I don’t usually see the need for a Scrum Master, an occasional Agile Coach is helpful, but a full time Scrum Master No. Kanban doesn’t mandate any such role, but you probably need someone with past experience far far more with Kanban than you do with Scrum.

Scrum doesn’t work for us; should we try Kanban?

Two and a half years ago I published a blog entry entitled “10 things to know about Kanban software development” which has consistently been the most read piece on this blog. At first I thought “Well there isn’t much published on Kanban relative to Scrum” but then David Anderson published Kanban and that wasn’t true any more.

Others have written about Kanban too over the last two years but still there is a hunger out there, people want to know about Kanban. I see this too when I deliver my Agile training courses, the moment Kanban is mentioned people want to know more.

Although Kanban is conceptually simple you really need a day or two to discuss it properly. Still, I regularly find myself squeezing an hour, or just 10 minutes, into a Agile training course because the audience want it. I say “I really need a day” but I can’t avoid it - they are paying for my time so they get what they want.

One of the reasons people give for buying an Agile (Scrum/XP style) course and wanting Kanban is, they say, “we don’t think Scrum will work for us so we might try Kanban.” Then when I return to coach teams I’ve given training to I’m regularly asked “Scrum doesn’t seem to work for us, should we try Kanban?”. I reserve judgement and enquire before I give my advice, and what I find is….

They haven’t really tried Scrum(XP).

Or rather, they have tried a Scrum(XP) type approach and they hit problems. Despite my telling them this would happen they think its Scrum. The thing is, Scrum fixes some problems “out of the box” but it doesn’t fix them all, in fact it exposes old problems and highlights difficulties which you then have to fix. Thats the way it works, that is the way it is meant to work.

Let me repeat that: Scrum and XP do, in and of themselves, right out of the box fix a lot of problems software development teams typically have. But Scrum and XP do not (despite what some people will tell you) fix all problems right off: you have to continue the work and fix them yourself. Sometimes Scrum, as officially described, will just not work, you have to modify it to your environment.

What worries me when teams say “Scrum doesn’t work, we want to try Kanban” is: if a team can’t make Scrum(XP) work because they cannot fix the impediments or make their own modifications then Kanban will never work. In fact, Kanban will probably be dead on arrival.

Some years ago Karl Scotland said “there is a subtle difference between kanban and typical agile processes such as Scrum. Scrum focuses on being agile which may (and should) lead to improving. Kanban focuses on improving, which may lead to being agile. However, being agile itself is not important - it just happens to be the best way we (or at least I) know at the moment.”

Thus: If you want to try Kanban because Scrum doesn’t seem to work for you then the chances are Kanban is only going to be a diversion. Set about fixing your problems.

Now there are reasons why Scrum doesn’t work, and there are reasons you should try Kanban, that should be a separate blog. One common case is because the development team has some support or maintenance responsibilities. In these cases my XP-Kanban hybrid Xanpan works quite well.

(I keep taking about Xanpan and a few people have asked me to write about it but a) I don’t think the world needs yet another Agile method, and b) until recently I wasn’t convinced there was enough difference between Xanpan and either Kanban or Scrum/XP to make it worth while. On the second point I’m changing my mind. If you want to know more about Xanpan let me know.)

Back to topic, although a team might not need to move from Scrum/XP to Kanban or Xanpan there might still be advantages to doing so. However moving because you can’t overcome your own obstacles is not a good reason for changing. If you can’t make Scrum work because you can’t fix your own issues switching to Kanban is not the answer.

Minimal Viable Team to create a Minimally Viable Product

Despite being a bit of a mouthful to say “Minimal Viable Product” and the even more difficult to say “Minimally Marketable Feature” (also known as a “Quantum of Value” or “Business Value Increment”) are very useful concepts. What makes gives them killer power is that they speak to a secret belief held by many people (not just managers) that teams gold-plate development and create products with more than is needed.

The trouble is, while we all agree the we should develop Minimally Marketable Features (MMF from here on) and delivery Minimal Viable Products (MVP) it is actually quite hard to do this. Nobody seems to want to sacrifice their favourite feature or the styling they have set their hearts on.

In Business Patterns I have a side box entitled “Strategy means saying No” in it I say “One of my personal rules of thumb is that strategy is about saying “No”. Strategy is not so much in what you decide to do – saying “Yes” is easy – but in the hard decisions about what you choose not to do.” The same applies to product development: saying Yes to a feature is easy, saying No is hard, but unless you say No a lot more than Yes you won’t have a MVP.

(After completing Business Patterns I discovered that both Michael Porter and Tony Blair have made the same argument, that No is more important than Yes.)

I have come to believe that one of the reasons teams find it difficult to create MVPs made up of MMFs is that the teams are just too big. I would like to suggest that if you want to create a MVP then you need a Minimally Viable Team. i.e. a team which is only just big enough to create the product.

Indeed, I will go further and suggest that your Minimally Viable Team (MVT) should be slightly smaller than you think you need.

Basically my logic is: if you have more people you will create more product. If you constrain the amount of effort that can go into a product then you will constrain the thing that is produced.

Way back I worked on Railtrack privatisation in the UK. When I joined the team is was 120 people (although only about 12 actual coders). The project was massive. Out of that came by belief that inside every large project is a small one struggling to get out. I sometimes call this Kelly’s First Law of project complexity: Project scope will always increase in proportion to resources.

A few years later I found myself working on a data feed project at Reuters. In retrospect the team was overstaffed. We didn’t need COM based system with interchangeable components. We made work for idle hands.

I know one team I’ve been visiting for over a year now, MMF and MVP frequently come up in discussion but I see the product backlog getting larger and larger and larger. The team is quite large by today’s standard - over 30 depending on how you count it. And ever since people started talking of enlarging the team the backlog has grown faster.

So much for the observations, lets try logic.

If you think about this the need for a MVT to create a MVP is a direct result of Conway’s Law: organisations will create systems that are copies of the organisation. (Seven years ago Lise Hvatum and myself investigated Conway’s Law in a conference focus group, the result was “What do we think of Conway’s Law now?”).

This can also be stated as the Homomorphic force: the structure-preserving relationship between two sets of things.

If you create a team which is too big it will produce a product which is too big. Once you have created a team which is too big it will find work for itself to do and it will be very very difficult to reduce the team size.

So start with a small team: actively try to break Conway’s Law, break the homomorphic force.

Staff the team with a fewer people than you think you will need, staff it with a range of skills but avoid making architecture assumptions in your staffing (if you include a DBA and you will have a database in the system.) As and when the team demands to grown then grow the team grow it.

Use Conway’s Law to your advantage: create a Minimally Viable Team to create a Minimally Viable Product.

How small is a MVT? I would say two people.

Really you need three people for it to be a team but in the belief that you should start smaller than you think I say two. One of these people should be focused on the business side of things - requirements, need, problem domain call it what you will; the other on the technology or solution domain. Let them work the problem for a while and see what happens.

Development Partners - branching out

My company, Software Strategy, has existed for over five years now and has been successful within the terms I’ve set for it. Now its time to try something new.

That doesn’t mean I’m giving up Agile - although I must admit to getting sick of the word - nor does it mean I’m giving up delivering training courses - if anything I’m expanding here, Giovanni Asproni now delivers the courses too. Nor am I stepping away from Coaching - although I increasingly prefer the term Consulting because I do give advice and I think true Coaches don’t.

No this is an addition. Software Strategy is now going to undertake work for clients - you can subcontract your work to us. I’m teaming up with Alan Griffiths and Giovanni Asproni, I’ve known these guys for years through the ACCU (formerly the Association of C & C++ Users if you want to know).

There are three reasons for doing this:

Opportunity: I believe that Agile working provides an advantage to companies, I also believe there are opportunities for companies that work this way.

Better: Alan, Giovanni and I truly believe we can do a better job than many of the companies out there, call it skills, call it experience, put it down to Agile or Craftsmanship or even the ACCU. We just do. And the engineer inside of me says: if you are better you will succeed.

Improvement: I believe that Software Strategy, and those who work for and with it, can benefit from adding this feedback loop. We can see how things work in our own practice and feed it into the training and consulting we do.

Thats the Why, but What?

To start off with we are focusing on the object oriented languages, specifically C++, Java and C#. We intend to provide requirements analysts, project management, coding and testing, i.e. full cycle. This might not be the fashionable end of development but it is where we have common experience and, despite fashion, is still a very active area of development.

We plan to undertake Greenfield projects (i.e. completely new code bases), Brownfield (i.e. enhancing existing products) and Rejuvenation. That last one deserves a few words, if only to say I expect large scale, significant refactoring, to be involved.

There are a lot of code bases out there which remind me of crippled Starship Enterprise, the Engineers are shouting “She cannae take it Captain” - well actually, they are probably saying “Technical debt, technical debt, technical debt…. can’t modify this system” and the more they say it the more the business switches off to engineering.

I, we, believe that it is possible to turn these systems around. We have the technology - or rather the tools and experience.

Anyway, as Software Strategy Development Partners proceeds I hope to have some interesting things to blog about. (And yes, there is a strategy map of business patterns in my notebook.)

So, if you have any work you’d like to sub-contract you know where we are.

Thoughts on Mind the Product conference

As I mentioned in the last blog posting I was at Mind the Product last week in London. For the first time in a couple of year I was just a paying punter at a conference, I was not speaking!

Here are a few notes on Mind the Product - partly as feedback to the organisers, partly for myself, and partly to encourage more people to attend next year.

Good points

• That the conference was happening at all in London was perhaps the best thing for me. 10 years ago product management (as practices in Silicon Valley) didn’t exist in the UK. 5 years ago a few forward looking companies got it. Now it is more broadly known but still not widely enough.
• Two concerns I had about the conference before hand both proved unfounded. One that it would be a little basic in terms of content and two that it would but all very Lean Start-up orientated. There was lots of mentions for Lean Start-up and “pivot” seemed to be the word on everyone’s lips but neither fear played out.
• Marty Cagan as a keynote, I’m envious, I tried to get Marty to speak at Agile on the Beach but he was already booked of this. I was interested to hear him say that he thought the UK was improving with product management over the last few years. I’ve been saying the same thing.
• Tom Hulme of Ideo London gave a great talk about purpose, although I think it was really a talk about business strategy.

Things that weren’t so good

• Some sessions really lacked focus or content - e.g. the talk on the games industry had lots of good ideas but they were not finished, they needed more research and thinking; some of the other presentations left me wondering “what is their point?” but then, Marty Cagan and Tom Hulme set a high standard to match
• The venue made things difficult, it was spread out, several vertically connected locations, getting food at lunch time involved finding the end of a queue and waiting. You could tell it was an old theatre where the intermission spaces were not meant to be occupied for long.
• The format (one track, presenter talking to the audience, no interaction) left us sitting in our seats for too long without enough breaks
• Recruitment is often an undercurrent in conferences but this time it was too much, everyone who spoke seemed to say “we’re hiring”. Many of the people at the conference had their tickets paid for by companies who would rather their people were not lured away. A bit of recruitment is to be expected this was too much. (At Agile on the Beach we made a conscious decision to not take sponsorship from recruitment agencies.)
• As is so often the case at conferences the wifi kept coming and going - maybe this was on purpose to stop too much twittering and checking of e-mail

I hope the conference runs next year, in which case, this is what I’d like to see change next year:

• Have two or three parallel tracks to give choice
• Have some interactive sessions to move away from the chalk and talk
• And thus allow for more frequent breaks
• Different venue: one that promotes mingling, discussion and interaction

Having said I hope the conference runs next year I have an even bigger hope: that product management becomes a feature of other conferences in the technology sector. Product Management shouldn’t be its own ghetto with product managers talking only to other product managers to agree that if only product managers ran the world everything would be right.

Agile Cambridge: Dialogue Sheets and Andy Warhol

Last week was busy with conferences. I was briefly at Agile Cambridge on Thursday and attended Mind the Product conference in London on Friday (more on that in the next entry).

I feel the need to apologise about Agile Cambridge, I was there for a little over half a day. If I go to a conference I like to be there for longer. As it was I saw Dave Snowden’s keynote (very good) and ran a Dialogue Sheets session.

As usual the Dialogue Sheet session was goo and I hope a few more people will try retrospective Dialogue Sheets - there were people in the workshop from the BBC and McKinseys who said they would be giving them a try. But the most amazing thing about this session was the room: there was an original Andy Warhol Marylyn hanging on the wall. I was warned by the organisers several times not to put anything on the walls or pictures!

As anyone who has used one of my Dialogue Sheets knows there are some thought provoking quotes around the edges. As chance would have it both sheets included an Andy Warhol quote: “They always say time changes things, but you actually have to change them yourself.”

Here are some pictures:

Postscript: The few slides I used are here Dialogue Sheets for Agile Cambridge 2012.

Does anyone actually use Quick Test Pro etc.?

Here is an observation which I’ve been been playing back to clients for a year or so now:

• There are two types of company in the world, those who talk about automated testing and those who do automated testing.
• Those who talk about automated testing buy tools like Quick Test Pro, WinRunner and Quality Centre and Rational Test Workbench, Rational Quality Manager, i.e. expensive products from the likes of HP and IBM.
• Those who do automated testing use tools like Selenium, Fit & Fitnesse, Cucumber (inc. Gerkin, RSpec, etc.) and other Open Source products they don’t need to buy

For completeness I should add:

• There are a few, very few, companies which don’t talk about automated testing but on the whole everyone thinks it is a good idea and those who don’t do it would like to
• There are a some companies who talk about automated testing and have no tools; this is a better position to be in than talking and spending money

This is my observation, and if I’m being honest I did see a company 12 years ago which used WinRunner in a load testing environment. But, on the whole the software test suites from IBM and HP tend to be shelfware. In fact, this blog is motivated by a trip to a company which found my observation very funny because QTP is sitting on a shelf unused.

What I would really like to know is: has anyone else seen this? Or, can anyone give a counter example?

I think there is even a logic here. The IBM and HP products are expensive, so expensive you have to ask the price (actually IBM does give the price of Test Workbench at $5,500 for a single user license). Consequently they need to be sold, it also means that they need to be bought at a high level in the corporation. The people who are sold these products are very disconnected from the day-to-day work of developers and testers. This means the products might not be suitable and even if they are then developers and testers need to adopt them: they aren’t part of the change decision.

In addition: I just don’t think these tools are any good. To be fair, I’m no expert on testing tools but I’ve seen Quality Centre in action. Quality Centre (QC) isn’t an automated testing tool, its a very expensive test tracker. Testers write their test scripts in natural language, e.g. English, into the tool, the tool runs on the side and as they execute the tests they click Success or Fail. At the end QC gives a report.

To my mind QC is a fancy version of Microsoft Word. English language test scripts, if they are any good, are so detailed that they take almost as long to create as automatic scripts but they are expensive to execute because they are manual.

QTP and similar products are often linked to the OS or browser. They result in fragile tests which break easily when a box moves a few pixels or the browser changes.

HP also follow a razor blades model, I’m told: once you’ve bought QTP you need to buy a plugin for every browser to OS you want to use it with. Every time a new version of the browser comes out you need to buy a new plugin - or so I am told, tell me if I’m wrong.

The net result is: these tools are high maintenance and easily fall into disuse if they are ever used.

On the contrary: Open Source tools are adopted by people who do the work because they see the benefit they bring and they don’t need budget or signatures to get the tools. People get them because they want to use them not because a Salesman comes around and convinces the boss that you need a tool.

And because many of the Open Source tools require developers to create glue code to interface the tool to the application they lead to a more collaborative style of working.

(Perhaps some companies who just talk about automated testing have downloaded some of the Open Source tools but never use them tool. Since they are free nobody notices.)

So there is my theory. Anyone else seen this? Anyone else explain it? Anyone got a counter example?

Success in Agile

Last week I made the London to Cornwall train journey for the first time in a few months. I made the journey many times during the course of the Agile Cornwall programme. This time however I was on my way to the Agile on the Beach 2012 conference. The conference itself is a product of the Agile Programme and looks set to run again in 2013 (although the final decision has yet to be taken.)

With the release of a report last week which claimed the Agile Cornwall programme created 50 jobs I felt Agile on the Beach is something of a celebration. I’ve now had a chance to read the report in full and while the report is not available for public download I would like to share a few comments from it.

First I should say that the report itself was conducted by a third party company commissioned by Oxford Innovation. Oxford Innovation runs the Grow Cornwall programme on behalf of public bodies and it was as part of this programme that Agile Cornwall was bored.

The report surveyed the companies who participated in the Agile programme to find out about their opinions of the programme and the changes which had resulted. They spoke to multiple people in most of the companies so it was pretty well grounded although it probably wouldn’t stand up to rigorous academic analysis. (If any academic out there would like to do a rigorous study then please get in touch.

Here are some highlights from the report:

• The Agile programme has met expectations for all companies who participated in the review. (I suspect one which was disappointed did not participate.)
• Businesses are finding they are more flexible and responsive to customers
• Money and customers
  • None of the business reported a decrease in turnover, profitability or customers since adopting Agile.
  • 7 companies reported increased turnover or revenue; 3 reporting the increase was significant
  • 6 reported increased profitability, 3 saying the increase was “significant”
  • 6 reported an increase in customer and/or business opportunities with 3 of the six saying the increase was significant

There are some downsides however:

• One company reported that it was now more difficult to hire because they could not find people with Agile skills.
• Small companies said it was difficult to find the time to get started on Agile

All of which makes me very happy.

There are also some very positive comments on the quality of the consultants used in the project, as I reported in my last blog it wasn’t all me. But, if I may be permitted to blow my own trumpet, I was there a lot, I worked with more companies more of the time than any other consultant. So I’ll be giving myself a pat on the back!

The report has, unfortunately, not been made public - although a summary version was given to attendees at Agile on the Beach. So if you would like to see the full report I suggest you go and knock on the door at Grown Cornwall / Oxford Innovation.

Agile creates jobs

Regular readers will now that over the last two years I’ve done a lot of work in Cornwall, I jokingly refer to “the Cornish Software Mines” (A case study and Light Touch Coaching to name two blog entries.) To date the most visible result of this work was the Agile on the Beach conference which is running again next week.

There is now a report out which has looked at the programme overall and concluded that the Agile Cornwall Programme created 50 jobs. That might not be a big deal for London but in Cornwall that is a big deal. Particularly since half the jobs were high paying.

So far I haven’t actually seen the report so I just have this report on the report to go on. I’m trying to get a copy as we speak.

I can’t take all the credit, I had help. The thanks need to go to:

• Oxford Innovation for creating and running the programme
• Michael Barritt who was the driving force while of Oxford Innovation and has since moved to Sullivan Cuff Software (one of the programme’s success stories)
• Jon Jagger who provided most of the technical coaching
• Benjamin Mitchell who coached several of the clients when I was fully stretched and gave intensive help to one of the more difficult clients
• Plus Kevllin Henney and Sebastian Bergmann who provided additional training, and Nancy Van Schooenderwoert who provided specialist embedded consulting

This report also looks it should help providing some evidence for Agile.

There is still time to book tickets for Agile on the Beach if you want to meet all these people and some of the people who benefitted.

Version 2 "The Rewrite" - don't do it! - never ever rewrite you system

“The second is the most dangerous system a man ever designs” Fred Brooks, 1975 & 1995

Brooks was talking about software designers, architects, but I think the statement holds true not only for all software developers but for the business people who commission replacement systems.

From time to time I come across a team who want to rewrite their system. Usually this is the technology team who are saying “The code is too complex, the thing was written by a jerk, She cannae take it Captain”.

Sometimes is the business side realising that their system looks out of date - Visual Basic 6 controls on Windows Vista, or turn of the century web style.

Either way there is a “Stop the world I want to rewrite” mentality. Sometimes the powers that be buy the argument, sometimes they bless a rewrite or, as one team called it a “Technical Improvement Programme” - TIP for short.

Lets me come to the point: this is always and everywhere a mistake. You should never, ever, rewrite a system just because the technology is old.

This is a mistake because customers don’t want to change. The may think that VB6 controls look odd but they know how to use it. Even if change would benefit them they are scared of it. Plus, they have their own business to run - or life to live - and changing systems is only going to get in the way.

Second, this is an utter waste. Statistics regularly show that as much as 80% of features in a typical software system are not used. So why implement five times more functionality than you need?

One company I worked with commission an cheap Indian company to rewrite an old PowerBuilder application in Java. They were told to make it identical. So they did, they spent a lot of time making Java look like PowerBuilder. They copy every feature, even the ones not used. It was buggy as hell and the users didn’t want it.

Third: the “just a rewrite” mentality leads people to under estimate the complexity involved and to overlook the opportunities for improvement. Why produce a second system which is identical to the first?

Finally, and worst of all: the world doesn’t stop, you can’t afford to stop and rewrite, nor can you afford to waste the opportunity to produce a really good system, one that is better than competitors.

Yes, I agree your current system might be at the end of its life and you might need to throw it away but the solution is not a rewrite. When this happens you have two options.

The first won’t satisfy every problem but is the least risk: refactor your way out of the problem.

Actively encourage programmers to refactor code, of course you need to embraced Test Driven Development at both the development and testing level first of all and start putting tests in. Thats not to say you cover it in tests first and then refactor, rather you start putting the tests in as you go. This is a bit of high-wire act without a safety net at first so I strongly recommend you get some help if you’ve not done it before.

This is real engineering, this is changing the engine oil while the engine is running.

That is the easy answer but it won’t work every time. Sometimes the changes are too much, say from VB6 to C#.

The second answer to develop a new product, a competitor product. One for which your current system/product is the competition. This development effort needs to be market lead and focus on what customers/users want the the current system doesn’t provide.

I’ll say it again: Market led. The marketing department, the product managers and/or business analysts should decide what goes in. Not the developers.

When companies rewrite they jump to the assumption: “The developers know what the current system does so they just need to copy it.” That is the biggest mistake.

Next, get into the market fast, release early release often. Test what you are building in the market, show your company real, usable progress. Remember, your competitor is the current system, if you can’t do better than it you want to know fast.

The objective is to build a system that your current customers want to use, one for which they are prepared to give up the system they have at the moment. Consider this a variation on Same Customer, Different Product pattern (available online or in the Business Patterns book).

Emphasis the things the new system does that the old one doesn’t. Don’t worry about all the features the old one has that you think you need to launch. You might need them in the end but nobody is going to buy your software because it does the same thing. Add them as you need them, you’ll find you need to bring across far fewer than you initially think.

Now I apologise to all the developers out there who are begging their managers to let them rewrite the system. Your code base may be terrible, the technology may be out of date but a development led rewrite is not the answer. Few companies can survive such a thing.

Selling scopeless contracts

The last two blog entries (“Scopeless contracts: the problem” and “Scopeless contracts: the solution”) have been working up to this one. Actually this is the blog entry I set out to write a few weeks ago but felt I had to explain why.

Even if you accept that leaving scope out of a contract for work is the right thing to do you still need to sell it to your customers. This is, perhaps, the real problem. However this is not always the case. Some clients will immediately recognise the problem, or might not wish to sign a big up front deal.

So, faced with a customer who wants to know “How much will it cost to do X ?” what should you say? Or rather, what can you emphasis to persuade them that you have a better idea?

(Note, on anything other than a trivial piece of work X will not be one thing but many, call them X1, X2, X3, … Xn.)

1. Flexibility: scope less contracts build in flexibility, if the client wants Y instead of, or in addition to X then thats OK. No need to scrap this contract and start again.
2. Work can go away: Suppose X3 become pointless, then it can just go away without any trouble. As opposed to the UK Government Connecting for Health project [[link]] were actually cancelling the project is more expensive than letting it finish.
3. Room for innovation: if you sign a big contract and later think of a bright idea then something has to give. Scopeless contracts allow for this because feature negotiation is ongoing.
4. Bounded by money: Rather than being bounded by scope contracts are bounded by the money available, or sometimes the time available. This forces the creators to work within a very real boundary rather than poorly defined scope and time estimates.
5. Governance control: bounding the contract in money provides for real governance rather than the governance by proxy that scope, features and line items provide. When the money runs out the work stops. (This of course assumes the team are making regular deliverables, if they aren’t you have even bigger problems.)
6. Inventive to reduce work: putting 2, 4 and 5 together you get an incentive to reduce the work to be done rather than inflate. In traditional work scope is problem because removing it creates contract problems and loss of face, but more importantly: there is little incentive to remove work until difficulties hit.
7. Reduce delay & waste, keep options open: All the time spent writing and negotiating scope delays the time when you actually start delivering something. If things change, and the longer you spend writing the scope the more likely things will change, then the more of that work which will be wasted. And the more you think you know what you want the more you constrain the options for producing the optimal solution. Selling without scope reduces this work because scope is derived as you go within the process and only enough is produced as is needed.

You can, and people do, do all these things within traditional scope based contracts but there is additional work to do to change the contract. And, more importantly, it starts the contract with an illusion, a myth: the idea that one day things might be done. Which gives us:

1. Traditional work relies on a scope myth “scope won’t change”. It does but we pretend it was because something was wrong. The result is we spend time and money managing the myth: managers need to revise contracts, staff change boards, write complex governance reports, etc. etc. In other words: because managing scope is doomed to failure the thing we call “scope” is a chimera which we waste time and money on “managing”.

Despite three blog entries I still don’t feel I’ve said what I wanted to say. This is a topic I need to return to….

Scopeless contract: the solution

In my last blog entry, “Scopeless contracts: the problem” I rehashed all the reasons why including scope in a contract for delivery is a bad idea. So having rubbished the most common way of working I had better explain what I suggest you do….

You have two choices.

Option 1: Set up a ‘Business as Usual’ contact.

The client nominates work to be done and the development team do it. Simple. This is the way most organisations work “business as usual” but once the idea of a project enters the scene people expect the project to have an end date, in order to have an end date they imagine that some amount of work needs to be done.

You can still have an end date but instead of measuring progress against “work to be done” (he scope, the backlog) measure progress against the value added.

After all, software is never finished. If you ever run out of requests for a software product it means nobody is using it. However it might not make sense to do all the work that is requested.

How do you size the team? You don’t, you start small. You start with just the money you can afford to loose. After a while, say three months you review what is bring done and decide whether you wish to go faster (expand) or slower (contract), or perhaps stop altogether.

(Another reason to start small is to reduce the effect of Conway’s Law: if you start big you will get a big programme of work, if you start small you might just come up with a small solution.)

Option 2: Set a Goal, work to the goal

If option 1 isn’t governed enough for you then set an overarching goal. Every three months (or what ever time you decide) ask yourself: are we making progress towards the goal?

You might even want some people on the team measure progress towards the goal (these would normally be Business Analysts). All the work you do should be tested against the question “Does this move us towards our goal?”

If you want more structure use my 10 Step Requirements model.

Yes, it really is that simple.

Drop the backlog. Set a goal. Work business as usual. 1, 2, 3.

The thing is, when you contract with someone to build you software its not like asking them to build you a house. You are buying a service not a thing. If you could buy a thing you probably would have done already. Once you make that mindset shift it all makes sense.

The only complication is, well there are two really: first you have to accept the project model is broken so you need to make more strategic decisions. If you need to keep projects for accounting purposes but projects should have nothing to do with development.

Second you need to do this on a rolling basis, review it every three months.

For most organisations the biggest problem is that they have come to believe the project myth, and they believe it all the way up the hierarchy. Until the people at the top really understand that IT doesn’t fit into the project model its difficult to change.

Scopeless contacts: the problem

I’ve discussed Agile contract models before and I’ve agued against points based contracts (‘Points based contracts? Just say No’) but the more I think about client-supplier software development contracts the more I think including any scope in the contract is asking for trouble.

Before I explain let me say: Yes I know this isn’t going to be possible either with your clients or your own sales folk, I’ll follow up this blog entry with another setting out what you should sell instead.

Now I’m using the word scope, although I hate the word it is the word most people recognise. I mean: “the work which will be done.” Call it a requirements document, a PRD, a product backlog or anything else. Its a shopping list of things you think you want.

Right now, here is why I think fixing scope inside a contract is a bad idea:

Scope is always unknown: you may think you know it but you don’t because things change, learning occurs (on all sides), when someone see something they change their mind about what it is they want, or what they ask for, or what they want next.

I’ve been over this ground again and again, first in Why Do Requirements Change (2004), again in Changing Software Development (2008) and more recently in Dear Customer (2012).

And the bigger your project is the more your requirements will change and grow. Capers Jones gives the industry average as 2% per calendar month for large projects. If you’ve got a project/programme that is up and running you can go and measure it yourself. One team I know saw their product backlog increase by 50% between January and July.

Someone said to me recently: “Surely the team has got better at this now, they’ve learnt.” Well yes, they will have got better, that might take the edge off, might rule out some human factors but the world hasn’t stopped.

The World is Changing: lets suppose for a moment you are running a big project, say its been running 2 years and you think it has another 2 years to run. Cast your mind back 2 years, what was the world like in July 2010 ?

First some context: David Cameron has been British Prime Minister for 2 months, Osama bin Laden was alive and well, few people outside Japan had heard of Fukushima, Deepwater Horizon had just been capped and oil was under $50 a barrel.

• The first generation iPad had been on sale for 3 months, it was far from obvious that Apps were about to take over the world
• Nokia were still pushing Symbian phones and were still the biggest mobile phone seller and Android was in version 2.2 (Froyo)
• JP Morgan was fined £33m for failing to protect client money
• Greek debt was downgraded to junk in April
• Barclays were fixing Libor

Go back a little further: Windows 7 was release in October 2009. Further still: the iPhone was released in June 2007 and the AppStore in July 2008. Not that long ago.

Did any of those events effect your business?

Did any of those events, or countless others I haven’t mentioned effect what your business wanted?

Now ask yourself: what could happen in the next two years, to 2014, that would change what your business, your customers want?

President Romney? Grexit? Finexit?

Scope should reduce as well as increase: when you fix scope you also make it difficult for scope to reduce. Requirements should be reducing as much as increasing.

Marty Cagan recently blogged about “The Opportunity Backlog”. Your backlog is not a list of things that will be done, it is a list of things which could be done, and you should do as little of it as possible (its expensive), instead focus on the opportunities which deliver the most.

You human, you make mistakes: well maybe not you personally but the people who work for you.

There are mistakes in what you ask for, mistakes in what you build and mistakes in how people use it. Sometimes fixing a mistake is more important than doing something new.

Second system effect: there is a blog entry waiting to be written about version 2 projects, for now lets stick with what Fred Brooks said in 1975: “The second is the most dangerous system a man ever designs. ... The general tendency is to over-design the second system, using all the ideas and frills that were cautiously side-tracked on the first one.”

Whether you are designing from a technical, user or business perspective you suffer from second system effect.

OK, am making my point?

Since requirements can and will change there is not point in writing them into a contract.

This point is doubly dangerous if you attached estimates and timelines to the contract because you will be wrong again. Human’s Can’t Estimate.

This is bad for the supplier who will sooner or later have to explain what is going on to the customer and its bad for customers who don’t get systems that work.