allan's blog - Agile, Lean, Patterns

The Meter: Tom Giib's greatest invention

2012-01-25T07:53:00.000Z

If you’ve ever found the time to read Tom Gilb’s work - and I’m thinking specifically of Competitive Engineering here - you’ll know that his work is packed full of good ideas. So many god ideas in fact that it can be hard to read his book - something I think he admits himself in the opening pages of Competitive Engineering.

One of his ideas is that of a Meter. This alway confuses me a little, still now...

You: A Metre you say?
Me: No, not a metric unit of distance, a Meter, as in a Gas Meter
You: What, how can a Gas Meter help software teams?
Me: No, the idea of measuring things, scrum that, say a Ruler
You: A ruler? Centimeters and inches?
Me: No, A yard stick
You: Distance again?
Me: No, A unit of measurement and a tool for measuring
You: Measuring what?
Me: Yes, Measuring what, that’s Tom’s great invention, we need to define measuring systems

Tom’s great invention is: define a mechanism for measuring the thing which is important to you.

OK, Tom takes the idea further with Plangauge he says we should define the meter, the starting value and the desired value. But forget values, its the measurement unit itself which I think is too frequently overlooked and worth more attention.

The thing is, the unit of measurement, and the means of measuring it is implicit in a lot of work: its in Kaplan & Norton’s Balanced Scorecard, its in Beyond Budgeting, its in Lean Start-Up and elsewhere.

But, as far as I know, only Tom calls the Meter out as an idea in its own right. In fact, I suspect Tom might not have realised the significance of this.

Once you know what the right Meter is it becomes - relatively - easy to measure things. So you can measure the starting point, you can set a target, and you can measure how far you have got.

We are surrounded my measurement units and meters which already exist

Currency: Pounds, Dollars, Euros, etc. etc. - and the associated profit, revenue, costs, etc. etc.
Accounts: Profit, revenue, costs
Website hits: visitors, unique hits, conversions, etc. etc.
Time: second, minutes, days....
Distance: inches, miles, centimetres, meters and kilometres
Location: Latitude and Longitude
Weight:.....

And we tend to use these measuring units again and their associated meters again and again.

New meters are not so common but they occur, e.g. Eric Ries in Lean Start-Up introduces a new meter: Cohort Analysis.

But - big but - if we use inappropriate measuring units and meters it gets hard, and it potentially gets wrong. (Another on of Eric’s points.)

Again, as far as I know, Tom is the only authority who says: define your own unit and define your own way of measuring it.

Now, as far as I can tell Tom is a master of this, the rest of us... well most people don’t get it and try and use pre-defined measurements and Meters, even those of us who “get it” struggle with it.

So, we need to start inventing new units of measurement and new ways of measuring things. We need to get good at both inventing these things and sharing them. History shows many of these measuring units are useful to other people.

Second, as more of these units are recognised then we will have more mechanisms - and language - with which to implement both our own tools and the tools like Balanced Score Card and Lean Startup.

Corporate Psychopathy

2012-01-19T19:37:00.000Z

The Oxford Dictionary of English on my Kindle defines:

Psychopathy n. mental illness or disorder.

Psychopathic is more interesting:

Psychopathic adj. suffering from or constituting a chronic mental disorder with abnormal or violent social behaviour

There is a reoccurring pattern of behaviour I have observed in corporate environments which I have taken to calling Corporate Psychopathy, maybe you’ll recognise it if I describe it.

A company brings together a team to do some work. At some date they decide the work is “done” and disband the team. They often call this work “a project” - projects by their definition have a start date and an end date, and involve a temporary organisation unit; i.e. a team is brought together at the start, they do some work, then the team is disbanded.

I think this state of affairs is crazy. So crazy I believe it is abnormal social behaviour.

A team is a social unit. When was the last time you dumped all your friends and started to forma a new group?

Think of the high performing teams you know: sports teams or successful political parties. They may change players from time to time, even key players, but the winning team usually stays together from one victory to the next.

By contrast think of the teams who are less successful. How often to do they change personnel How often do they work together?

Anyone who has studies team dynamics will probably have come across Buckman’s “Storming, Forming, Norming and Performing” model. The idea is that when you bring a team together they pass through each phase as they learn to work effectively together.

If you have an effective team - one that has passed through storming, forming and norming and is now performing - why would you break them up? Let alone break up one team and create a new team which has to pass through three phases to get to the productive fourth?

It doesn’t make sense.

Just to make it worse when corporations break up teams many of the people leave for ever. Companies which use contract staff release the contractors who will go and work somewhere else. Even permanent staff may well find themselves release to a pool, or to previous roles and the same people are unlikely to work together again. Knowledge evaporates in the process.

If that isn’t enough think of the admin overhead in managing all this.

It also makes the start of a work incredibly difficult because you don’t know how effective a team will be or when they will break through into Performing. And if you are following a development method where you measure capacity (velocity) and benchmark against past performance you can’t use this method for the first few weeks.

Its psychopathic: anti-social, performance reducing, expensive, undermines forecasting and increases risk.

Put it another way: It is plain stupid.

My solution is keep the performing team together as much as possible and bring the work to the team. Have the team move from one piece of work to another together.

This parallels the way we timebox work. In timeboxing we fit the work to the time allowed - a two week sprint or a quarterly release. Similarly we should bring the work to the team.

Of course teams might not have very clear cut switch over dates. Perhaps they have to ramp-down one stream of work while ramping up another. More complex to manage.

Or perhaps a performing team splits into two - like cell division. One part continues working on the first piece of work while the second part starts something new. This way the team can carry the culture from one piece of work to another.

To do this is might be necessary to let the team grow organically, split the team, then grow both sides organically.

The point is: there are ways of managing this. You don’t have to split teams up and form them all anew.

Dialogue Sheets on show at Skills Matter - 13 February

2012-01-04T14:57:00.000Z

Quick follow up to the last entry about Dialogue Sheets. I’ll be doing a evening session at Skills Matter on the sheets next month - 13 February, “Dialogue Sheets a New Tool for Retrospectives”. The event is free but registration is required.

Retrospective Dialogue Sheets in InfoQ

2012-01-03T13:32:00.001Z

Happy new year! - my opening missive for the new year is elsewhere...

I’ve written more about Retrospective Dialogue Sheets for this weeks InfoQ - “Dialogue Sheets: A new tool for retrospectives.”

This piece summarises many of the findings I discussed a few weeks ago in Dialogue Sheets feedback #1 and Dialogue Sheets feedback #2 blog entries.

A footnote to a footnote, I discussed dialog sheet print-on-demand a few weeks ago. After that I was able to reduce the price of printed sheets. As I suspected at the time the price reduction has made no difference to the number of people using the service. I therefore conclude that it is not the price of the sheets but rather the fact that there is a charge at all the creates the block.

News from the Cornish Software Mines

2011-12-18T15:46:00.000Z

My work in Cornwall, helping companies adopt Agile approaches to software development, is set to continue into the new year - so more jokey tweets about the Cornish Software Mines. I hope to write more about this work - the benefits and what we’re learning in future.

Right now there are two pieces of news I’d like to share:

At the start of December the author of PHPUnit, Sebastian Bergmann, joined in me in Cornwall to provide training and coaching to teams using PHP. Oxford Innovation (who run the Grow Cornwall programme) have put out a press release about this.
Oxford Innovation have also produced a Video Case Study of our work with Research Instruments in Falmouth. RI are a great success story and the case study is worth the two and a half minutes it takes to watch.

I first met the team at RI in October 2010 when I provided Agile foundations training to development team. Since then I’ve been in many times to provide follow up coaching.

About six month after the Agile foundations course Jon Jagger delivered a Test Driven Development (TDD) in C# course for the team and has also been visiting to provide follow up coaching. All in all a great success, as the video shows.

Xanpan, ToC, Guilt, plagiarism and sorry

2011-12-14T15:27:00.000Z

When I was doing my Masters degree I had it hammered into me: credit your sources, don’t plagiarise. To this day I take pride in trying to credit the source of my thoughts, back-up my ideas with research and apportion credit. I won’t always succeed but I think I do a reasonable job. Heck, I’ll go further, I think I do a better job than most.

So image how I felt after my last blog post, about Xanpan, when Benjamin Mitchell tweeted:

“@allankellynet I'm unclear why you don't mention Theory of Constraints 5 Focussing Steps. What's your thinking on this?”

I was mortified because, well, he was right. My blog post discussing how to improve throughput on a Xanpan board was pretty much a Theory of Constraints description.

I had plagiarised. I had failed to credit. And I had written something that other people have written before - probably better than me.

I apologise, I give due credit: these ideas are inspired by the Theory of Constraints - or ToC for short.

So how did I come to make this mistake?

Well, I think the answer is in two parts:

First, I have ToC thinking to heart and it now seems so obvious I’d forgotten it needed crediting

Second, the blog entry was written a day after I’d had a similar conversation with a client. ToC hadn’t been mentioned in that conversation so it wasn’t mentioned here.

So maybe the plagiarism occurred in my conversation with the client. Well yes, but.... my first point still applied and second I tend to avoid crediting with clients. Now I’d better explain that second point some more.

In conversation with clients I tend to avoid saying things like “This comes from....” or “I suggest you go and read....”. Yes I do say those things but every time I say it I believe there are two ways for a client to take it: they may take it as “good, Allan is giving me the original source” or they may take it as “Hmmm, this consultant is name dropping, doesn’t have ideas himself and is very academic.” (Plus, they are paying for my time so perhaps I should just explain things to them.)

In short I don’t think it always helps.

I also remember all those Professors at Business School who uttered “Smith Smith & Smith, 1904” (or whatever) at the end of every sentence. And those turgid academic journal articles where every paragraph was credited and thus become very very boring to read.

I could go on but you get the message.

Now, one more thing, this blog entry is really really defensive. I feel a little attacked by Benjamin - I’m sure he didn’t mean to attack me, its just in 140 characters people come to the point. Maybe he feels attacked by this, don’t know we could get deeper and deeper.

For anyone still reading, I’m sorry for droning on. However I think this does raise an interesting point: in trying to explain things, in writing and in conversation making things accessible and easy to understand is - in my humble opinion - at odds with academic integrity and full disclosure.

As someone who makes a living doing this I need to be more aware of it than most.

Xanpan: refining the process

2011-12-12T15:48:00.008Z

In my last blog entry I discussed a Xanpan board layout which allows for planned and unplanned work to co-exist in one workflow. Here is a picture of the board I had just put together with a team in Cornwall:

This might be the perfect board layout for the team for ever. More likely it can be improved over time - in the words of Kevlin Henney, it is a Stable Intermediary Form.

I think of Xanpan boards like the Pompido centre in Paris, they externalise the plumbing. In this case the workflow and work in progress. You can see the stuff that is normally hidden inside.

My rules are quite straight forward for Xanpan boards:

Stage 1:

Model the current workflow on the board: avoid changing it too much to “how it should be” but you will probably need to modify the workflow a little in order to be able to model it.
Make sure you talk through various scenarios for work to make sure you have a workflow that will, well, work. As I said in the original Xanpan blog post columns normally come in pairs: queue then work.
Operate the board for at least one iteration (several weeks) and learn from it.
Reflect and refine the board: in doing so you will modify the workflow, this should be an improvement.
Operate the board some more an start gathering data, qualitative and quantitative about the flow of work.

This alone might improve your teams performance: you have visibility of the work in progress, you can reason about it, people are better informed, you will make better decisions. But, this will not solve or your problems, nor is it the end state, it is just the beginning.

Stage 2: (Assuming you want to improve the throughput)

Think of the board as a pipeline: there are two ways to improve throughput, make the pipe bigger or make things move through the pipe faster.

First seek to move things through the pipe faster:

Look at the blocks: why are they blocking or impeding? What can you do about removing them, not just on this occasion but permanently
Where are the queues building up? Can you rebalance the board, or rather how your people (and possibly other resources) are deployed to even this out

Continuing reviewing the board and looking for improvements for ever. But if the time comes when you want a bigger pipe:

Add people to the team gradually over time: adding people slows down the existing team, the bigger the team the smaller the proportional slow down, above all else: avoid foie gras recruitment. Again this is likely to be a long term activity.

Finally:

Look to level workflow: reduce the peaks and move work to the lows.
Look to increase the value of the items moving through the pipeline - more and better business analysis and product management

Those are the rules of thumb. Of course there are exceptions, of course there are other issues, of course they don’t cover every eventuality.

Xanpan: planned and unplanned work

2011-12-09T16:03:00.001Z

I’m on my way back from another visit to the Cornish Software Mines. I’ve been with four different companies this week, laid the first plans for Agile on the Beach 2012 conference and got to know Sebastian Bergmann who has been visiting the mines this week working with the PHP teams.

One of these companies is new to the Agile programme, we did training for the entire company last month and this week I was there to get them started for real. This meant we had to design their board layout and talk about workflow. Once again I found myself advocating Xanpan, the cross between Extreme Programming and Kanban.

The board layout - effectively workflow - is on which reoccurs in Xanpan systems and I would like to discuss here. (I’ll explain the layout here and in a second blog entry I’ll post a picture and talk about the future.)

The problem companies/teams face is: they have planned and unplanned work to do. Typically this is some project they are working to deliver but they need to work on support issues - bug fixes - too. I say that, but it is not just support issues, sometimes it is clients who shouts very loud, or managers who can’t say no, or customers who refuse to wait a few days - indeed sometimes to do so would be the wrong thing to do.

The Scrum answer to this is to stamp on the unplanned work. The team are locked - remember commitment and Sprint-goal. This unplanned work is a problem the Scrum Master should stop - or declare Abnormal Termination of Sprint. Particularly in small companies this isn’t a very useful answer, the truth is, things are more complicated than that, planet-Scrum is too simple.

The ultimate solution is to look at the reason these unplanned items are arising and try to address the root course. In time it might well be possible to do that but right now we don’t want to block the team’s transition. Even in the long term this might be the way of the world for such a team.

The Kanban answer would be to go in the other direction: run all work as “unplanned” and just restock the to-do queue as needed. However this robs the team of the rhythmic certainty of a Sprint. Many of the teams I see need to move away from seat-of-the-pants decision making and knee-jerk reactions. Adopting regular iterations builds in decision making points and helps individuals understand their roles.

So: Xanpan, keep iterations/sprints but allow for unplanned work. To do this we design a board which begins with the three columns:

Planned
Unplanned
Prioritised (usually this queue is limited)

The next column is usually something like:

WIP: In development (usually WIP limited)

The team start Springing as usual: they hold a planning meeting, review Blue Cards (Business requests, User Stories) and if necessary break them down to White Cards (tasks). These then populate the Planned column. These blues are probably drawn from a traditional product backlog.

At any time work can be added to the Unplanned column. The column is unlimited.

A few minutes before the morning stand-up meeting around the board the person with authority populated the Prioritised column reviews and populates it. This could be the Project Manager, Business Analyst, Team Leader or someone else, the role and title doesn’t matter, what is important is that this person has the authority to do this. On one team this was the joint responsibility of the Project Manager and Business Analyst.

This person looks at the work still in the Prioritised queue, and they review the Planned Work and Unplanned work. From these sources they decide the priorities for today. When the team gather they can see what the priorities are. For the sake of simplicity it normally makes sense for the Prioritised queue to be limited and for the priorities to be ordered: 1, 2, 3, up to the limit.

If need be priorities can be changed during the day: maybe something even more urgent arrives - or goes away, maybe the prioritised queue empties, or some such.

Effort estimates are normally assigned to planned work during the planning meeting. For unplanned work some teams don’t both adding estimates, some added a quick estimate from the person who picks up the card when they pick it up and some put an effort estimate on when it is complete.

Importantly, the original source (planned or unplanned) is recorded on the card - maybe a different colour card, maybe a dot, a word, whatever. At the end of the Spring the team can review how much planned and how much unplanned work was undertaken.

Thats it really: planned, unplanned and prioritised. The first two are effectively queues for the third. Somebody, or bodies, are responsible for balancing priorities.

It is possible that this workflow/board layout is actually a transitional layout. After running with this flow for a while there will be data on how much unplanned versus planned work actually occurs.

Dialogue sheets: more sheets & pricing

2011-12-01T10:57:00.000Z

A quick follow up to my last two entries about Dialogue Sheets (Retrospective Dialogue Sheets: feedback & updates 1 of 2 and Retrospective Dialogue Sheets: feedback & updates 2 of 2).

Several people who provided feedback said they found the cost of the sheets to be off putting. Since the soft copies are fee the true cost of the sheets depends on where you live and what access you have to large printers, I have little control over that. However, I did set up a print-on-demand service for getting pre-printed sheets.

I’ve looked over the settings and I’ve managed to reduce the price on most of the sheets - while I was there I’ve also uploaded some new sheets - a general discussion sheet, a Sprint level retrospective sheet and a Kick-Off sheet.

The cheapest sheet is now priced at $13.75, or £8.74, €10.20. Since the printer is based in the US the exact price depends on your currency.

Unfortunately, shipping needs to be added to that. Within the US the cheapest sheet should total about $30 - if you order multiple sheets the shipping costs doesn’t rise proportionately so it gets cheaper per sheet.

Apologies to everyone in Europe - myself included, or Asia, or elsewhere - shipping something across the Atlantic costs more. Again, if you buy several sheets at once its cheaper - and is what I do, I have a stack in my office.

I would be interested to know from people who do find cost a barrier just how much they could pay. If I know the restraints I can look into other options.

However, the skeptic in me thinks that the actual price is not important: $13 might as well be $3, $30 or $300. It is the fact that any money needs to be spent that is the barrier.

Which is silly really because sitting half a dozen software developers in a room for an hour isn’t cheap: say $100,000 per developer per year means $50 x 6 = $300, the cost of the sheet is trivial. (This might be why so few teams do retrospectives in the first place.)

Look at it another way: A dialogue sheet costs about $30, which means you save $50 (at least) it would have cost for a facilitator.

Cost arguments always loose so look at it another way: if as a result of the exercise the team save more than one hour it has paid for itself.

As I said, if you have any information that can help me find the right answer please e-mail me.

Retrospective Dialogue Sheets: feedback & updates 2 of 2

2011-11-28T15:46:00.001Z

This is the second of two blog entries about the use of dialogue sheets. The first entry contain some simple statistics and findings from recent experiences and feedback from users. This entries discusses a few findings which deserve longer comment.

Those who use the sheets consistently report that they engage more team members (everyone gets to speak). Several people report that quieter team members are more likely to contribute in a dialogue retrospective than in a regular retrospective.

Another common finding is that energy levels are higher, there is more discussion and removing the facilitator changes the focus of the retrospective.

These are the results I hoped for when I created the sheets. Yet, to some degree I think these findings are simply the result of changing the format - doing something different. Perhaps a team that uses dialogue sheets for every retrospective will find that in time some of the energy is lost.

Right now I don’t know, I don’t have enough not enough data. However, I have always said these are one technique and I encourage you to use them as one for several retrospective approaches.

A few people report that they don’t like the idea of removing the facilitator. This dislike has stopped them from trying the dialogue sheet. While I understand this fear I hope they will try them all the same. After all, it is the nature of Agile to experiment.

My advice: try a dialogue retrospectives and see what happens, then decide if you want to try again.

One person did report that they felt that without a facilitator some issues were not explored as they might be. I fully understand this finding although it does cause me to ask: were some areas explored which might not have been explored? Again this could be a argument for having some facilitator-less retrospectives and some with.

There is a strange case reported from a company I will not name. Here the management vetoed facilitator-less retrospective. My correspondent doesn’t understand why and I am a little baffled. This gets curious when you know the name of the company concerned. Someone else from the same company replied independently (different country) and has had success with the sheets. I’ve put them in touch and hope this will be resolved.

Interesting, a previous correspondent told me that they kept a facilitator even when using the retrospective dialogue sheet. This hybrid could address both management concerns and the fear of missing topics. I think there is more experimentation to be done here, perhaps using the dialogue sheet to start the retrospectives and then having a facilitator follow up. Or using a dialogue sheet retrospective to identify issues and in following retrospectives deal with them in detail.

Finally, two unexpected findings which carry lessons for those doing regular retrospectives.

All the retrospective sheets include Norm Kerth’s Prime Directive:

“Regardless of what we discover, we must understand and truly believe that everyone did the best job he or she could, given what was known at the time, his or her skills and abilities, the resources available, and the situation at hand.”

To retrospective facilitators this is a holy commandment, something we hold close to our hearts. Yet left alone it is interesting to see how groups react to it.

While one corespondent reported “Kerth's prime directive is also in our company's culture, therefore there is no need to remind people of it, it comes natural for us.” Another reported:
“One group poked fun at the "prime directive", but the other didn’t”.

My own experience is similar. Some teams - usually those who are familiar with the directive - just accept it. One team commented that although they knew it they tended to forget it and it was a good reminder. However, several teams have agonised about the directive, they stall at what should be a routine reminder. Sometimes it is obvious that team members implicitly do blame individuals for problems they face.

I think this behaviour says something about the company, the culture and the teams familiarity with conducting retrospectives. It is, perhaps, the one place where I would like a facilitator on hand. However intervening at such an early stage in the sheet would compromise the approach.

Perhaps the lesson here is that the dialogue sheets work best when teams have trust, and accept the Prime Directive. But then, isn’t that true for all retrospectives?

The other lesson is: while retrospective facilitators hold Kerth’s directive close to their heart the same isn’t true for many others.

The second unexpected finding came at a company in the North West of England. I visited and observed a retrospective using dialogue sheets. There were over a dozen people so we divided into two groups each with a retrospective dialogue sheet.

Unusually the two teams used different sheets - I think one had a T1 and one had a T3, so a similar format but slightly different questions. The first, finding here was that using different dialogue sheets not only does work but can help add insights.

The second, much more significant and unexpected finding was a result of who joined each group. I thought I had divided the groups randomly but as it happens the majority in one group were developers and the majority in the other group were Business Analysts.

At the end of the exercise one of the Business Analysts commented: In a normal retrospective the developers dominate (there are more of them) and so the retrospective usually talks about issues of concern to developers, the issues of concern to BAs are not usually discussed.

It sounds obvious once you see it: the largest group will tend to dominate a retrospective.

However, separate them out, as we did here, gives both groups an opportunity to talk about their concerns. Now the really interesting bit.

The two groups were retrospecting the same time period, however they approached it from different points of view. Some of their conclusions were different but some were the same. To my thinking, when different people, coming from different starting points come to the same conclusions, then those conclusions have added weight.

Overall then: Retrospective Dialogue Sheets are a success.

Retrospective Dialogue Sheets: feedback & updates 1 of 2

2011-11-28T15:39:00.001Z

Every couple of months I e-mail everyone who has downloaded one or more of my Dialogue Sheets to get some feedback. Getting feedback is why I make people register to download a Dialogue Sheet - sorry, I know some people don’t like doing this, and I know some people fake their e-mail addresses but unless I do this I get very little feedback.

I’ve blogged about my Retrospective Dialogue Sheets before but I thought this would be a good opportunity to update readers with some feedback and findings. This is the first of two posts on this subject, the second post will examine a few findings in more depth.

First simple facts and comments:

40% of people who download Retrospective Dialogue Sheets do retrospectives rarely or at best every six months.
Many who download them and reply to my requests for feedback say “Sorry we haven’t used them” and I guess that many of those who don’t reply haven’t used them either. Given that I expect those who are interested in retrospectives will download, and given that downloaders are measured in hundreds, not thousands, this indicates that actually doing retrospectives is rare.
46% of downloaders do them once a month or more frequently, while 15% of downloaders classify themselves as Retrospective Facilitators and perform any retrospectives.
25% of downloaders are in the USA, 9.5% in the UK, just ahead of Germany (9.09%). Otherwise only India (5.79%) tops 5%. Though Holland, Canada and France all score 4.5%. I don’t find this surprising given the spread of software development internationally and the spread of dialogue sheet publicity.

A few people find the sheets too complicated, a few people find them too simple. I had hoped that a variety of sheets would address both ends of the spectrum: the lesson for me is to create some even simpler sheets.
A couple of people have reported that they don’t feel their team is open enough to use the sheets. My guess is this will present difficulties for any type of retrospective.
A few people say the sheets are too expensive to print: the print on demand services charges about $40 dollars, the expense is in the postage rather than the printing, if you order a lot the price per sheet is a lot lower. (I must find a European print on demand provider.). However 40% of people say that they have a suitable printer, this surprises me a bit, I never expected large printers were so common.
To my surprise a couple of people have said there is not enough space to write on the sheets! I will create some new sheets with fewer questions and more space.
A few replies say that the teams are now using the sheets regularly, and in some cases they have spread from Agile to non-Agile teams.

As I said, one common response is that downloaders have “not had a chance to use the sheets yet.” On the whole I assumed this was down simply not having a retrospective. However a few correspondents have hinted that their company has a prescribed retrospective process. This worries me, if the retrospective approach is prescribed in such detail I wonder how much else is prescribed and how much opportunity there is to change things.

I’ve also been asked if it is possible to customise the sheets. The short answer is No, because they are PDFs its kind of difficult - although I’m sure you could with the right tools. Still there is no reason why not to create your own - a few correspondents say they are thinking of this. (Anyone thinking of creating their own sheets might find this guide for teachers creating dialogue sheets useful.)

Finally for this entry, I’m also hoping to provide translations of the sheets shortly. I have offers of translations into Spanish, German and Finnish - plus I hope to persuade a friendly Russian to help. Main problem at the moment is getting the raw files out of OmniGraffle into Visio.

You are not Steve Jobs (and don't try to be him)

2011-11-18T17:34:00.001Z

Many people better than I have commented on the passing of Steve Jobs a few weeks ago, I too was sorry to see him go. He touched my life (indirectly): I’m writing this on a Mac, we have an iPad in the house, I might not like iPhones but my new Android phone clearly owes a lot to Jobs vision (i.e. it is a copy).

Reading, watching and listening to the many obituaries about Jobs I started to get worried. My concern is simply that Jobs was not a good role model. His phenomenal success guarantees that more than a few people will attempt to emulate him, in so doing they will sow the seeds of their own failure and make the lives of their employees miserable in doing so.

Here are a few examples.

Jobs was a perfectionist: products didn’t get launched unless he approved of them. However, being a perfectionist is something of a luxury, Apple existed, had revenue, had a brand. If you are a start-up perfection is just too expense.

Jobs started the MacIntosh project because he had Lisa taken from him: MacIntosh and Lisa could only exist because Apple had revenue from the Apple II line.

The first Macs were far from perfect; the team undoubtedly learned from Lisa - what we might call a pivot in Lean Start-Up terms but not quite that clean cut.

Jobs demanded the first Macs would work in 64k, the team knew it needed 128k and Jobs, late in the day gave in. Secretly the team engineered the Mac to accept 512k which is what it really needed.

The moral of the story: there is more to it than being a perfectionist. Even perfectionists need to compromise sometimes. Surrounding yourself with good people helps.

More recently the original iPhone was launched as a 2G GSM phone, without features like cut and paste. It seems incredible now. Apple launched an under-spec phone and refined it in the market. Which leads us to....

Jobs would spurn products/employees who he didn’t think were up to scratch: if employees are loyal to the company and to you, and if you have a deep talent pool, and (perhaps) the stock-options are worth a lot you might get away with this. Otherwise its a sure fire route to staff turn-over.

Jobs may have been a perfectionist but, it seems to me, he wasn’t a perfectionist about quantity of features, quite the opposite. Apple products are simple because they lack so much, once launched they are refined and elaborated in the market.

Jobs didn’t use customer understanding or focus groups to tell him what the public needed. He was a visionary who just knew. This is perhaps my greatest fear: to read some of the obituaries is to condemn the whole discipline of Product Management to the dustbin in favour of senior managers who know what is right for customers - a disease too many companies suffer from.

This argument doesn’t stand up. Jobs was an, perhaps unique, visionary who did understand what customers wanted. He had an ability to see what technology could do and how people could use it. Few of us are blessed with this ability. Thus we have the discipline of Product Management to help us.

Jobs may have shunned market and customer research but his Product teams didn’t. In some cases Apple conducted internal, closed, demonstrations of new products; they used their own staff for research. And, as I just said: Apple launched products and rapidly refined them in the market.

Apple also had many products which were not hits, with and without Jobs: Apple III, Lisa, Newton, Mac portable, Pippin, Apple TV, A/UX, Pink, MobileMe - the iPod took several years to break through and even the Mac came close to failure on several occasions - and iTunes is probably the worst piece of Mac software on the market.

OK, Jobs was not even at the company when several of these products launched but he sowed the seeds and above all it was his company, a culture he had created. And while he wasn’t at Apple he founded NeXT which wasn’t exactly a success either.

But look again: Apple and Jobs were prepared to try things. Some worked, some didn’t. Some took a long time to come to success. Some failed but somewhere, in Jobs head, deep in the bowls of Apple, lessons were learned for future products. We will never know just how much the Mac team learned from Lisa, or how much MacOS X learned from Pink, or even how much the iPad learned from Newton.

Apple and Jobs could do this because they had deep pockets, they had revenue.

Lets put this simply: as much as you might like to think you can emulate Steve Jobs you probably can’t. What worked for Jobs worked for him because he was unique in vision, timing and experience. Just because Jobs could ignore volumes of business advice, armies of experts and accepted wisdom doesn’t mean you can.

The lesson I would take from Jobs and Apple is not about perfectionism, design, arrogance, or the super-CEOs but about the willingness to try something simple, iterate and refine in the market.

Me @ Agile on the Beach

2011-11-13T11:48:00.001Z

Many of the sessions at Agile on the Beach were recorded and are now available online.

You can find my “Objective Agility” presentation here.

I have a bit part in the The Agile Cornwall case study which is also online. Actually, this video, or rather these videos, are the best version of the Agile Cornwall case study. Although myself and Mike Barritt have given versions of this presentation elsewhere (and probably will do again in time) this version is the best for two reasons.

First, these videos include people on the receiving end of the programme - Research Instruments and Sullivan Cuff Software. They tell about their experiences moving to Agile and the results they have achieved. Second, this version of the case study goes into more depth of about the nature of the programme and how it worked.

Personally I’ve come to see three audiences for the Agile Cornwall case study: anyone who wants to adopt Agile, those who want to know the lessons of our Cornish Agile Laboratory (the Cornish Software Mines), and third, Government agencies who want to see how Government support can help companies adopt Agile.

Finally, I was also part of the panel discussion which is online too - this include Mary and Tom Poppendieck, Toby Parkins and Jason Gorman.

Agile will never work in Investment Banking

2011-10-19T08:34:00.001+01:00

In the past I’ve worked for banks, as an employee, as a contractor and as a consultant. And I’ve worked for companies that supply banks in one way or another. And I have lots of friends who work for banks and tell me stories. While I’m not an expert on banks and banking I know a little....

One of the things I know is that modern banking - in all its forms - is highly dependent on IT. Banks spend a lot of money on IT and develop a lot of their own software. As a result they have a lot of software developers. And I also know that quite a few of these developers and their teams have been, and are, experimenting with Agile working in one way or another.

It seems every bank has pockets of Agile, a team here, a team there. And not very often joined up.

Some of these teams are successful, some are less so. They often find themselves battling against big company / bank culture. The banks seem to be very good at killing their Agile teams: they get disbanded, forced bank to traditional (pseudo-Waterfall) approaches by bank procedures, or constrained by “the way we do things here.”

It also seems - notice we get further and further from fact and more into conjecture as I build this argument - that many banks have cottoned on that Agile is good and can help them. To this end many banks (and this is fact) have established Agile support programmes of one form or another. There are people at many banks who are trying to make the bank Agile - at least in software development. For many of them it is an uphill struggle.

At this point we should differentiate between retail banking - what you do on the high street with your bank - and investment banking - or casino banking if you prefer. (Yes there are other divisions but lets limit ourselves to two.)

In retail banking I think there are grounds to be a little optimistic. The stories aI hear are that Agile pockets are having success and things are gradually, slowly, improving and the efforts to link things up are slowly bearing fruit.

In investment banking I’m very pessimistic. I have come to the conclusion that while individual development teams may be successful at Agile for a while inside an investment bank all such teams are ultimately doomed. Your team might be Agile for a few weeks or months or even a year or to, but it will eventually be killed by the bank antibodies.

I have come to the conclusion that investment banking culture is polar to Agile culture. My reasoning is thus:

Investment banks are extremely hierarchical in nature; Agile is not. Agile can live with some hierarchy but not investment banking extreme.
Short-term in outlook: investment banks are short-term in the extreme - microseconds in some instances. Agile teams can react in the short-term but only by taking a long term view of engineering and quality.
Investment bankers are not engineers and do not understand engineering. Nobody from IT ever got to run an investment bank, you don’t get up the ladder from a support function, you get there from selling and banking. A large part of the Agile story is about returning to good engineering principles. The lack of engineering skills, thinking and culture in investment banks undermines Agile.
Investment bankers think you can solve every problem with money. Throwing money at Agile can work in pockets (by the best team) but doesn’t last or scale.
Investment banks are individual focused: individual bonuses, the power of the one trader to make money or break the bank (from Kweku Adoboli to Nick Leeson and before.) Agile is team focused.
Investment bankers think applying more pressure is the way to get results; Agile people now that applying pressure breaks things. A little pressure can help but apply too much and things snap, bankers don’t get engineering so don’t get this.
Risk aversion: yes, investment banks are highly risk averse when it comes to IT. Perhaps because they take so much “risk” directly with money they very risk averse in their operations.
Investment banks are contradictory: they say they embrace and manage risk but actually they are risk averse (at least in operations); they say they value the team but their actions say otherwise; they call themselves “financial engineers” but they no nothing about engineering; and so on. Agile is about honesty, facing up to truth and acting on it. Investment banks recent track record shows that honesty is sometimes questionable, to say the least.

I’m sure many of you reading this will say “Rubbish, I know at team at ....”. I am sure you do. While that team might be successful for a while the investment bank culture will eventually kill the team.

So how can we help investment banks overcome this problem?

We can’t. Its the way it is. Take the money, and run, or keep you head down and hope for some more.

Eventually new financial institutions will emerge which get Agile, not just at the software level but elsewhere. These institutions - which might be banks or might take some other form - will eventually replace the investment banks.

This is going to take time. Investment banks are fundamentally a broken business model which are propped up by Government support and market failure. This means normal economic logic does not hold for these institutions.

Cornish Case Study

2011-10-16T10:21:00.000+01:00

In the last few weeks Michael Barritt of Oxford Innovation and myself have presented several versions of “Agile in Cornwall” - a case study of our work introducing Agile to companies in Cornwall.

All three versions of this Agile case study presentation are now on my website. I recommend the latest two version - Agile Business Conference and Agile Cambridge. The version from Agile on the Beach in Cornwall was supported with presentations from two of the companies concerned - Research Instruments and Sullivan Cuff Software. In fact, you can find my own case study of the work at Sullivan Cuff on the Software Strategy website. A video version of this will be available in the near future.

If this interest you, or you want to hear it not read it, then I’ll be doing a revised version of the presentation at Skills Matter in a couple of weeks - “Agile Cornwall case study: How the EU is helping Cornish companies get Agile” - a free event, registration required.

Layered burn-down charts

2011-10-10T16:48:00.005+01:00

I’d like to conclude this Burn-down chart mini-series (Burn-downs are not just for Sprints and Advice for creating burn-down) by showing a variation on the classical burn-down chart I created with one of my Cornish clients. I’ve recently been advising an SOA project at another client to adopt a similar approach.

In fact, I’m finding this variation makes even more sense then the original, this is the Layered burn-down chart, here is an example:

I’ve redrawn this example to remove client details. What it shows are three things:

First the classic burn-down, just look at the overall line declining.
Second the velocity line as described in my earlier blog
Third the different colours layers show different releases, let me explain...

This company had taken the product backlog and divided it into monthly releases: November, December, January and so on. By the way this was physical on cards and each month’s cards were held together with a bulldog clip.

The contents of a release can - and would - change. But the company felt it knew what it wanted and since the team knew their velocity they could take a stab at which release it would be in. If something new was added the question was simply: when in which release do you want it? And the related stripe on the chart would increase.

That by the way is why the three strips of work yet to do are approximately equal size. If one grew it was obvious, work needed to be removed - either thrown away, or moved to another release, perhaps one that didn’t appear yet.

This was fairly straight forward when the company (thought) it knew everything to be done and ball-pack estimates could be given. With my SOA client they don’t know what the future holds but they can produce a striped chart for the work they do know about. In their case I’m suggesting every stripe represents a requested service rather than a release.

As new services are requested, and estimated, extra stripes can be added to show the total work required. Together with the velocity and slope it should be easy to see when a service will be available to their customers.

Advice for creating burn-down - and other capacity measurements

2011-10-06T09:43:00.001+01:00

Continuing from my last blog entry (Burn-downs are not just for Sprints) I’d like to offer some advice on the subject, burn-downs, cumulative flow diagrams, velocity, etc..

1) Don’t track hours. Hours are input and its better to measure outputs. Hours are subjective, see my Humans can't estimate tasks blog. Worse still we have too many psychological hang-ups about hours to make them usable. Instead you want your own currency, call them story points, abstract points, nebulous units of time or what ever you like. Having your own currency allows it to adjust to your team.

Each team has its own currency. Its like having the dollar, euro and pound. They all float independently, there is an exchange rate but it changes.

2) Draw your graphs by hand, process your data by hand. Get a feel for your data. Unless you have a feel for the data and an understanding of how it comes to be you can’t reason about it.

When I say “by hand” I allow you to use Excel (or similar), graph paper is better, but whatever you use don’t use a tool like Rally, Version One, etc. because you won’t have a feel.

Jack Kilby, co-inventor of the silicon chip and pioneer of the electronic calculator, lamented that his invention(s) deskilled engineers. Before the calculator engineers used a slide rule, this meant the engineers needed to know where to put the decimal point. They needed a feel for the data. Kilby lamented that the calculator meant engineers didn’t have the same intuition because of the calculator and led them to make mistakes elsewhere.

4) Use estimated data, forget actuals: estimates and actuals are apples and oranges, or rather, they are future facing estimates and retrospectives estimates.

Estimate tasks just before you do them (e.g. iteration planning meeting), put ball-park estimate on stories which you aren’t going to do any time soon (i.e. later than this iteration), and use averages for really long term stuff.

When you break the stories into tasks, or epics into stories, don’t expect the numbers to match. In fact, you might want to track the variation for longer range forecasting.

5) From 2 & 4: if you have a time-tracking/time-sheeting system leave it to one side, let it exist in its own little fantasy bubble. Don’t use the data, its toxic - see my entries on Capers Jones work. Live by your velocity measurements and charts.

Ultimately, burn-downs are about capacity measurement: John Seddon in Freedom from Command & Control talks about creating capacity measurement charts within organisations. The various graphs we use in Agile are our version of capacity measurement. Accepting its about capacity measurement means its not about targeting. Targeting is wrong see my entry form last year - Velocity targeting and velocity inflation - for more about this.

Burn-downs are not just for Sprints

2011-10-04T17:56:00.004+01:00

One of the key tools in Agile is: make things visible. When you can see the thing you can share the thinking and reason able it.

Capacity charts - usually burn-down, burn-up or their cousins Cumulative Flow Diagrams - are amazingly useful things. They show you the state of a development effort. They show this in data. Data by itself is hard to reason with but put it in a graph and wonderful things happen.

It has recently come to my attention that what I thought was obvious about these charts isn’t. So this blog entry, and maybe a few more to come, I plan to discuss charts, and specifically burn-downs, in a bit more depth,

I always encourage, sometime coerce, teams into produce some graphical tracking of their work. For someone like me who’s favourite tool is an spreadsheet this is a piece of cake. But have to remind myself that not everyone finds this so easy.

Importantly there are two burn-down charts you might want to create: a Sprint burn-down and A Product Burn-down. It should be immediately obvious that these correspond to the Sprint Backlog and the Product Backlog.

Just to be clear:

Sprint based burn-down chart which shows the progress in the current sprint towards completion of all scheduled work and shows work on a day-to-day basis.
Product based burn-down chart which shows progress through some larger quantity of work, e.g. project, release, milestone. These are updated on a sprint to sprint basis.

Superficially both types of chart look the same, the difference is in the X-axis - Sprint based charts have days, product based ones have iterations. An example will help:

Now I’ve never found sprint based burn-down charts very helpful. Perhaps because my background is working on large development efforts which take months or even years to deliver I rarely see an actual delivery at the end of a sprint. We may have something to show but its only a step on the way to something bigger.

Or perhaps its the economist or statistician in me, trained never to read too much into one set of data figures, and I know we can all have bad days, so I don’t see the point of acting when yesterdays velocity falls.

But I know some people do find value in sprint burn-down so I don’t object if you want one.

However, I always see Product Burn-down charts as vital.

Except on the smallest pieces of work there will be something coming next and a product burn-down can give a good sense of context, overall progress and, most importantly, when you could be finished - especially important on large pieces of work. Armed with this information you can construct sensible release plans.

I recently came across two teams at different companies who had burn-down charts but the charts only covered the iteration (sprint). This seemed wrong to me, it doesn’t tell me much. On closer examination the explanation become clear: the product backlog for both teams wasn’t in a state you could use for a burn-down.

Now if you are taking a truly evolutionary approach this isn’t a problem, you re-evaluate iteration-to-iteration. But actually, few teams take a completely evolutionary approach and these two teams where far closer to the iterative style (i.e. they had original requirements documents to work from).

The lack of a longer-term burn-down was a problem and a sign of a bigger problem.

One enhancement I like to make is adding velocity to the burn down chart. Take this one for example:

This adds a second axis (in Excel you have to hunt around for this option) which allows you to see the team velocity sprint-on-sprint.

The key thing is, without this data, velocity and burn-down - and the graphs to understand it - each sprint exists in isolation. That might be OK if you are delivering every sprint and you don’t need to peek into the future. But the kind of work I see does need to look forward and this information is essential.

Finally, I’ve most of what I’ve said about burn-downs applies equally well to cumulative-flow-diagrams (CFDs). In fact I prefer CfDs and readily use them over burn-downs because they help show how work increases. However, I have also discovered that CFDs are not only harder to construct and update but also harder to explain to people. There is something obvious about burn-downs.

Anyway, you pays your money, you takes your choice. Its up to you.

Agile Cambridge: Cornwall Case Study online

2011-09-29T17:37:00.000+01:00

The slides from my presentation at Agile Cambridge are now on my website: Agile County, Making Cornwall Agile (a case study)

Propogation delays

2011-09-28T10:02:00.001+01:00

Propagation delays occur when there is a chain reaction, a sequence of reactions which take time to work down the chain. Think of a line of traffic waiting at the red light. The light turns green: the driver of the first car doesn’t move instantaneously, it takes a moment to registers the change, perhaps move the car into gear or take the hand break off, moment to press the accelerator and a moment for the engine to start moving the car.

The second car in the line might be watching the traffic light but they are also watching the car in front. Only when the break lights (if on) go off and the car begins to move can they move. And so on.

It takes time for the change to work its way down the line.

The same thing happens in integrated circuits, silicon chips. Even electrons take time to move. Reducing propagation delays is one of the ways chips get to go faster.

Once you know about propagation delays you start seeing them everywhere. Increasingly I not only see them in software development teams but I talk about them. But not everyone knows the concept, hence this blog.

Propagation delays occur when one team, or one developer, changes some code, or perhaps a database schema, and it takes time for all the other team members to update their code base and make any resulting changes.

Propagation delays occur when a company decides to start doing something, or stop doing something, and it takes time to work down the chain and to actually stop or start.

Propagation delays cause tension because different people, or teams, are working to different assumptions.

As with chips and traffic reducing propagation delays can speed things up.

Updates for Dialogue Sheets

2011-09-23T16:11:00.000+01:00

Two dialogue sheet updates today.

Firstly I’ve created a mailing list to make announcements about dialogue sheets and for discussions about the use of dialogue sheets. This list is open to all and is hosted on Google groups (so you need a Google account).

Second, I’ve uploaded a new dialogue sheet. Another retrospective sheet this one is designed for use at the end of training course. I’ve used it at the end of several Software Strategy Agile training course and it has been well received. This sheet fulfils three purposes:

It allows participants to experience a retrospective
It provides for a retrospective on a training course at the end of the course
It introduces dialogue sheets as a technique for retrospectives

Although the sheet was intended specifically for use at the end of an Agile training course there is little that is specific to Agile in it. I’d really appreciate feedback from other trainers who try this sheet.

As usual, all this is at www.dialoguesheets.com.

Cover for Business Patterns for Software

2011-09-23T10:59:00.008+01:00

My new book, Business Patterns for Software Developers, now has a cover. Here it is:

The Business Patterns is available for pre-order, it will be published until early in the new year, I’m guessing February but it could move either way.

Dialogue Sheets in Methods & Tools

2011-09-20T13:24:00.000+01:00

The latest issue of Methods and Tool (Fall 2011) contains an article about Retrospective Dialogue Sheets. If you are new to Dialogue Sheets it will make a good introduction, and if you’ve used them already thenI’d love to hear your feedback.

Dear client, the truth about software development

2011-09-15T14:19:00.001+01:00

Dear Customer,
I think its time we in the IT industry came clean about how we charge you and why our bills are sometimes a bit higher than you might expect.

The truth is, when we start and IT project we just don’t know how much time and effort it will take. Consequently we don’t know how much it will cost. This isn’t a message you like to hear, particularly since you can be absolutely certain that you know what you want.

Here in lies another truth, I’ll try and put it as politely as I can, you are, after all, a customer and really I shouldn’t offend you. The thing is, you don’t know what you want. O yerh, you know in general (usually) but the devil is in the detail. And the more you try and give us the detail before hand the more likely it is to change. Each time you give us more detail you are offering more hostages to fortune.

Just to complicate matters the world in uncertain. Things change, companies go out of business, customer tastes change, fashion changes, Governments change and competitors do their damnedest to make life hard.

Yes it is true we can sometimes gold plate things. Yes some of our engineers like to do more than needed, and yes we have vested interest in getting things added so we can charge you more.

It is also true that you have add things: you quite legitimately think of things after we’ve begun, you quite naturally assume something is “in” when we assumed it was “out” and you also try and put one over too.

(Lets not even talk about bugs right now, it just complicates everything.)

Frankly, given all of this it is touching that you have so much faith in IT to deliver. But then, when IT does deliver, boy, does it deliver big. Look what it did for Bill Gates and Larry Page, or Jeff Bezos of Amazon, or FedEx, the list goes on.

So how do we ever talk you into this?

Well, we package up this unsightly mess and try and sell it to you. To do this we have to hide all this unpleasantness. We estimate how much time we think the work will take, actually these “estimates” are little better than guesses; human’s can’t estimate, most companies don’t have historical data and for those that do have data most of it is worse than useless.

So we get this guess, sorry estimate, and double it. Well maybe we double it, we might treble it, and when we see the new number, if it looks too much we might reduce it. And then we work out whether you will pay it.

That might sound awful but remember: we could have guessed higher in the first place.

Anyway, we don’t know which number is “right” but to make it acceptable to you we pretend it is certain and we take on the risk. We can only do this if the number is sufficiently padded. And even then we go wrong.

If the risk doesn’t happen then we get a fat profit, if the risk comes to pass we don’t get any profit, maybe we make a loss, and if its really bad you don’t get anything and we end up in court.

Really its all a question of risk: we try to present a clean, low-risk solution to your problem, and in doing so we take on a lot of risk. The alternative is that you take on the risk - and the mess - and do it yourself. (Unfortunately, another sad truth, is that as far as anyone can tell in-house IT is generally not as good as specialist providers so if you do decide to take on the risk you actually increase it.)

There really isn’t a nice simple solution to this, you have to work with us on this.

Thats the first part of the solution: you, the customer, have to be prepared to work with us, the supplier, and work with us again and again. This will reduce the risk.

The second part of the solution is: you need to be prepared to accept the mess and share the risk with us. If you aren’t prepared to take on any risk then we will take it on, and we will charge you for it, and since the risk is a lot we will charge you a lot.

Sharing the risk also has the effect of reducing the risk. Once you share the risk you have a motivation to reduce it.

If you are prepared to accept those two suggestions then lets press reset on our relationship and talk some more.

Business Patterns for Software Developers on Amazon for pre-order

2011-09-06T14:45:00.000+01:00

Milestones in the writing on Business Patterns for Software Developers are coming think and fast this week.

Monday saw me reviewing book covers - the proposals look really good. Like many other patterns books the cover will feature architecturally significant buildings. Unlike most other patterns books these three are also a World Heritage site.

Now the book is listed on Amazon for pre-order - order now!

Which means, it also has an ISBN number - 978-1119999249 is Business Patterns for Software Developers - or if you like the old style number 1119999243 - spot the difference :)

(Amazon are listing the release date as May but I’m pretty confident we’ll better that, I also suspect the page count and price will be higher than currently listed.)

I’m pretty close to the final text, mainly images I need to fix now.

The keen eyed among my readers will notice the words “Agile” and “Lean” are absent from the book title. Indeed, the words are largely absent from the text. They are in there, somewhere, they are difficult to ignore, but this isn’t a book about Lean or Agile. Yes it contains lots of advice for Lean and Agile companies but it isn’t just about them.

Read my lips: Its not about Agile, its about Business. Thats all it has ever been about, really.

In the meantime, the business patterns which form the book are still available on my website and will continue to be so. The versions which will appear in the book are a lot more polished and professionally edited, plus the book will have a lot of other new material.

'Objective Agility' & 'What is Agile' presentations online now

2011-09-05T14:25:00.000+01:00

Last week I presented a revised version of “Objective Agility: What does it take to be an Agile company?” at Skills Matter in London. The presentation is now on my website (Objective Agility: What does it take to be an Agile company?) and Skills Matter have a PodCast available too (Objective Agility PodCast), so you can here my words too!

By the way, I will be teaching my Agile for Business Analysts course at Skills Matter in November if you like what you see and would like to see more of this.

I’ll be presenting Objective Agility again at Agile On The Beach in Falmouth in a couple of weeks. There are still a few tickets available for the conference but the early bird rate is about to expire.

As part of the build up to Agile On The Beach I did a webcast a few weeks ago entitled “What is Agile?”. The slides for this are also on my website (previous link) and the AgileOTB folks hope to have a podcast available soon.

Two more business patterns: Customisable Product, Customer Understanding

2011-08-31T14:08:00.000+01:00

At this year’s EuroPLoP conference I presented two more business patterns for workshop review. There are:

I have now updated the paper with the workshop comments and the patterns can now be downloaded from my website. The other patterns in the Business Patterns series can also be downloaded.

These are the final two patterns for my forthcoming book Business Patterns for Software Developers. The text of the book is just about finished and it will be going to the publisher soon. It should be available early in the new year.

Is "Faster Better Cheaper" axiomatic in Agile?

2011-08-17T14:31:00.001+01:00

“Faster Better Cheaper” - a cry heard often from management types!
Or at least engineers think thats the sort of thing managers say. Personally I don’t think I’ve ever actually heard a manager say it but I have heard engineers say managers want it “Faster Better Cheaper” with scorn in their voice.

The idea that you could have it (whatever it is) “Faster Better Cheaper” seems to have gained popularity when Daniel Goldin was the head of NASA. As many engineers, including myself, are quick to point out the original quote was “Faster Better Cheaper, choose two”. I say “original quote” but I have no idea who said it first, WikiQuote doesn’t help, and Google doesn’t shed much clear light. (Anyone out there know?)

Whatever, last week, I heard another engineer-type say that he believed management wanted “Agile” because it was “Faster Better Cheaper” and, not for the first time, it got me thinking. Is Agile Faster Better Cheaper?

My knee-jerk reaction is “No!” but the more I think about it the more I think it might actually be so.

Firstly, Faster than What? Better than What? and Cheaper than What?

Presumably “Faster than traditional Waterfall development”. Or at least, what-ever-it-is-you-are-doing-now, which is probably to some degree based on waterfall.

In the short run, when a team are first changing the way they work thy are quite likely to go slower than they would do otherwise simply because they are learning something new and not using a set of practiced reflexes.

Agile most definitely promises to deliver something sooner - if you want it, plenty of business folk seem unhappy with getting things too soon or too often. But Agile will NOT deliver the whole thing Faster, it can only promise to deliver a part sooner.

The logic of Agile recognises “You will change your mind when you get something”, meaning, if the business/customer sees part of the final thing they will add, remove and change the thing they originally asked for. In some cases, and I have seen this happen, they remove work. Lots of work.

Thus, the whole might come faster, but the whole is smaller than the thing you thought it was.

So, Faster - in the short run no, in the medium run you should get something sooner, and in the long run, Yes, it will look like Faster.

What about Better? - how are we to define better: better quality (fewer bugs), better suited to need, more fully filling the initial request.

Yes: Agile promises better quality (fewer bugs). Indeed if you don’t improve quality and remove bugs you are going to have problems doing Agile. If you do improve quality - fewer bugs - then the evidence suggests you will have shorter schedules, i.e. you will go faster.

Better suited to need: again, Yes. If those who made the requests are seeing something sooner, and having their views on functionality taken into consideration then one would expect the final product to be better suited to need.

More fully filling initial request: No, complying with “better suited to need” means we are not aiming to build the thing we first thought of but rather build the thing we need to satisfy the need.

So, if your criteria for success, for better, means complying with a requirements document that was frozen at some arbitrary point in time then No, Agile falls at this hurdle.

Cheaper: again there is a short run and a long run dimension here.

In the short run the development team need training, coaching and time to practice. Of course you could skip the training and coaching (plenty of teams do) but that means they will need more time to practice (make more mistakes, go down a few dead ends, etc.) Practice time costs.

In the short run I don’t think Agile is cheaper. Indeed, in the short run, if you are doing it properly you will see expenses rise as you have the likes of me come to train and coach your teams. (If you are not doing it properly you probably won’t see your costs rise but they will all the same as teams (slowly) learn by themselves.)

In the longer run, when you’ve finished the training, coaching and when the team are proficient then those extra costs will be gone and you should be cost neutral as least. But if you are not writing, finding and fixing so many bugs, and if parts of the requirements are being left undone - while satisfying your business customer - then costs should fall because you are expending less effort and doing less work.

Even if you are not doing less work costs should fall because on a software project costs are dominated by the number of people you have multiplied by the time you have them. If you are going faster then time is reduced and costs come down.

Therefore the project will be cheaper.

Summary: Better (fewer bugs) provides for Faster (delivery) which results in Cheaper (software).

Agile is “Faster Better Cheaper”, QED.

When you put it like that “Faster Better Cheaper” looks axiomatic over the long run.

The engineer in me hates this conclusion, I’m not prepared to accept it but if I follow the logic it is true. (The engineer in me would love someone to find a flaw in my logic so please go for it! Shoot me down!)

Note the three assumptions in this logic:

The reference point is the “Waterfall” model of development
Your definition of better considers a low-bug count important and emphasis fitness for purpose over conformance with initial requirements
Your are prepared to spend in the short run for quality (defect prevention) to save in the long run

How can we explain this? (Notice the change from ‘I’ to ‘We’, I’m pulling you into the conspiracy.)

Well, if this is true then its not so much that Agile is “Faster Better Cheaper” but that the model we are comparing it with - the traditional (“waterfall”) model - is so utterly broken.

Waterfall breaks Faster Better Cheaper everywhere:

Waterfall leads to large batch sizes and economies of scale thinking: in software development there are dis-economies of scale so large batch sizes makes everything slower
Waterfall project managers believe quality (bugs) can be traded off against time but actually the opposite is true. Reducing quality reduces “better” and slows a work down. The same project managers are trained to resist requirements change so almost guaranteeing business customers will be dissatisfied with what is delivered and consider it “worse.”
Slowing software development down makes it more expensive, remember: Cost = People x Time

Its not so much that Agile is a good model but that the competition is hopeless. It isn’t even a fair comparison. My belief is that Waterfall never really worked anyway, so comparing Agile to Waterfall is a false comparison - bang goes a big assumption.

In which case the, returning to the original question: “Is Agile Faster Better Cheaper?” the answer is really “Depends on what you are comparing it to.”

Dialogue sheet observations

2011-08-15T10:07:00.000+01:00

Regular readers will know I’ve been pushing Dialogue Sheets as a new retrospective technique. I have observed a number of dialogue sheet sessions - both the retrospective sheets I’ve made public plus some other sheets I’m still testing. Someone from Motorola Mobility recently asked for my observations on using dialogue sheets and I thought it would be a good opportunity to make my observations public.

As a facilitator I find there is very little for me to do, the sheets are designed to be used without a facilitator. As an observer I have been asked to intervene occasionally. Usually I try to say nothing, if the group is willing they can usually work out what to do themselves. When people in the group are either suspicious (something new) or unsure about the retrospective they usually need a few words of explanation.

The most common sticking point is Kerth's Prime Directive. Some people seem unwilling to accept it for the duration of the retrospective. They start making comments about how they don't think people did not do their best.

I don't like stepping in when this happens because I think the group needs to come to an understanding themselves. This eventually happens even if it is something like "well we kind of ignore that" but it can take a while.

The second thing I have noticed is time: It is not how many questions there are on the sheet that determine how long the retrospective will take but the number of people involved. When there are 4 people it is quite fast, when it is 8 it takes longer. So far I haven't found a good rule of thumb for determining just how long the sheet will take.

Sometimes I intervene to just remind people of the time and move them along.

The other bit which can be difficult is the end questions. They are intended to bring the group to set of conclusions which they can act on. Still sometimes the items are "Better communication" which while right isn't very specific or actionable.

There have been plenty of dialogue sheets downloads now and I occasionally e-mail downloaders for feedback. Of those those who have used the sheets “energized” is the most common comment.

I’m planning to revamp the three existing sheets and add one for project start-up. I’ve also had a request to translate them into other languages - specifically German. Once the revamp is complete I’ll start on some translations. I’m also thinking of creating a mailing list for discussion dialogue sheets.

If you haven’t looked at the dialogue sheets they are free for download, and you can buy pre-printed dialogue sheets.

Skills Matter talk postponed

2011-08-10T14:43:00.000+01:00

Apologies to anyone who was planning on attending my “Objective Agility: What does it take to be an Agile company?” talk at Skills Matter tonight. The presentation has been postponed until 1 September.

Paul from Skills Matter and myself agonised this morning about postponing the talk. We both wanted to go ahead but with recent events in London we thought it best to postponed until things were a little quieter.

Quite a few people were booked for the event, I hope you can all make it for September, and those of you who couldn’t make it, we’ll you can come too!

Xanpan

2011-07-21T14:13:00.009+01:00

For a little while now I’ve been quietly talking about my new Agile method. The clue is in the name: Xanpan - pronounced “Zanpan”. Most obviously Kanban and XP (Extreme Programming), its a fusion.

Its not so much than Xanpan has any new radical ideas in it, its more than it is a synthesis of ideas from several Agile methods plus my own. It contains a fair chunk of what I would call “product management ideas”.

Xanpan is a little more than that, in takes from Scrum as well as XP, and from Lean as well as Kanban, plus there are other ideas in too.

I’ll write more about Xanpan in future but for now here are the essential features, and where it steals them from. A little rough and ready but I wanted to share this and get some feedback.

From Extreme Programming: The technical practices

Test Driven Development plus Automated Test Driven Development. If you want to adopt BDD then go right ahead
Lightweight, near time code review, better still pair programming if developers are happy with it
Continuous integration: with tests and static analysis tools as part of the build system
Rough up front design: no up front design is wrong, but so too is design that takes weeks or months. If you need a design session it is probably part of the planning meeting with all the developers around the whiteboard for an hour or maybe two
Refactoring to keep the code soft - its software, emphasis the soft
Shared code ownership
Minimise documentation, accept tacit knowledge as part of the development process.
Velocity measurement and “yesterday’s weather” approach to deciding what to put in the next iteration

From Kanban: Process flow

Have a visual board to track progress
Divide the board into columns which represent you process - probably more than just: Todo, In Progress and Done. Columns are usually come in pairs: a queue then a action which draws from the queue. Work to improve the flow and change the columns as you go
Work in progress limits - usually on action queues, occasionally queues may be limited too
Improvement comes from improving flow
Minimise the time spent estimating, if possible abolish estimates and use statistically derived approximations, i.e. averages
Cumulative flow charts

From XP / Scrum: Process rhythm

Use iterations to establish a regular rhythm to the team
Iterations start with closing the previous one: review work done, update statistics, conduct a retrospective. This is followed by a planning meeting: work is presented, estimated (if need be) and scheduled
Stand-up meetings where appropriate
Burn-down/up charts where appropriate
User Stories are the default requirements capture tool although Use Cases, Planguage and more informal approaches are acceptable too. If using User Stories then do not use more than three levels of division

For task management Xanpan builds on Blue-White-Red, which is itself a XP/Scrum fusion.

From Lean: Culture of improvement, Kaizen and Learning

Retrospectives are not enough: Leaders should undertake personal reflection
Rehearsal: Teams should undertake formal and informal training together; deliberate practice as Kevlin Henney and Jon Jagger like to say
Team Coach: Each team should have a coach, different coaches may focus on different things so there may be more than one coach. Except in large teams (over 12 people) and in the early days of adoption the coach is usually not full time on a team. The team is there to help the team adopt practices, learn and reflect.
Individuals have multiple skills and should all muck in, but there are specialists in some areas.

Somethings Xanpan doesn’t take:

Kanban’s stop the line quality control: sometimes it is appropriate, sometimes it isn’t. Keep regular retrospectives, part of your rhythm
Scrum Master: teams will have a Manager or Project Managers, or may be lead by an non-commissioned manager, e.g. Team Leader, Technical Lead, or similar
Anti-manager ethos: Management is part of the solution, not the problem. Unfortunately the quality of IT and development managers is general is poor; good management can be really powerful

Something Xanpan doesn’t like but doesn’t outlaw (because it can’t):

Matrix management
Distributed teams, particularly teams in different timezones

Something I find missing in all Agile methods to date is the right balance between engineering and requirements. Therefore....

Xanpan broadly follows the 10 Step Requirements Model but recognises this model is not broad enough to cope with all scenarios. No model ever will be.

On requirements Xanpan believes:

There should be a dedicated requirements role staffed by a trained/experienced Product Manager or Business Analyst; both is probably overkill
There should be a clear business case setting out why a team or project exists. The business case is not a shopping list of features, nor is it a large document but it should allow someone to decide the work is finished
Customers are not the only stakeholders. Each stakeholder will make their own judgement about the success or failure of work therefore each stakeholder needs to be considered
Evaluating the benefits of completed work is as important as deciding what work should be undertaken next. There is no point to developing more software if the software that has been delivered so far is not valuable.

And some more:

If the team has an architect they should also code; the architects role is to help create a shared understanding of the architecture and educate less experienced team members.
Teams should be kept together and not broken up when work finishes or changes; bring the work to the team, not the team to the work
Block and impediments should be tracked to see which ones occur repeatedly. If a team cannot resolve an impediment themselves then the leader or manager steps in, in effect it is the start of an escalation
There are three version of Xanpan: Evolutionary, Incremental and Iterative - these build on the ideas in my Agile Spectrum article. Iterative is the most compatible with existing corporate culture and project management methods - salami slice requirements; Evolutionary is the extreme - goal directed working
Xanpan adopts the three planning layers described in my Three Plans for Agile article and encourages Goal Driven Projects.
Xanpan believe management has an important role to play in the development process (but a lot of development managers are not up to the job)
Whenever possible put bug-fixing in a separate stream of work; but staff it with the same people, rotate people
Keep teams together: bring the work to the team, rather than the team to the work

Underlying it the Xanpan philosophy is about:

Quality is free: automate as much testing as you can, although you probably can’t automate 100%

Prioritisation is vital and omnipresent: there is no such thing as too little time, just mis-understood priorities

People are key to good software development but there aren’t enough good people to go around, so we need to improve the people we have and help them work more effectively. And we need to grow more good people

Agile is not about what you do but what you achieve; measure outputs not inputs

You can always improve

Customer/End user involvement is key to building a successful system.

Xanpan is a pick-n-mix of bits of various development methods and adopters are encouraged to continue the approach

No process or Methodology can cope with all situations, and if it could it would be too big to write down or learn, there adaptation is always required and people need to think

The process should be changing, if you are doing Xanpan the same in six months as you do today you aren’t doing Xanpan

Cornwall nominated fror Agile awards

2011-07-18T18:28:00.005+01:00

I’ve tweeted and blogged about the Cornwall Agile Programme before - something I like to call the ‘Cornish Software Mines’. So it is with great pride that I’m please to announce that the programme has been nominated for an award.

The Agile awards are presented as part of the Agile Business Conference and ‘Agile Cornwall’ has been nominated under the ‘Best use of Agile in the Public Sector’ - officially the programme is part of the ‘Convergence Programme for Cornwall & Isles of Scilly’ so the nomination is under this name.

This autumn I’ll be delivering a couple of case studies of the programme and the companies which have benefitted. The first of these will be, fittingly, at the Agile on the Beach conference in Falmouth. This isn’t a coincidence, the conference is a spin-off from the programme. This will be the most comprehensive case study as it will include presentations from several of the companies who have benefitted from the programme.

A few weeks later a cut down version of the case study will be appearing at the Agile Business Conference. If you miss it there I’m sure it will have another outing before long.

In the meantime, I’ve posted a case study of the work I’ve done with Sullivan Cuff Software on the Software Strategy website.

Retrospectives - common or not, a small survey

2011-07-12T14:10:00.011+01:00

I’m still experimenting with Dialogue Sheets for Retrospectives (the download page, and my earlier blog entry) and so are other people. Of the feedback I’ve had so far it has been overwhelmingly positive. Perhaps people who aren’t positive don’t bother trying them.

The sheets are free to download but I do request people register. My objective here is to obtain more feedback. Periodically, like today, I get the e-mail addresses of those who have downloaded and I send a polite note saying “Any feedback?”.

In registering I ask a couple of other questions. One of these questions is: “How often do you hold a retrospective?”. I thought it would be interesting to share the results of this data so far:

How often do you do a retrospective?

38%	Every two weeks or more often
21%	Never
16%	At least once a month
15%	I am a retrospective facilitator and so hold many
6%	Rarely
3%	At least once a quarter
1%	About every six months

This is good to see, about 54% of people are holding retrospectives with the frequency you would expect from a Scrum, XP or other type of Agile team. But sadly the second biggest group is never holding retrospectives, 21% of people. And 10% are holding them rarely or very occasionally.

Now think again, this data is not representative. 15% of people are not retrospective facilitators (e.g. Scrum Masters, Agile Coach, etc.). The people who download these sheets have an interest in retrospectives, this group is self-selecting.

The implications of this are that an awful lot less than 54% of people are doing retrospectives with anything near the frequency one should expect from an Agile team. Given that retrospectives are the primary means of learning in an Agile team I suspect that means that an awful lot of teams are are not really practicing Agile as described in the books.

I have long suspected that retrospectives are actually one of the more advanced Agile techniques and are far from common. I think this data supports that argument, but at the same time I think they are more common than I tended to think, maybe thats the progress of 3 years.

Abandon Hope all ye who enter Agile

2011-07-05T22:01:00.001+01:00

Earlier this week I had a conversation with Benjamin Mitchell about an entrepreneur he’d done some work with. Benjamin had been suggesting a lean start-up approach to the business in which the entrepreneur worked to validate his business idea early and gradually build the product one validated step at a time.

Things didn’t go well, the understanding I came to from Benjamin’s story - although his might be different - was that the entrepreneur was very attached his business idea. It was pretty well formed and he could imagine it working. In popular culture entrepreneurs are often seen as iconoclasts who battle doubters, naysayers and banks to bring their ideas to life against the odds. Benjamin was just another doubter.

Now I tend to agree with Benjamin’s approach because it is very difficult to tell if a new business idea, indeed any project, will work until you try to build it. But building it is expensive so I advocate a “Try, fail fast, fail cheap, move on to the next thing” approach - I don’t think this is that different to lean start-up thinking but as I’ve not read the lean start-up book I don’t really know.

I don’t just advocate this approach for new businesses, I advise it for established businesses embarking on a project. My logic is: you can’t be sure a product will work in the market till you try it, nor can you plan a project until you know how fast (velocity) the team will work. In fact, until the team trys to work together and build something you don’t have any data on which to project plans. Making estimates without data is little better than guessing.

Thus, I advise a Mao like approach to corporate projects, “let a thousand flowers bloom” - start multiple small projects, small teams and use portfolio review to remove those that don’t seem promising and reallocate resources to those which do seem promising - or launch more experiments.

Unfortunately, like the entrepreneur, people get very attached to the idea of a project so it gets momentum. Big corporations respond by only starting projects in which they have a high degree of confidence. That means they do a lot of pre-project work - planning, designing, etc., this in turn makes it difficult and expensive to start projects, which in turn means that any project that does get started already has momentum. And it means that once underway the project finds it difficult to cope with change.

It seems to me that successful projects in corporations are often the result of one person who wants the project to happen and does what is necessary, not unlike entrepreneurs. In both cases a strong willed, capable, person makes something happen. That person needs to be motivated, they need hope, they need ambition and hope. Such people are going to kind who ignore naysayers and doubters.

The Agile approach - try, gather data, evaluate, or my “fail fast, fail cheap, retry” approach are not going to go down well with the kind of pig-headed, obstinate, passionate, do-anything-it-takes, approach which has made many successful projects a success. Quite the opposite.

Interestingly Benjamin had another example of a company which does take the “fail fast, fail cheap, try again” approach and was staffed by passionate, hopeful, people. From what he told me these people were passionate about the process as much as they were passionate about the products they tried. They could accept regular failure of good ideas because they saw the process working, and perhaps more importantly, could transfer their hope to the next idea. There was always a next idea.

What I am saying is: Don’t rely on hope to get your projects done, use data - be empirical not emptional.

Finally, I should say, that while I talked much of this over with Benjamin he would be the first to point out unvalidated steps in my thinking, and ungrounded assumptions I’ve made. So maybe this entire entry is really a hypotheses. The hypothesis is:

“The trial and error approach of Agile is unlikely to please those who get passionate about an idea and want to move heaven and earth to make it happen.”

I’ll let you, dear reader, decide for yourself on that hypotheses. If you have any evidence - to support or disprove - my hypothesis please add a comment below.

When did Scrum start loving project managers?

2011-06-22T17:52:00.000+01:00

One of the things I’ve always found paradoxical about Scrum (specifically ScrumTM) is its position on management. On the one hand, Scrum is very management friendly - see my Scrum has Three Advantages over XP post. Basically Scrum has done a very good job of marketing itself to managers.

But Scrum is a little like a Monty Python Spring Surprise - “that's our speciality - covered with darkest creamy chocolate. When you pop it in your mouth steel bolts spring out and plunge straight through-both cheeks.”

Hidden inside the tasty Scrum case is a sometimes evangelical dislike of managers, and in particular project managers. Take these excerpts from the Scrum Primer (Deemer and Benefield, some versions have Larman as a co-author too):

“The ScrumMaster is not the manager of the team or a project manager; instead, the ScrumMaster serves the team, protects them from outside interference, and educates and guides the Product Owner and the team in the skilful use of Scrum.”
“unlike a project manager, the ScrumMaster does not tell people what to do or assign tasks – they facilitate the process, supporting the team as it organizes and manages itself. If the ScrumMaster was previously in a position managing the team, they will need to significantly change their mindset and style of interaction for the team to be successful with Scrum. In the case that an ex-manager transitions to the role of ScrumMaster, it is best to serve a team other than the one that previously reported to the manager, otherwise the social or power dynamics are in potential conflict.”
“Note there is no role of project manager in Scrum. Sometimes an (ex-)project manager can step into the role of ScrumMaster, but this has a mixed record of success – there is a
fundamental difference between the two roles, both in day-to-day responsibilities and in the mindset required to be successful. ”

Indeed I once sat in on a course entitled “Agile Project Management” by a well known Scrum trainer. In response to the a question from a project manager “What does a project manager do in Scrum?” the answer was “If you have a project manager in Scrum you aren’t doing Scrum.”

Take another example, Bas Vodde’s Nokia Test, the final question asks “are project managers (or anyone else) disrupting the work of the team?” Every time I show that question to a project manager we have to discuss it, project managers don’t generally believe they disrupt the team unreasonably. It certainly looks like Bas doesn’t like Project Managers.

A few years ago when Jeff Sutherland spoke in London I recall him saying there was little future for project manager. Many needed to revert to programming (which they had done before project management), a few would become Scrum Masters, a few Product Owners and a very few could continue being project managers on the largest projects.

Sutherland’s own Scrum Handbook seems pretty clear: “there is no team manager or project manager in Scrum.” (Actually, if you read what else the handbook says about project manager it looks like the text is taken directly from the Scrum Primer, Sutherland seems to feel the same way as Deemer, Benefield and Larman.)

Given all the anti-manager, specifically anti-project manager project manager noise from some in the Scrum camp I found it surprising a couple of months ago when I noticed that Scrum Master Certificate now come with the implicit endorsement of the Project Management Institute.

For example, take Martine Devos’ London Scrum Master Course, it is worth 14 PMI Professional Development units. Jeff Sutherland’s own Scrum Master course boast 16 PDUs (one assume these are PMI units although he doesn’t state so explicitly.)

So, if the PMI are now crediting Scrum Master Courses, and the Scrum folks are making their courses compliant with PMI rules then one assumes that the two are reconciled. Why would a project manager who is intent on being a Scrum Master, and therefore no longer being a project manager, want credits?

Maybe Turkey’s are voting for Christmas. It looks odd for me.

Actually, to be fair to Scrum, the situation is a little more complex than I’ve laid out here. Looking a little bit further into Scrum history we find the following.

Schwaber and Beedle in Agile Software Development with Scrum say “The Scrum Master is a new management role introduced by Scrum” and a little later “The team leader, project leader or project manager often assume the Scrum Master role.” This final statement describes what I have seen happen most often in practices.

In Agile Project Management withScrum Schwaber says “The Scrum Master fills the position normally occupied by the project manager. I’ve taken the liberty of redefining the role.”

One explanation might be that the view of the Project Manager has changed over time. Initially the Scrum originators saw project managers as candidates for filling the Scrum Master role - or at least not the source of problems. As Schwaber almost says: it is the same role filled in a different way.

Later Project Managers, and maybe all managers, came to be seen as a problem, and, most recently as it becomes clear that Project Managers can be Scrum Masters and can be a force for good on a project the position has returned to the original view.

Alternatively it might be there are multiple opinions on how Scrum and the Scrum Master relates to the traditional Project Manager role. It benefits some people, at some times, to claim the Scrum Master is not a Project Manager. And it benefits some people at other times to reconcile the two roles.

One of the advantages of Scrum, specifically ScrumTM, is that is defines what is it, and is not, much more clearly than “Agile”. However with time this picture is becoming muddied.

Finally, this entry is a bit critical of Scrum, I won’t pretend it isn’t, but really, I’d like to understand what is going on here. If anyone can shed some light on the thinking, whether it changed or not, please add a comment.

Agile on the Beach: Falmouth, September 15-16, 2011

2011-06-02T19:25:00.000+01:00

I’ve mentioned some of the work I’ve been doing in Cornwall over the last eight or nine months in this blog a few times, and anyone who follows me on Twitter will have seen my Tweets about the “Cornish Software Mines.” Well, one of the spin off of this work is a Cornish Agile Conference - Agile on the Beach.

Agile on the Beach is happening in Falmouth on Thursday 15 and 16 September 2011. Speakers include: Kevlin Henney, Tom and Mary Poppendieck, Rachel Davies, Steve Freeman, Jon Jagger, Jason Gorman and quite a few more - see here. Some more speakers will be announced nearer to the conference.

Tickets went on sale today, early bird price is £230 - if you are quick you might get one of the limited number of £195 tickets available.

The organisers (and yes, I’m one) hope that people attending the conference will take the opportunity to extend their visit to Cornwall into a long weekend. After all, Cornwall is much better know as a holiday destination than a software development centre.

All the details on the Agile on the Beach website. In addition, if you plan to attend sign on to the LinkedIn group and you can see who else will be there. More details are being announced all the time so to stay up with the latest announcements follow the Twitter hash tag #AgileOTB and Twitter User @Agileonthebeach.

Parrot cages, the true story

2011-06-02T19:10:00.000+01:00

I heard this story a few months ago from Benjamin Mitchell, it is good and I wanted to believe it so I did. (Benjamin by the way tells it with a good deal more embellishment.)

Anyone who has had a course from me in the last six months has probably heard me retell the story with a large health warning saying “I’ve heard this story, I believe it but I don’t have a source to prove it.”

Now, Toby Parkins has tracked down the link. So here it is: Parrot cages, the true story, care of the BBC - so it must be true!

Dialogue Sheets at Skills Matter - 19 May, 6.30pm, free

2011-05-05T15:46:00.000+01:00

I’ve blogged before about the Retrospective Dialogue Sheet technique. Later this month I’m doing a Skills Matter In the Brain Session about this technique. This will last about an hour to an hour and half and is free for all to attend, register at the Skills Matter website.

I won’t actually be doing much talking, I’m bringing along some specially created dialogue sheets which will give everyone an brief experience of holding a retrospective using dialogue sheets.

For more information about dialogue sheets see my previous blog entry, or the www.dialoguesheets.com website.

Light Touch Agile Coaching in the Cornish Software Mines

2011-05-04T16:29:00.000+01:00

I’m off to Cornwall again next week for my monthly visit. Anyone who follows me on Twitter (allankellynet) may have noticed that every few weeks I put out a few tweets along the lines of “On my way to the Cornish Software Mines again”. They may not actually dig source code out of the ground in Cornwall but there are some very active software companies there.

Under the auspicious of Grow Cornwall I have been in providing Agile coaching to several companies as part of an Agile programme designed to bring better development practices to multiple companies. To date six companies have been part of the programme to some degree and next week we are adding some more.

In the next few months I hope more of this work will come into the public domain as articles and case studies. There are even plans to hold an Agile conference in Falmouth, to be named Agile on the Beach. (More to follow soon.)

This engagement has really given me a chance to think about my coaching style and develop some techniques and approaches which I’ve thought about in the past but not really codified. One of these I call Light Touch Agile Coaching.

Many description of Agile Coaching assume a full time coach. Scrum specifically just about mandates a full time Scrum Master for each time. The Scrum Master is closest Scrum gets to an Agile Coach, indeed, some people would see them as the same thing. I don’t, and thats worth a full article in its own right but not today.

In Cornwall there are three companies I’ve be working with more than others and a clear pattern to the work has emerged. It goes something like this:

Companies joining the programme receive a training course over two days, preferably offsite. This has worked best when the net is spread wide and the whole development team and managers are involved.
The team are allowed a little time to digest this then asked: do you still want to try Agile? The programme is designed with the expectation that they say Yes, in which case the coaching begins. If they say No then fair enough, its is there decision.
I visit once a month, I spend between half a day and two days with the team. How much time I spend depends on the size of the team and the company, how long I have been working with them, and the particular set of problems they are faced with.
My visit usually starts with an inspection of the white board - call it a Kanban board or an Information Radiator if you prefer - its the board with all the work on it.
I read the board, I ask question, I get asked questions, I understand the state of play and get an idea on what needs addressing.
Sometimes, some places, I sit down with a manager and talk about the last month. What has happened, what progress they have made, what understandings they have reached, what has puzzled them, how the team have performed, and so on.
Next I sit down with the development team and conduct a similar type of exercise. Sometimes the teams have conducted their own retrospective and I review that, other times this exercise takes on elements of a retrospective.
(One of my challenges is to make all the teams proficient with retrospectives so when my work is finished they can continue themselves.)
In some companies I sit with the marketing group for a conversation too. All the companies I’m working with develop software for external customers and some of them have marketing teams. As the development teams have become more effective a lot of my focus has shifted to marrying up marketing with development so the right thing gets built.
During the course of these meetings I start to construct a list of “Expected next time”: things they managers and teams tell me they will change, or things they will put in place for my next visit. Naturally, as a I talk to people I’m also looking through the last “Expected next time” list and seeing how the team have done.
I started doing the “Next time” list for my own benefit but I’m open about it with the teams and it has become a useful little tool in both tracking whats changing and getting commitment form the teams.

I like this light touch approach. On the whole it has worked well, I get to help the managers and teams while they keep responsibility and feel ownership. Yes, there are cases were I wish I had been there more frequently, and there are cases where I feel I could just step in and fix it far more quickly, but that is not my role.

While I’ve concentrated on Agile processes and management (both product and more company) coaching we have also introduced more technical coaching. Jon Jagger and Nancy Van Schooenderwoert have helped out here. They have taken a similar light touch approach. In Jon’s case he normally spends his time pairing with the developers on his visit while Nancy works remotely.

There are some variations to the approach outlined above, things haven’t always gone smoothly and one company dropped out of the programme right after the training. But, on the whole it is working and working well.

Some successes to date:

One company develops a software product for doctors, they recently received ISO-13485 using an Agile process I helped them develop. ISO-13485, for those who don’t know, is a standard based on ISO-9000 which certifies a company to create medical devices.
The day after completing TDD training with Jon Jagger one team wrote a unit test which found a bug in their live system.
One manager when asked “So this estimation process and charts are more accurate than the way you used to do it?” replied “This isn’t estimation, that is Mystic Meg stuff, this is science, I know when we will deliver.”
By adopting Agile one company has won, and is successfully working on, a new website for one of the worlds largest telecom’s companies. The same company has now spawned a spin-off which is born Agile and will work with the same telco in a contract worth over a quarter of a million pounds.
The three companies I am working with have all been strong enough to hire new staff in this year.

I’m not saying that light touch coaching will work in every case - indeed, at one of the Cornish companies we decided they needed a more intensive programme. I do think that this is a model that can be used elsewhere.

Public training 18-20 May

2011-05-03T08:19:00.000+01:00

I’m giving a public training course at Skills Matter in London in a couple of weeks, 18-20 May. This is the Agile Foundations course specifically for Business Analysts. If you are not a BA I’m sure you will find it useful and interesting; you’ll just get a bit more on the requirements and Product Owner side of things than the vanilla version.

As it happens, I don’t give that many public courses, mostly they are private courses for specific companies or invited audiences - like the one I’m doing next week. So this is one of the few occasions to just turn up.

Quiet blog - Business Patterns for Software Developers in progress

2011-04-28T15:28:00.000+01:00

Just a brief note to explain why this blog has been a bit quiet lately.

Two reasons.

Anyone who has seen me in the last 5-6 weeks will know we had a big family event four 4 weeks ago. So a lot of my time is going in that directions.

And second, about the time, I finally signed the contract to write “Business Patterns for Software Developers.” The business strategy patterns I have been writing and bringing to conferences for the last six or seven years are finally going to be made into a book.

So, most of my spare time is absorbed by getting the book project rolling. Fortunately most of the text is written but there are still gaps. Plus there is a lot of editing and reviewing to do.

The text of the book is due to be finished in September and the book in the shops in January. Expect to see if on Amazon long before then, once the ISBN number is issued it is in the catalogues and will be picked up by online retailers.

Exciting times.

Slides from ACCU 2011 - 10 Step Model for Agile Requirements

2011-04-15T19:03:00.000+01:00

The slides from my presentation at ACCU 2011 conference today are available online, the Agile Requirements 10 Step Model.

Retrospective Dialogue Sheets

2011-04-15T10:52:00.000+01:00

I’m at ACCU 2011 this week, in my lightening talk I publicly unveiled, for the first time, Retrospective Dialogue Sheets.

I have been experimenting with dialogue sheets at conferences for a few years now. For a long time my aim has been to use them for team retrospectives. I’ve now produced a few variations and tested them with some teams. Still I need to do more tests, so I’m looking for volunteers.

I’m making these dialogue sheets available as a community resource, without charge. However, there is a catch: you need a large printer (A1 size) to print them. So I’ve set-up a print-on-demand service where you can buy printed versions.

Right now you can:

If you do try one of these sheets please e-mail me and let me know about your experiences.

Finally, I’m still looking for volunteers who will let me watch teams using these sheets. I’m happy to give teams printed dialogue sheets for free if you will let me observe the team doing the exercise. If you can help, please e-mail me.

Strategic Staffing, Or, How many people should I put on this (Agile) project?

2011-03-30T18:36:00.001+01:00

One of the questions that comes up again and again in the context of Agile work is “How do I know how many people to put on this project?”

(I’m using the word ‘Project’ in the loosest sense, think “piece of work”)

My usual answer is: Staff Strategically, allocate people to a piece of work in relation to how important this piece of work is relative to everything else you are trying to do. Keep teams together for as long as possible, only make small, gradual changes to teams over time.

This isn’t really a problem when work is underway, you simply sum up the ball-park estimates on the work remaining (in abstract points), measure the velocity (again in abstract points), divide total by the velocity and if you get an answer you like then all is well. If the answer is not soon enough you can (gradually) add more people, and if the answer is very good you might even slow down.

However, the problem really comes when you are starting a piece of work. Particularly if you are bidding on a contract to do work for someone else. The problem occurs because without any data (velocity, sum of outstanding work) you can’t estimate when you will be done, and without doing the work you can’t get any data.

Rule 1: Without data you can’t forecast anything so don’t pretend you can.

So, what do you do?

Option 1 is to pretend you aren’t doing Agile; use whatever methods and techniques you’ve used before, get an estimate (or maybe more of a guess) on how many people you need for how long then get started with those numbers. As soon as you start throw away the plans. Wing it for an couple of iterations, by then you have the data and can re-plan.

Not a perfect solution but one that could work.

Option 2 is to just start. Start as small as you can, I think two people is about as small as you can get - below that you are into the realm of micro-projects which have their own dynamics.

Let this mini-team run for a few iterations and then, as if by magic, you have data.

Once you have data you can: keep as is, grown, shrink or cancel entirely.

Rule 2: Start small, get data, grow later

Starting small is essential because it reduces risk and because it minimises the momentum which can make it difficult to cancel work.

It also minimises the influence of Conway’s Law: “every organization will produce systems which are a copy of the communication paths in the organization.”

If you start with a big team you will get a big project - allocate a C# developer, a DBA and a UI designer and you’ll get a three-tier architecture. If you start small you should get a small project.

In the longer term how you staff a project is less a function of “How many people do I need to do this work?” and more a question of “How many people can I afford to do this work?”

Projects are never really staffed on a “how many people do I need” basis, we just pretend they are. Writing today, March 2011, you will probably put more people on a piece of work than you would 18 months ago, but fewer than you would have three years ago.

Rule 3: Staff work strategically relative to the other work to be done and corporate priorities, rebalance only occasionally

Strategic staffing means looking at how any one piece of work stacks up against the other pieces of work in play - or proposed - and the resources that are available to use. It means considering: what is the benefit of a piece of work? What is the risk? What work absolutely must be done, and what work can be left undone? What work is experimental?

The trick is to arrange work so teams can grown, or shrink, as resources and priorities change.

That is not a recipe for changing team composition every week. Indeed, the general principle should be: keep teams together and consistent for as long as possible. Only change teams occasionally, and only grow teams slowly.

However, things do change and work needs rebalancing. This should be an active, measured, and occasional process. Better to change staff allocations once a quarter in a review meeting than every week as a knee-jerk reaction.

That is what strategic staffing is. If you find yourself changing staff allocations every few weeks in response to events you aren’t working strategically.

As for deciding how many people to put on a piece of work when you are bidding for a contract, well, its another argument for constructing contracts as ongoing, rolling contracts.

Telling your client: “We think we need six people for six months” is little better than lying if you don’t have data to back it up. Have these six people worked together before? How productive will they be with the client’s environment? With the clients requirements? What work might emerge?

Better to tell the client: “We propose to start with three people for three months, at the end of that time we will review progress with you and decide with you, when we have data, whether to increase or decrease staffing. If you want to move faster than we can allocate four people and review after one month.”

It might not be what your client wants to hear but it sure beats guessing, or lying. Involve your customer, give them choices. It might not be a great sales technique but if the client won’t engage in this conversation there are probably other conversations they won’t have either.

Final roundup of facts from Capers Jones

2011-03-23T10:08:00.001Z

In two previous entries I’ve reported some interesting statistics and findings - possibly facts - from Capers Jones book Applied Software Measurement (see Software Facts - well, numbers at least and More Facts and Figures from Capers Jones). I want to finish off with a few notes from the later chapters of the book.

On packaged software

Modifications to existing COTS package - I assume he includes SAP and Oracle here - is high-risk with low productivity
When changes exceed 25% it may be cheaper to write from scratch. (I’m not sure what it is 25% of, total size of the package, 25% of function points, 25% of features?)
Packages over 100,000 function points (approximately 1m lines of code) usually have poor bug removal rates, below 90%

On management

Jones supports my frequent comment that comes redundancies and downsizing Test and QA staff are among the first to be let go
Projects organised using matrix management have a much higher probability of being cancelled or running out of control
Up to 40%(even 50%) of effort is unrecorded in standard tracking systems

On defects/fault

“Defect removal for internal software is almost universally inadequate”
“Defect removal is often the most expensive single software activity”
WIthout improving software quality it is not possible to make significant improve to productivity
“Modifying well-structured code is more than twice as productive as modifying older unstructured software” - when he says this I assume he doesn’t mean “structured programming” but rather “well designed code”
Code complexity is more likely to be because of poorly trained programmers rather than problem complexity
Errors tend to group together, some modules will be very buggy, others relatively bug free
Having specialist, dedicated, maintenance developers is more productive than giving general developers maintenance tasks. Interleaving new work and fixes slows things down
Each round of testing generally finds 30-35% of bugs, design and code reviews often find over 85%
Unit testing effectiveness is more difficult to measure than other forms of testing because developers perform this themselves before formal testing cuts in. From the studies available it is a less effective form of testing with only about 25% of defects found this way.
As far as I can tell, the “unit testing” Jones has examined isn’t of the Test Driven Development type supported by continuous integration and automatic test running. Such a low figure doesn’t seem consistent with other studies (e.g. the Nagappan, Maximilien, Bhat and Williams study I discussed in August last year.)
Formal design and code reviews are cheaper than testing.
SOA will only work if quality is high (i.e. few bugs)
Client-server applications have poor quality records, typically 20% more problems than mainframe applications

Documentation

For a typical in-house development project paperwork will be 25-30% of the project cost, with about 30 word of English for every line of code
Requirements are one of the chief sources of defects - thus measuring “quality” as conformance to requirements is illogical

Agile/CMM/ISO

There is no evidence that companies adopting ISO 9000 in software development have improved quality
Jones considers ISO 9000, 9001, 9002, 9003 and 9004 to be subjective and ambiguous
Below 1,000 function points (approximately 10,000 lines of code) Agile methods are the most productive
Above 10,000 function points the CMMI approach seems to be more productive
I wold suggest that as time goes by Agile is learning to scale and pushing that 1,000 upwards

Jones also makes this comment: “large systems tend to be decomposed into components that match the organizational structures of the developing enterprise rather than components that match the needs of the software itself.”

In other words: Conway’s Law. (See also my own study on Conway’s Law.) Its a shame Jones missed this reference, given how well the book is referenced on the whole I’m surprised.

Elsewhere Jones is supportive of code reuse, he says successful companies can create software with as much as 85% reused code. This surprises me, generally I’m a skeptical of code reuse. I don’t disbelieve Jones, I’d like to know more about what these companies do. As has to be more about the organisational structure than just telling developers: “write reusable code”.

Overall the book is highly recommended although there are several things I would like to see improved for the next revision.

First, Jones does repeat himself frequently - sometimes exactly the same text. Removing some of the duplication would make for a shorter book.

Second, as noted above, Jones has no numbers on how automated unit testing, i.e. Test Driven Development and similar, stacks up against traditional unit testing and reviews. I’d like to see some numbers here. Although to be fair it depends on Jone’s clients asking him to examine TDD.

Finally, Jones is very very keen on function points as a measurement tool. I agree with him, lines of code is silly, the arguments for function points are convincing. But, I’m not convinced his definition of function points is the right one, primarily because it doesn’t account for algorithmic logic.

In my own context, Agile, I’d love to be able to measure function points. Jones rails against Agile teams for not counting function points. However, counting function points is expensive. Until it is cheap and fast Agile teams are unlikely to do it. Again, there is little Jones can do directly to fix this but I’d like him to examine the argument.

I want to finish my notes on Jones book with what I think is his key message:

“Although few managers realize it, reducing the quantity of defects during development is, in fact, the best way to shorten schedules and reduce costs.”

Humans can't estimate tasks

2011-03-17T13:27:00.001Z

As I said in my last blog entry I’ve been looking at some of the academic research on task time estimation. Long long ago, well 1979, two researchers, Kahneman and Tversky described “The Planning Fallacy.”

The Planning Fallacy is now well established in academic literature and there is even a Planning Fallacy wikipedia page. All the other literature I looked at takes this fallacy as a starting point. What the fallacy says is two-fold:

Humans systematically underestimate how long it will take to do a task
Humans are over confident in their own estimates

Breaking out of the fallacy is hard. Simply saying “estimate better” or “remember how long previous tasks took” won’t work. You might just succeed in increasing the estimate to something longer than it takes to do but you are still no more accurate.

Curiously the literature does show that although human’s can’t estimate how long a task will take, the estimate they produce do correlate with actual time spent on a task. In other words: any time estimate is likely to be too small but relative to other estimates the estimate is good.

Second, it seems the planning fallacy holds retrospectively. If you are asked to record how long you spend on a task you are quite likely to underestimate it. There seems no reason to believe retrospective estimation is significantly more accurate than future estimation.

Something else that comes out of the research is: psychologists and others who study this stuff still don’t completely understand what the brain is up to, some of the studies contradict each other and there are plenty of subtle differences which influence estimates.

Third, although we don’t like to admit it deadlines play a big role in both estimating and doing. If a deadline is mentioned before an estimate is given people tend to estimate within the deadline. People are also quite good at meeting deadlines (assuming no outside blocks that is), partly this is because estimating then doing is a lot about time management. i.e. managing your time to fit within the estimate.

While deadlines seem like a good way of bring work to a conclusion it doesn’t seem a particularly good idea to base deadlines on estimates. Consider two scenarios:

Simple estimate becomes a deadline: If we ask people to estimate how long a piece of work will take they will probably underestimate. So if this estimate is then used as a deadline the deadline may well be missed.

Pessimistic estimate becomes deadline: If people are encouraged, coerced, or scared into giving pessimistic estimates the estimate will be too long. If this estimate is then used as a deadline work is likely to be completed inside the time but there will be "slack" time. The actual time spent on the task may be the same either way but the total elapsed (end-to-end) time will be longer.

I’ve written my findings down in full, albeit somewhat roughly, together with a full list of the articles I examined in depth. “Estimation and Retrospective Estimation” can be downloaded from my website.

I would love to spend more time digging into this subject but I can’t. Anyone else want to?

Apology, correction and the Estimation Project

2011-03-17T09:28:00.000Z

Over the last few months I've been thinking about how people perform estimates when developing software. I've take time - not enough - to look through some of the research on estimation and try and understand how we can improve estimates.

I’ve written a long (11 pages, over 6000 words), and rough, essay recording some of my findings. You can download “Estimation and Retrospective Estimation” from my website, I don’t intend to publish the essay anywhere but I expect some of the findings to be included in some future pieces. In this blog entry and the next I want to report some of the motivation and findings.

To start, a correction and an apology.

In January Jon Jagger sat in on one of my Agile training courses. Jon asked me about two references I appeared to cite. I have failed to be able to support these references with the necessary research and I must apologise to Jon and others who heard me say I could.

Now if someone, me or anyone else, goes around saying “there is evidence for this” or “this is backed by research’ they should be able to back it up. I have failed on this count. I apologise to Jon and everyone else who has heard me say these things. I can't track down my references. I should not have claimed there is research which I can't provide a reference for.

I’m going to repeat my claims now for two reasons. First, I’m hoping that someone out there will be able to point me at the information. Second, if you’ve heard me say either of these things and claim there is research to back it up then, well I apologise to you too. And if you hear anyone else say similar things, please pin them down and find the original source.

The first thing Jon pulled me up on was in the context of planning poker. I normally introduce this as part of one of my course exercises. Jon heard me say “Make your estimates fast, there is evidence that estimates fast estimates are just as accurate as slow ones.”

Does anyone have any references for this?

I still believe this statement, however, I can’t find any references to it. I thought I had read it in several places but I can’t track them down now. I’m sorry.

Thinking about it now, I think I originally heard another Agile trainer say this during a game a planning poker a few years ago. So its possible that others have heard this statement and someone has the evidence.

Turning to my second dubious claim, I said: “I once saw some research that suggests we are up to 140% in judging actuals.” It is true that I read some research that appeared to show this - I think it was in 2003. However, I can’t find that research so I can’t validate my memory, therefore this is not really valid.

By "actuals" I mean: the actual amount of time it took to do a piece of work.

Again, does anyone know the research I have lost the reference to?

And again, I apologise to Jon and everyone who has heard me state this as fact. It may well be true but I can't prove it and should not have cited it.

Why did I say it? Well, I find a lot of people get really concerned about the difference between estimates and actual, whether some task was completed in the time estimated. To my mind this is a red herring, actuals only get in the way.

This is where things start to get interesting. I decided to look again at the research on estimation and specifically recording “actuals.” The first thing I learned was that what we call “actuals” is better called “retrospective estimation.”

The “Estimation and Retrospective Estimation” essay record my journey and findings in full. In my next blog entry I’m record some of my conclusions.

Why Quality Most Come First & a comment on the 'System Error' report

2011-03-10T10:02:00.000Z

On Monday night I presented “Why Quality Must Come First” at Skills Matter in London, the slides are now online (previous link to my website) and there is a pod-cast on the Skills Matter website.

This was a revised and updated of my talk at the Agile Business Conference last October. There are two key arguments:

High quality is essential to achieve Agile: if you have poor quality you can’t close an iteration, you can’t mark work done, and schedules will be unpredictable.
High quality (fewer bugs) is the means to achieve shorter delivery time: better and faster are not alternatives, they not even complementary, they are cause and effect.

That second point is one that I don’t think is appreciated enough, in fact I think a lot of people have a belief that you can deliver faster if you turn-down the quality. Many years ago I worked with a great developer called Roy. One day Roy went to our manager, the conversation went something like this:

Roy: Would you rather have the software sooner with bugs or later without?
Manager: Sooner with bugs

Both Roy and the Manager were seeing a trade-off that does not exist.

Resetting this broken mental model, unlearning this trade off is one of the biggest challenges that faces out industry.

Read my lips: High quality software is faster to develop. Invest in quality and you will finish sooner.

This is also one of the reasons I prefer the XP approach over the Scrum approach. It is also one of the reasons I was disappointed with the “System Error” report from the UK Institute for Government last week.

The reason the report got so much attention was because it recommended the UK Government adopt Agile and Iterative development in IT projects. By the way, the report is from a think tank, it is wrong to say this report means the UK Government will adopt Agile, or that the UK Government wants to adopt Agile, it just means some “thinkers” suggest it should.

I haven’t had a chance to do anything more than skim the report yet but what struck me was the lack of discussion about quality. The report continues the belief that quality is an effect not a cause.

The report lays out four principles for Agile projects:

Modularity
Iterative approach
Responsiveness to change
Putting users at the core

There is one big thing missing here: Quality.

Without high quality you can’t take an iterative approach because nothing is ever finished.

Without high quality you can’t be responsive to change because nothing is deployable and over time crud builds up which slows you down.

Without quality your users will not trust you, they will not believe you have changed, and will spend a lot of their time. Pushing bad buggy software one to users is not a sign that you have put them at the core.

Leaving aside the fact that Modularity is one of the most over used and consequently meaningless words how can anything buggy be modular? Do you want to (re)use a buggy module? Bugs are side effects, not something you want in modular code.

The Agile 10 Step Requirements Model

2011-02-22T14:46:00.000Z

I’ve been talking about the Agile 10 Step Requirements Model for a year or so now. It has appeared in a few conference presentation and will be at the centre of my ACCU 2011 conference presentation. But I’ve not written about it.

Now I have. This months ACCU Overload has a short overview of the model. I say short because each time I’ve sat down to write down the model it has got very long. This was a deliberate attempt to keep it short.

You can ready it for free in Overload, or I’ve put the article itself on my website, The Agile 10 Step Model.

Many thanks to Ric Parkin and the Overload team.

How does Agile relate to CMM Level 5?

2011-02-20T14:04:00.000Z

A question in my mail box: “How does Agile relate to CMM Level 5?”

As I started to tap out the answer I thought: this might as well be a blog entry. So here it is.

Think of CMM, or rather CMMI which replaced CMM about 10 years ago, as a ruler. It is a way of measuring software development effectiveness. Is your development process 1cm good? 2cm good? or 5cm?

CMMI doesn't say how you should do development, only how you should judge effectiveness. Agile is a way of doing things - as opposed to "the state of Agile" which is a measure of effectiveness.

You can use Agile to be any CMMI level you like. CMMI doesn't care how you do it. Watts Humphrey (the father of CMM) used to say something along the lines of “Work on you quality and processes first, the levels will come by themselves.” That statement is very Agile. Unfortunately some organisations get it the wrong way around. I was at Reuters when they imposed CMM and destroyed a big chunk of their development capability.

The late Watts Humphrey did issue some process recommendations in the form of the Personal Software Process and Team Software Process.

That’s the easy bit.

"The state of being Agile" might, or might not, conflict with Level 5 CMMI simply because different things are considered important. You might be CMMI level 5 and decidedly unAgile.

Ironically CMMI level 5 might mean you have to investigate Agile. Because being level 5 means you are “self optimising”. If an organisation is level 5 and hasn’t looked at Agile it should because that may help them improve.

Paradoxically, being level 5 itself makes it becomes harder to improve. The risk of change is a lot greater because the organisation has more to loWaterfallse - and probably lots of procedures to update making any change expensive.

CMM(I) tends to be associated with a certain way of doing things. Partly this is historic: when CMM(I) appeared Waterfall was the dominant model, NASA was the first organization to reach level 5 and (at the time) it was very Waterfall based, and because CMM(I) tends to be more common in military work which are also big and paper work intensive.

So, some people believe that CMM(I) means following a very structured, heavy, WaterfallWaterfall process. It doesn’t have to mean this but historically the two do tend to coincide.

The Software Engineering Institute issued a report in 2008 which discussed how CMMI and Agile could work together: CMMI or Agile: Why not embrace both! I don’t agree with everything in the report but I do agree with the general tone. CMMI and Agile are not alternatives and can be complementary.

InfoQ: Agile contact options

2011-02-08T17:43:00.000Z

I have a piece on the InfoQ website today entitled Agile Contracts (well options for really).

Offshoring: not as simple as its often reported

2011-02-04T12:36:00.001Z

A couple of footnotes to my last blog post on Capers Jones book, Applied Software Measurement. One of the points I noted was Jones suggestion that rising prices and costs in India and elsewhere would mean IT offshoring loose its financial benefits from about 2015 onwards.

Jones, by the way, later says that Indian and Chinese outsources are aiming to compete on quality not price in the long run. So we can expect to see the nature of offshoring change.

Shortly after posting that blog entry I noticed a piece in The Economist on the Indian ID programme, Identifying a billion Indians (27 January, subscription required.) The Indian Government is embarked on a scheme to give unique identity numbers to the entire population.

The first thing that first got my attention was not that this was an IT based scheme (no surprise there) but who was doing it. Not Tata (TCS), not Wipro, not Infosys. It is Accenture and L-1 of the US and Morpho of France. I’m sure much of the work is being done in India but the Indian Government has given the work to foreign firms. Lets give a round of applause to the Indian Government for not feeling compelled to give work to local firms.

And then lets note that offshoring/outsourcing goes both ways. Its not all about US and Europe loosing out to India or China. It goes the other way too.

The second thing that got my interest was this: the work is split between these three firms: 50%, 30%, 20%. Progress is reviewed regularly and work reallocated. The most effective firm during any one period gets the bulk of the work next time. That is a very enlightened way of working and gives real incentives the firms to do their best. It also shows flexibility with contracts.

Separately, some US readers might be please to learn that Financial Times reports that the big Indian outsources plan to do less work with US firms. European readers won’t be so happy to learn that Europe will replace the US as their target.

The reason: not competition, not prices or quality. US Visa prices.

More facts and figures from Capers Jones

2011-02-02T15:07:00.000Z

I continue my reading of Capers Jones Applied Software Measurement as discussed a few entires ago - Software Facts and I’d like to report some more of Jones findings.

These numbers are very insightful but, and its a but Jones acknowledges, the data is very shaky. As Jones says “software measurement resembles an archeological dig. One shifts and examines large heaps of rubble, and from time to time finds a significant artifact.”

He readily admits that there is not really enough data to make solid conclusions (less than 1% of the data needed is available) but this is where we are at. This is as good as it gets. Before anyone rushes to say “Capers Jones is wrong” ask yourself: “Do I have better data to make conclusions one?”

I find two weaknesses in Jones work. Firstly his reliance on Function Points. I agree with him that lines of code is not a good measure but I have some doubts about function points for two reasons.

They do not incorporate algorithm complexity
Automating function point counting is difficult and becomes an approximation. Thus true function point counts are expensive which makes them difficult to work to calculate often

But, function points, for all their faults seem to work.

My second issue with Jones work concerns his assumption of the waterfall model. Yes he acknowledges Agile, he even says it is the most productive approach in some circumstances but all his data and assumptions are cut through with the waterfall. I suppose this is only natural, it was (is) the dominant model in the industry for 30 years or more.

OK, on with some numbers and conclusions, again these are almost entirely US based but are probably very similar elsewhere....

Productivity

Jones repeatedly states and shows how quality and productivity are related. The most productive teams have the lowest bug counts and shortest schedules.
The three biggest costs on a software project, in order: Rework (Bug fixing), paperwork and meetings & communication. Sometimes managerial costs are actually greater than coding costs.
Jones states as little know fact “excessive staffing may slow things down rather than speed things up”
Inspections (code, design, requirements) are the most efficient known ways of preventing problems.
Studies show that some developers are 20 times more productivity than others, and some make 10 times as many errors as others.

Costs

Paperwork (requirements, technical specs, etc.) on the largest systems can be larger than one person’s ability to read during an entire career. (Think about that: you join a new project at 21, by the time you hit retirement you haven’t finished reading the documentation.)
Most corporate effort (time and money) tracking systems are incorrect and miss between 30% and 70% of the real effort.
Thus this data is essentially useless for benchmarking, creates cost over runs because they are so inaccurate and is actually dangerous because it puts future projects at risk when the data is used for estimates. (Before you question this statement go and check, does your tracking system measure unpaid overtime?)

Projects

“More than half of large projects are scheduled irrationally” - a delivery date is set external to the development group without reference to capabilities.
Over 50% of large (more than 10,000 function points) projects are cancelled and the rest are almost always late and over budget.
In 2008 at least 25% of new projects used some elements of Agile.
Maintenance is now the dominant form of software development and is likely to stay that way.
There are several Y2K like problems likely to occur in the next 40 years.
Even waterfall projects overlap phases - typically 25% of any phase is incomplete when the next starts. This rises to 50% when the handover from design to coding is considered.
It is even difficult to determine when a project really starts - less than 1% have certain start dates - and 15% have ambiguous end dates.

Outsourced & Offshore

Outsourced software development companies are generally better than their clients and have a better record of delivering large systems.
Offshore work tends to be less product than onshore work in the US - that does not mean it is not cost effective but it is less productive in terms of function points per man/time period
Inflation and other cost changes mean that by 2015 the economic advantages of offshoring development work are likely to be gone. (Considering that it takes time to transfer work and offshore teams to become productive it is likely that very soon it will not make sense to send work overseas.)
Worryingly Jones reports that 75% of companies have deficient requirements practices. Since offshore work is particularly dependent on good requirements I would expect offshore projects to be more troublesome in this regard.

Standards and Quality

Jones finds no solid evidence that ISO 9000 improves quality or the ability to deliver; almost the opposite it does increase the cost of projects because it increases the amount of paperwork. Ironically some ISO 9000 advocates have defended the approach to him on the grounds that ISO 9000 is not designed to improve quality! (Sounds like many people have been misled over the years)
Military systems are the largest systems in the world, some over 350,000 function points. They are also the most expensive and usually paperwork driven.

Department of Defense (US) standards do not seem to improve software quality, they have sometimes been impediments and have certainly raised costs.
Microsoft, Oracle and SAP are singled out as having very poor quality control.
Interestingly he also suggests that once 25% of a large COTS application needs to be customised it is less expensive to build the same application from scratch.

Jones says that as of 2008 there were 700 known programming languages with one a month appearing. The average life of a language is 15 years but the average life of a major software system is 20 years. These figures might be a little misleading, C is now nearly 40 years old, other languages survive only a few. But the point is clear: there are lots of dead languages and orphaned languages out there (over 500 Jones thinks.)

As noted before, most schedules are planned with an externally determined date. Jones gives this as the number one reason for project failure - number two is excessive schedule pressure which is very similar. Later he proposed schedules are so bad as to constitute professional malpractice.

This is interesting. One the one hand businesses need those deadlines. On the other, if projects were left to run their natural duration would the failure rate be so high?

He also comments that US development teams tend to be strong on technical skills but are let down by weak management and organisational issues. Later in the book he attributes much of this down to western managements pre-occupation with cost control which he contrasts with eastern concern with quality.

This argument seems reasonable but I’m not sure it stands up to analysis. Although it might explain why Scrum can be “successful” even when technical practices are ignored.

In my experience US and European teams don’t always have the technical skills they should have - certainly talk to Steve Freeman or Jason Gorman and you will hear some horror stories. While India has more CMM certified organisations than anywhere else my personal experience is, while India has some very good developers the IT bubble has attracted in not just second rate developers but third rate ones as well. Its also worth pointing out that Japan isn’t a software powerhouse, Toyota even struggle to apply Lean thinking to software.

Jones believes, from his studies, that management is the greatest contributor to success and failure at the project and company level. In some ways that is reassuring, it shows that there might be something in good management.

Interestingly Jones also supports one observation I have frequently made myself: when redundancies occur Test and QA staff are usually the first to be let go, and this is counter productive in anything except the very short tun.

I’ll continue reading and maybe blog some more data. In general, the Applied Software Measurement has a tendency to repeat itself and I wish he used more graphs for his data but I still think it is worth reading and I recommend it.

Scrum + Technical practices = XP ?

2011-01-25T08:54:00.001Z

My last blog post (Scrum has 3 Advantages Over XP) attracted a couple of interesting comments - certainly getting comments from Uncle Bob Martin and Tom Gilb shows people are listening. And it’s Tom’s comment I want to address.

But before I do, lest this come across as Scrum bashing I should say that I think Scrum has done a lot of good in the software industry. Yes I had my tongue-in-my-cheek when wrote “Scrum has 3 advantages over XP” but if Scrum hadn’t made “Agile” acceptable to management it is unlikely that XP ever would have take Agile this far.

I suppose I’m a little like Uncle Bob, its a bit of a surprise to me that Scrum Certification has come to mean so much to the industry when really it means so little. I’m also worried that brands poor people “certified”, good people “uncertified” and stifles innovation.

Second the ideas of the Scrum originators are good. And lets not forget that while Schwaber and Sutherland have become celebrities in the community there were five authors of the original “Scrum: A Pattern Language for Hyperproductive Software Development”, Beedle also co-authors one of the early Scrum books, Devos still works as a Scrum Trainer, while Sharon seems to Scrum’s Fifth Beatle - where are they now?)

Getting back to the point. Tom Gilb commented: “Sutherland did say in a speech that Scrum works best with XP”. Chris Oldwood also says he’s heard Jeff say this.

In fact, I think I’ve heard Jeff Sutherland say something similar, heck, not similar, the same and reported it in this blog. And Jeff’s not alone, I’ve heard the same thing from other Scrum advocates, trainers, and cheer-leaders.

But, this view is not universal. There are Scrum trainers who don’t. Specifically there are those who train Scrum and specifically say XP practices like TDD and on-site customer should not be used. And as I sarcastically commented previously, part of the attraction of Scrum to management types is that it doesn’t delve into the messy world of coding.

Lets just note: there are multiple interpretation of what is, and is not Scrum. Nothing wrong with that, it just complicates matters.

Back to XP though....

There are, broadly speaking, two parts to Extreme Programming (XP). The technical practices (Test Driven Development, Continuous Integration, Refactoring, Pair Programming, Simple Design, etc.) and the process management side (Iteration, Boards, Planning Meetings, etc.)

It is these technical practices that Jeff, and others, are saying Scrum software development teams should use. Fine, I agree.

Look at the other side of XP, the process side. This is mostly the same as Scrum. Do a little translation, things like Iteration to Sprint and it is the same.

(OK there are a few small differences. I just checked my copy of Extreme Programming (old testament) and to my surprise stand-up meetings aren’t there. I think many XP teams do stand-up but it isn’t originally part. The book does include “40 Hour Work week” which isn’t present in The Scrum Pattern Language, or Agile Project Management with SCRUM but I’ve heard Scrum advocates talk about sustainable pace. I’m still not sure how you marry this with commitment but I’ve written about that before. However these points are knit picking.)

Essentially Project Management XP style is Scrum.

This shouldn’t be a surprise. In the mid-90’s when Scrum and XP were being formed and codified many of these people knew one another. If nothing else they saw each other’s papers at PLoP conferences. The Scrum Pattern language was presented at PLoP 1998, Episodes (the basis for XP) was at PLoP 1995, and both draw on the work of Jim Coplien.

I have every reason to believe that XP and Scrum didn’t appear in insolation. There was cross-fertilisation. I have also been told there was competition.

If you accept that Scrum and XP project management are close enough to be the same thing, and you accept Jeff Sutherland’s recommendation (namely do XP technical practices with Scrum) then what do you have?

XP = Technical Practices + Scrum Style Project Management
Jeff Sutherland says effective development teams = Scrum + Technical Practices from XP

Think about that. Even talk about it, but don’t say it too loudly. For all the reasons I outlined before, XP isn’t acceptable in the corporation. Scrum is. Therefore, we must all keep this little secret to ourselves.

Scrum has 3 advantages over XP (but XP is better)

2011-01-21T15:40:00.000Z

For corporations Scrum offers three advantages over Extreme Programming:

1) It offers certification - corporates can hire people who are “certified” and have their own people certified.
2) Scrum traces its roots back to the Harvard Business Review - so it must be serious
3) Scrum does not contain the words “extreme” or “programming”. No need to dirty our hands with messy code, keep that stuff as far away as possible (the further the cheaper, right?)

On the other hand, XP has two advantages over Scrum:

1) It actively seeks to address the quality problems most software development teams suffer from and which cripple productivity
2) It excites the people who actually do the work - dispensing with “resistance to change” in one fell swoop

At Agile Cambridge last year I saw a really good presentation from people at GE Energy about their Agile adoption. Frankly I’m sceptical about the ability of any large organisation to adopt Agile but these guys had some real success to show.

Two things stuck in my mind. When summing up the presenter said:

“If I was doing the same thing again I would start with the XP technical practices not the Scrum process” and he went on “We have adopted Scrum, we are now advancing to XP.”

XP and Scrum date from about the same time (mid-90’s). Perhaps because XP become popular first and Scrum later succeeded it as the Agile-poster-child many people seem to think Scrum is superior, or at least a superset of XP. Actually, the reverse is true.

XP is a superset of Scrum, and, in my humble opinion, superior.

Software Facts - well, numbers at least

2011-01-14T18:02:00.000Z

About a year ago I needed some numbers about software development - industry norms really: effectiveness, productivity, bug counts etc. etc. Its actually pretty hard to get these numbers and after hunting around I found myself with a copy of Capers Jones Applied Software Measurement.

Jones, for those who don’t know, has made a career out of analysing software and software teams numbers. Applied Software Measurement is his latest work on the subject - although its nearly three years out of date.

Unfortunately, for me, Jones didn’t have the numbers I wanted - I forget what I wanted now. But he does have lots of interesting facts and numbers. You can dispute some of these numbers, you can argue with him but on the whole he knows more about this subject than you or me so he’s probably right. That said, there are a few points where I disagree with him and I might blog about these in future.

For now, I’d like to share some of the number in Applied Software Measurement. Overwhelmingly Jones researched American companies and American teams. I expect most of these numbers will be broadly the same in Europe and elsewhere. All figures are for 2008 unless specified.

Productivity

Broadly speaking, in the USA, software productivity rates were the same in 2008 as they were in 1988
The best US companies in terms of productivity and quality are about three times better than average (US)
The worst US companies for productivity and quality are about 50% worse than average
Only about 15% of companies are improving productivity and quality
Similarly, about 15% of companies are getting worse in these terms
About 70% of companies are neither improving or getting worse
Where productivity is improving it is usually teams developing web applications who use Agile methods. Some benefits also accrue from use of Team Software Process and Personal Software Process
Claims by tool vendors of 10-to-1 or 20-to-1 displacement of human activity are generally not justified. It is counter productivity to invest in tools before resolving organisational and methodology issues. i.e. save the tools until you’ve improved things (John Seddon should be happy with this finding.)
Office environments can have as much of an impact on productivity as tools and methods
Looking after employees well increases productivity and reduces turn-over
Companies providing 10 day or more of training per employee per year have higher productivity rates than similar companies who do not
Accounting errors (and use of unpaid overtime) can hide the true cost of software production by 100%, i.e. the work costs twice as much as the accountants say
Formal design reviews and code inspections are effective but can fall into disuse because (new) managers do not understand this. (Tom Gilb is right!)
“Software does not age gracefully, and tends to become increasingly unstable and difficult to modify safely. The Unites States and indeed much of the world are facing some huge geriatric problems with their inventories of ageing and decaying applications” - maintenance, renovation and enhancement are now the dominant activities in the industry and it is likely to stay that way
On average military software is 25 times larger than end-user software, and the specifications and other documentation 200 times larger.
Productivity and quality seem to be better in object oriented languages

Documentation & Bugs

Producing paper documents for software development is more expensive than producing software itself
Up to 400 words may be written in specification for every line of code in large systems.
In a large software project requirements change and increase at the rate of 2% per month
Repairing defects in software is more expensive than coding and producing documentation
The finding and fixing software defects (bugs) is historically the most expensive activity and will account for 50% of total costs over the product life time
Approximately 7% of changes and fixes contain new defects
Most forms of testing finding fewer than 30% of all bugs

Projects

Over 50% of all software projects are one-man projects
Productivity and quality are directly coupled: projects with high quality have high productivity and vice versa
IBM was the first company to discover the link between quality and productivity. They made the information public in 1975 but it is still not widely know.
User satisfaction and defect counts are also directly correlated

I’ve mined these numbers out of Applied Software Measurement, I might mine some more and blog again. But really, it isn’t a book of numbers. Its a discussion of how to measure software and software teams.

Finally for now, I’m only at chapter 3, Jones says that currently available data on software production is less than 1% of what is needed. He also says that most databases of such data are privately held and not accessible by most of use. Well, I guess that explains why I couldn’t find the numbers I wanted when I wanted them.

Announcing EuroPLoP 2009 Proceedings in print

2011-01-07T14:29:00.000Z

A long long time ago, well summer 2009 I was Programme Chair of the EuroPLoP conference. One of my responsibilities was to produce the conference proceedings. For the first time we published these proceedings online using the CEUR repository.

Now another first. I am pleased to report we - well me actually - have made the proceedings available in print using the Lulu system.

So, roll up, roll up, get your EuroPLoP 2009 Patterns Conference Proceedings for just £13.54 in print, or free as a PDF.

This is something of an experiment. In the past EuroPLoP had printed proceedings but these were only available if you actually attended the conference or knew someone who did. This limited their circulation.

Secondly, the proceedings were paid for out of the conference budget which added something like €100 to the price of attending.

Whether future EuroPLoP conferences continue to publish on Lulu, or CEUR, is a matter for debate. If you have a view let me know.

Inbound & Outbound marketing

2010-12-28T12:05:00.000Z

Its nearly the end of the year, I’m about to close one Blog archive and start another - don’t worry, this means nothing for readers, its part of my admin. But it does mean that I put behind me all those half written blog posts, and single line ideas, that have been building up during the year.

Before I do so though, there are a couple of days to quickly get important backlogged entries out!

Two weeks I blogged, not for the first time, about Product Manager (Product Management an open secret, a differentiator). When I talk to people about Product Managers I often find myself in a conversation about marketing. Clued up people realise that good Product Management is, in part, a marketing function, others are surprised. They think marketing is about advertising and sales.

It is important, very important, to differentiate between the two sides of marketing: Inbound and outbound. So, for the record, I want to record it:

Outbound marketing: is the marketing most people think of when someone days “marketing.” This is about letting people know you, and your product, are here. Its advertising. Its public relations. It is generating sales leads. It is improving awareness. You get the picture?

It is called outbound because it is from you and your organisation to the wider world.

Inbound marketing: is from the wider world (customer, potential customer, competitors) to your company. It is about bringing that information inside and then acting on it. It is about building the products your customers want to buy. Get it right and sales should be like a knife through butter, people want your products.

In software companies, and other tech companies inbound is largely the role of Product Management. Product Marketing is about outbound. Although Product Manager is the usual job title some places call this role an Architect, others Programme Manager, and occasionally Project Manager. More often or not it is just absent.

Most companies realise, sooner or later, that outbound marketing is needed. Unfortunately, in all too many companies inbound marketing does not exist. Or inbound is “what the sales guy says the customer wants.”

As much as we love sales people (after all, they bring in the stuff that pays us) they are not the best people to decide what goes into a product. The reason why they are good at selling is that they get over customer objections and sell the product. This means the customer, the next customer, the next sale, is king. What the next customer wants isn’t what every customer wants.

Lets make this simple: if you are running a software product company and you don’t have a Product Manager then get one.

ACCU 2011 Conference registration open - beat the VAT rise

2010-12-23T10:02:00.000Z

Just in time for Christmas, that special present for someone you love...

Buy them a place at the ACCU Conference in April.

If you live in the EU you have an very special reason to buy in the next few days: the UK is increasing the VAT rate so the price is going up on 1 January.

If you really want to save money, and you are not already an ACCU member then join the ACCU now, and immediately book the conference. It is cheaper, honest; no, its not an accident, it is meant to be like this.

I’ll be there, in fact I’m speaking, I have two sessions, one on requirements and one on patterns.

(Twitter tag for this is #accu2011)

Dialogue sheets, retrospectives and quotes (Send me your quotes)

2010-12-22T11:40:00.000Z

Know any good quote about software development? Agile? Lean? Teams? Code? or anything else in this space?

I’m working on a little project and I need some though provoking quotes so please send them over - leave a comment here, e-mail me (allan at allankelly.net) or Twitter me (allankellynet).

Why?

Well, I’m working on a little project for retrospectives I’ve been dreaming of for a few year and I need some thought provoking quotes.

There are two problem I see with retrospectives - OK, there are other problems but there are two I think I can do something about. (And none of these problems undermine the reason for doing retrospectives or the value, just fly-in-the-ointment stuff.)

Firstly, retrospectives need a facilitator. That might not sound like a problem but ideally you want the facilitator to be a) experienced, b) impartial, c) available. Sometimes its not easy to find that person.

Second, I hear of teams who hold regular retrospectives but the retrospectives because boring, they go over the same ground again and again. To be effective, to get new insights they need jazzing up once in a while.

One way to resolve both problems is to use an outside facilitator once in a while but that needs organization and, most likely, money.

So, what if you could have a retrospective without a facilitator?

Over the last few years I’ve been experimenting with a technique called a Dialogue Sheet. Both in training and conference sessions I’ve used this to facilitate discussion. For more about dialogue sheets see my website with examples from XP Day, SPA and “Lean into Action” at the Agile Lean Exchange last month.

Now I’m working on a Retrospective Dialogue sheet. The idea is a team us this for a facilitator-less retrospective.

I hope to have the sheet ready by New Year and then I’ll be looking for some guinea pigs...

And the quotes: these get placed around the dialogue sheet to provoke thought and reflection.

When I’ve got this working I hope to allow you to download these sheets from my website or order printed versions directly.

Product Management an open secret, a differentiator

2010-12-14T20:48:00.000Z

At the Skills Matter Agile Lean Kanban exchange the other week someone - sorry I missed you name - told me about a report from the BBC on Product Management. It turns out the report is from a branch of the BBC I didn’t know about, “BBC Academy” and it entitled “The State of Product Management 2010.” Its well worth reading if you have an interest in Product Management or the UK software development scene.

Although I’ve not blogged about it for a while Product Management is one of my passions.

Think of the development team as the beating heart of the work effort. Someone has to keep the blood flowing, someone has to keep the arteries clear and make sure the right requests for work can channelled to the development team.

This is a role. Someone needs to do this. Someone needs to understand customers and be in a position to prioritise. And someone needs to be able to have ongoing conversations with customers, developers and everyone else.

This is not a document, attempts to do this with a document makes things worse. Documents don’t have conversations. Documents are static.

One of the reason why Silicon Valley companies are so successful is that they understand this. In Silicon Valley there is a well developed role called the Product Manager. This is the person who looks at the customers, understands their problems and needs, looks at the market and a whole bunch of other stuff, and this person decides what is needed for a successful software product.

I point to Silicon Valley because certainly in Europe, and I suspect in much of the rest of the USA, this role isn’t so widely recognised.

This role doesn’t exist in corporate IT. Here the Product Manager’s cousin the Business Analyst fills a similar role but in a very different way. However, there is change afoot here, stay with me.

Here in the UK I find I have to explain this role to clients again and again. They make four common mistakes:

They assume the development team just know what is needed. Sometimes they do, sometimes they don’t. They may get luck once but can you rely on luck the second time? Third time?
They assume that the Project Manager knows. Again this is wrong, Project Manager training is about delivering a defined project to a date within constraints. It is not about understanding customers. (True there is overlap but Project Managers training is different.)
Experience IT Managers see the need and appoint a Business Analysts. This is the right thing to do when you are a corporate IT department, or if you are writing bespoke software for one customer but, if you are trying to create a product to sell to many customers it is the wrong choice.
Managers assume it is obvious, or that they, the “managers” know. If they are Product Managers or have been Product Manager they might be right. Even if they do know what the customers want someone has to explain it to developers and be on hand to answer their questions. Senior Managers just don’t have the time.

So its good to see the BBC highlighting this role. The description in this report is very media centric but it is recognisably the Silicon Valley Product Manager role. There are some good case studies in the report.

As I said, over in the corporate IT world this role doesn’t exist because customers are captive. One of the key aspects of the Product Manager role is understanding what customers (who have a choice) will pay for. In the corporate IT world this doesn’t exist: you get what you are given. So the BA role is similar but different. (There are other difference but they are for another day.)

(Interestingly several of the Product Managers quotes in the BBC report state they have Business Analysis backgrounds.)

The thing is: Business Analysts need to become more like Product Managers because more and more corporate IT departments are creating systems for external customers who do have a choice.

For example, 15 years ago just about all the IT created by travel companies was for their own use. Today travel companies have customer facing IT systems which can both win and loose the company sales. Online offerings are part of the buying experience, part of the customer experience and a potential product differentiator.

Even though the Product Manager role is not as well understood as it should be it is becoming more important. For those who get it Product Management is a potential differentiator.

Return to Requirements @ Skills Matter

2010-12-12T14:55:00.000Z

I’m presenting a Skills Matter In the Brain Session this week, entitled “Return to Requirements.”

The key thing about this presentation is that it give a very very brief overview of the “Agile 10 Step” which is the model I use to discuss requirements in Agile work.

It is Tuesday 14 December, 6.30pm to be exact. Its free but you need to registration is required.

Salami Agile

2010-12-01T09:29:00.010Z

More than one software development team has encountered the situation when the team want to be more “Agile”, the organization and management might even be asking them to be more “Agile” but, there are still many “requirements” in a big requirements document and the expectation is that all these will be “delivered.”

Ideally I would not set up and Agile development like this. Yes I’d set a few requirements to seed the first iteration but I would take a more goal directed approach. I’ve written about this elsewhere - see Time for Goal Directed Projects.

Basically Goal Directed means: the team have a goal, the team will determine what needs doing (requirements) and do it (implementation) as part of the same project. The team will be staffed with everyone they need to do both sides of the work (BAs, Developers, Testers, etc.) and will be judged by their success on reaching the goal. They will not be judged on how many features they implemented from some big initial set.

But, as I started by saying, some teams do have a shopping list of things they are expected to deliver and they will be judged on how many of those items are delivered and when.

So what do they do?

This is where Salami Agile comes along.

Think of the Big Requirements Document as an uncut piece of Salami. Someone on the team - preferably someone with Business Analysis skills but it could be a developer, project manager, or someone else - needs to slice the requirements into thin pieces of salami (story) for development.

There is no point in slicing the whole salami in one go. That would just turn a big requirements document into a big stack of development stories. The skill lies in determining which bits of the document are ready (ripe) for development, which bits are valuable, and which bits can be delivered independently.

Some slices of salami will need to be thicker than others but thats just the nature of the world. Over time, with more skill at slicing salami it will improve.

Ideally, the pieces of salami are delivered to the customer early, and over time they start to realise they don’t need some parts of the requirements document, some salami can be left unsliced and simply thrown away. Other pieces of salami will be cut and then thrown away. Thats not failure that’s reducing the work and is good.

And some requirements which were not though of can be easily incorporated. To stretch the analogy, you switch from German Salami to Danish Salami and back again if you so choose.

I imagine some readers will recognise this development approach, good. Now we have a name for it: Salami Agile.
(Picture from André Karwath on WikiCommons under Creative Commons Attribution-Share Alike 2.5 Generic license.)

Agile in Cornwall

2010-12-01T08:59:00.000Z

As I’ve mentioned before, one of my current projects is to help companies in Cornwall adopt Agile software development. The programme is bring run under the Grow Cornwall umbrella by Oxford Innovation.

Next Thursday there are two open to everyone sessions to introduce Agile to companies in the Cornwall area and talk about the work we are doing.

If your interested you need to register at MeetUp - “Agile in Cornwall - Meet the Experts.”

Three plans for Agile (long version)

2010-11-25T17:40:00.000Z

As noted in my last blog entry RQNG has published my description of Agile planning using Iteration plans, Release Plans and Roadmaps. I’ve added a little bit to this an produced a longer version of the same piece - it only adds about a page.

Three Plans for Agile (long version) is now available on my website.

Three Plans for Agile on RQNG

2010-11-17T17:07:00.000Z

RQNG has published one of my articles entitled “Three Plans for Agile”. This looks at the role of iteration plans, release plans and roadmaps, and how the three fit together.

I have to say thanks to Jon Harley, we had an e-mail discussion months ago which was the seed of this article.

After writing this I discovered (I’d like to say remembered but truly, I’d forgotten) that I actually blogged about this early last year as part of my Project Plans mini-series.

Reverse-Tolstoy continued - from Agile EE

2010-11-08T17:16:00.000Z

In my last posting I suggested thatif Leo Tolstoy worked in a corporate IT department the opening lines of Anna Korenina might have read:

“Unhappy corporate IT departments are all alike in the same unhappy way; all happy departments are happy in their own way.”

I call it Reverse-Tolstoy (because he actually wrote the reverse, well kind of). I want to continue that rant with some other ways corporate IT departments repeatedly mess up.

Belief that process change is it
I wrote in my previous entry that corporate IT departments do not focus enough on the technical side of Agile, specifically they don’t think about code quality and technical practices. Part of this is caused by a belief that process change is the change they require. That is to say, that changing the process they use, whether that means “Agile” rather than “Waterfall”, or Scrum over ISO-9000, or Kanban over ad hoc, or what ever, well, thats the change they need.

Such a view ignores the technical side of change required. As I’ve said before, and I’ll say again, I don’t believe you can be Agile if you don’t address the quality issue. I also believe that the best place to start your Agile change is with the quality improvement (thats a TQM, ISO9000 or Six Sigma quality programme by the way).

No customer involvement
For these IT departments actual, paying, customers are a long long way away. This is changing a little but most of the customer they have are internal, i.e. other parts of the business. These customers have little involvement in the development of IT system. Indeed, you will frequently be told that they want minimal involvement.

The internal customers have an image that they can hand over and idea, sign a requirements document, and leave IT to get on with it. They might get involved in some acceptance testing but they don’t want to hear about it before then.

I think this view has been encouraged by corporate IT. I think IT as an industry has trained customer very well, customer now believe if they just write it down, hand it over then it will be. Kind of double-think really because most customers have also seen IT failure. However, they think its “IT failure”. The idea that they, the “customers”, might be part of the solution is strange to them.

Instead they cling to....

“Long” UAT cycles that are separate from development
UAT, User Acceptance Test, the thing that the “business side”, the people who want the software do to ensure you deliver what they originally asked for. Having thrown it over the wall, having waited, having complained the “customer” wants make sure you deliver what they asked for. So the have a user acceptance test cycle.

Unfortunately this normally comes too late in the day to make a real difference, and, real “users” don’t actually want to be involved. So you often find UAT is carried out by dedicated UAT testers who don’t actually do the customers job.

UAT ends up being just another test cycle. All too often it is where real testing takes place, by the time the software reaches UAT it is still buggy. Rather than being about seeing if the users like the software UAT is about finding bugs.

UAT is normally one of those “no go” areas when you start talking Agile. Frankly, the business doesn’t trust IT enough to believe they will deliver anything, let alone that it is of high enough quality or actually meet customer needs.

Personally, I think you can abolish UAT, but I don’t want to try until I’ve fixed the quality problems in the team and got the basics of Agile up and running. But during this time people keep waving big red UAT flags warning you not to mess with UAT.

This is another example of....

Adherence to existing processes and constraints
All the companies have existing processes in place. Many of them are simply “off limits.” You get looked at as if you are mad when you suggest they annual planning cycles aren’t Agile, or time tracking systems are pointless.

The trouble is, too many of these “off limits” areas and you end up with no change. One big off limits process is the time tracking system....

Time tracking systems - measuring inputs not outputs
Nether mind that time tracking is an act of fantasy on a par with Arthur Conan Doyle, or that it wastes a lot of people time and even more energy one thing that is never on the agenda for change is time tracking.

All these big companies seem addicted to measuring inputs rather than outputs. They know the cost of everything and the value of nothing. Or rather, they think they know costs but really the time tracking systems are so hopelessly inaccurate they don’t reflect what is actually happening.

Time tracking gets really dirty when resources are not visible, that is, when they are offshore. Manager worry that their teams aren’t working hard enough but my experience is the teams are working to hard and just not logging all the hours - the late nights, weekends. The offshore staff thing they are “doing whats needed.” Unfortunately when this short term fix gets ingrained into regular practice its a big problem. It is also a sign, and the cause of, a lack of trust.

SWAGs “High level estimates”
Large companies like to get early “high level” estimates, sometimes called “Seriously Wild Assed Guess”. The idea is to get a ball park number for initial planning and refine it later one.

Trouble is, they don’t. The SWAG is it. It is treated as commitment and people are expected to honour it. In the worst cases the SWAGs appear to work, but thats really because the time tracking system is such a fantasy-land.

Quick Test Pro and Quality Center
All these big messed up IT departments use Quick Test Pro and Quality Center. I’ve looked at these tools, albeit a little, and my conclusion: Emperor’s New Clothes. There is nothing there.

So why so many test groups are stuck on an old version I don’t know.

Message: Save money, throw them out, get FIT, Selenium, JUnit, etc. The best things in life are free.

I look forward to the day when writing a test script in a natural language like English is a criminal offence you can be sent to jail for. If you are going to write a test script then automate it. Precise English takes just as long to write as code. (Even then it isn’t so precise.)

Not really knowing why they are doing “Agile” - seems to be the fashion of the day
At the end of the day most of these corporate IT departments don’t actually know why they are trying to be Agile. I think if they were to really think hard the answer would be Fashion. Yes it sounds good but so did Business Process Re-egineering, ISO-9000 and offshore outsourcing.

Most of the staff inside the companies regard Agile as just the latest change programme. They know it will pass in time. Yes there are some very enthusiastic people in the company, and some highly paid consultants too but that was the case with BPR, ISO-9000, and every other change initiative.

The trick is: keep you head down, look like your going along with it and wait for the next fashion to replace this one. In other words: Don’t want to rock the boat too far.

Reverse-Tolstoy, or Tolstoy reconsidered

2010-11-05T14:42:00.001Z

Rather famously Tolstoy started Anna Korenina with the words:

“Happy families are all alike; every unhappy family is unhappy in its own way.”

I like many others have repeated this quote, and even applied it to business and, in particular the act of creating software. I think, although I dread to check now, that I even open one of the chapters of Changing Software Development with the quote.

I dread because I have come to the conclusion that it it the wrong way around - at least for companies, and specifically the bits of corporations which produce software but which sell something else (i.e. Corporate IT departments) the quote should be:

“Unhappy companies are all alike; every happy company is happy in its own way.”

Blame (credit?) Jason Yip and his “Problems I know you have” blog post that sowed the seeds. But after a series of corporate clients in the last couple of years I’d like to offer my own list.

Here is a quick list of the mistakes I see big corporate IT departments making in the development of software. Actually, there are too many points here, there will have to be a part 2 to this post...

Too much Work in progress
Companies ask so much of their software development groups and these groups don’t say No. As a result they have far more projects in the pipe than they can ever hope to deliver in the supposed timeframe. Earlier this year I met a manager who had about 10 or 12 people reporting to him, he was trying to schedule them to about 17 projects. Any attempts to do anything other than run all 17 at once were an anathema to him and his managers.

This leads to....

Resource foobar
For “resources” read people. There are only so many people in the corporate IT department and in an effort to do work people are assigned to work on multiple projects. So #1 their productivity drops because they are now task switching, #2 quality drops because they are thinking of so many different things, #3 predictability falls because you don’t know when they’ll be working on which project, #4 arguments follow over who should be working on what.

At one client I analysed the (highly inaccurate) time tracking system and found some people worked on 14 projects in one week. Although about 54% of employees only worked on one project a week those who worked on more than one averaged 2.7 projects per week each.

Part of the problem is...

Resource (people) pools
People aren’t allowed to stay with the same piece of code for very long. Which means those who manage them must argue over what project they are assigned to and when. And when they aren’t assigned to a piece of work, but they are the only person who can fix a problem, then all hell breaks loose which destroys the predictability of the project they should be on.

Successful sports teams (think Manchester United) don’t break the team up at the end of the season. Could you imagine the manager saying: “Well the season is over, I don’t need you for a few months so I’ll assemble a new team then, goodbye.”

Unsuccessful sports teams (think Everton, I know I do) usually sell the best players at the end of the season and need to rebuild the team at the start of the next.

Sometime they are pulled off work because of....

Year long portfolio planning
Which means the company decides on some arbitrary date what it will work on for the next year. Which supposes #1 the company has perfect foresight, #2 nothing will change, #3 all projects will run as planned.

Question: How can you be Agile if you decide at the start of the year what you will be doing at the end?
Question: Did your company have a year long plan on Friday 12 September 2008? Was the plan still valid on Monday 15 September 2008?

One side effect of this is: demand for “architects”, “designers” and such is very high at the start of the year, while at the end nobody wants an “architect” but everyone wants Testers. So you create your own resource crush.

Too many Chiefs, not enough....
Big companies don’t like coding, its dirty “engineer” work. So that is all sent offshore or hidden somewhere. Which means they need people to manage it: Development Managers, Project Managers, Programme Managers, Work Package Managers, Test Managers, Development Leads, Test Leads, and their friends the: Architects, SMEs, Business Analysts, etc. etc.

Rule of thumb #1: if the number of coders and testers working on a project is less than than half you have a problem.
Rule of thumb #2: if you find yourself sitting in a room with other (none coders or testers) discussing the work of coders and testers with none of them present then you have a problem

Of course, that assume you have a room....

Too many meetings; too few rooms (projectors, teleconferences, etc.)
When you can’t have a meeting because you can’t get a room its a sign that either you have too many meetings, or.... no, its just a sign that you have too many meetings.

When a company saves money on projectors, teleconferences kit, video conference kit, etc. it saves money on the bottom line but people waste far more value faffing around with making do with inadequate resources.

No quality practices at the code face
And when you have too many chiefs you are likely to find that all talk about “Agile” is about Iterations, Stand-up meetings, User Stories and so on.

Wake up and smell the coffee: if you can’t get quality code done Agile isn’t going to save the day. Invest in TDD, CI, Refectoring, etc. etc. While some big companies do this many don’t - many small ones don’t either but they are not the subject of this post.

There are assumptions at work here: quality is expensive, quality isn’t achievable (all software has bugs doesn’t it?), quality is something developers do, or should do, either way, its something people with dirty under their finger nails are involved with. (I talked about this at the Agile Business Conference last month, Quality – How much quality can we afford?)

Even raising the quality question can lead to accusations that the questioner doesn’t “see the bigger picture.”

(Sometime when you dig into things developers are already moving in this direction but they are doing so without making a song and dance about it.)

Hierarchy
This deserve a blog entry on its own. Inherent in Agile as a whole, but not spelt out explicitly, there is an assumption of a equality. That we are all in this together, we all have a common goal, and we aren’t going to let job titles, length of service, age or other stuff get in the way.

Unfortunately most of the corporate IT groups I’ve seen in the last few years have various people who do stand on hierarchy.

Functional silos
Why, o why, do companies organise themselves as: Test group, Development Group, Analysis Group, and so on?
No one group can deliver a product on its own.

Once you have silos you have people (managers) who need to justify and protect their silo. Solutions which mean people are more integrated worry them.

Lack of knowledge / Knowledgeable people
Another reoccurring bottleneck is the lack of people who actually understand the systems and code base. This is sometime made worse by earlier redundancy programmes.

Kelly’s Knowledge Law #1: It takes time for people to become expert on a software code base and system
Kelly’s Knowledge Law #2: Knowledge transfer programmes don’t fix this any time soon
Kelly’s Knowledge Law #3: You are where you are because of earlier decisions not to recruit more people, not to train more people and to fire people

Don’t believe me? Check out Brook’s Law

And to make things worse....

Knowledge doesn’t accrue in offshore partners
The likes of Wipro, TCS, InfoSys and so on rotate their people every few years. So just as Sid is getting to be proficient in your code base he is moved to another client.

That is, of course, assuming Sid stayed with your supplier long enough to get knowledge of your systems.
Assuming he did: remember he works for your supplier not you

Which leads to....

Hollowing out your knowledge pool
In the early days of outsourcing some really stupid companies gave their staff to the new supplier. When they tried to take IT back they found the people and knowledge stayed with the supplier.

So today most (clever) companies keep the key people - people with experience, “architects” and the like. But, these people are growing older, and sometimes they leave of their own accord. Which begs the question: Are you creating the next generation?

How does someone get to be an Architect? (i.e. a developer who might code)
Or an Subject Matter Expert? (e.g. a system analyst who doesn’t code)

Answer: they get to know this stuff because they once cut code.
These people became Architect and exports because you decided their knowledge was too valuable to have leave the company or do dirty things like coding.

So what are you going to do when they move on? Or retire?

Remember: the thing that makes them valuable is a) they work for you, b) they worked for you for so long they know how it works.
Outsource the doing today and you loose you knowledge tomorrow.

I see these issues repeated over an over again. Now I’ve told you it isn’t rocket science so why do companies do to themselves?

As Tolstoy might have said: “Unhappy companies are all alike; every happy company is happy in its own way.”

I’ll continue this rant in another blog posting soon.

Agile elevator pitch

2010-11-02T17:05:00.001Z

My (our) entry in the Agile elevator pitch competition:

“[Agile] Provides philosophy, techniques and tools to alleviate the pain of traditional development and make teams more effective thus increase your profit.

Companies such as the BBC, GE Energy, Yahoo, the Financial Times, The Guardian and others have already adopted the approached.”

As some people know, I’ve been doing a lot of work in Cornwall recently. This involves working with a variety of companies all involved in software development - from online e-commerce website builders to companies creating embedded software for medical devices.

My partner in this endeavour, Michael Barritt of Oxford Innovation and Grow Cornwall, suggested we really need an elevator pitch statement for what all this Agile is about. The above is our result.

Of course this is context specific. Too many senior managers this is irrelevant because they don’t know anything about software development. At that kind of level Agile itself becomes meaningless because it is a solution to a problem which they know nothing about. And actually, they don’t want to know about.

There is always a danger with Agile elevator pitches, or any other type of elevator pitch, that it just becomes “Will increase your return on investment.” At some point such pitches become meaningless, you don’t know if the product will fix your software development issues, cure cancer or make you tea in the morning.

So what do you think?
Good?
Bad?
Indifferent?
Got a better one?

Red, Green, Kaizen

2010-10-27T16:29:00.000+01:00

Back at the beginning of September Kevin Rutherford tweeted:

‘Anyone know the Japanese for "red, green, refactor” ?’

Now as it happens I know a Japanese speaker very well. Red was easy “akai”. Green was a little tricky because the Japanese actually have two words for Green, “aoi” and “midori.” We went with midori.

Refactor was a little bit tricky. After all, there was no such word in the English language until Martin Fowler published Refactoring. Indeed, “refactor” still haven’t made it into my English dictionary.

My translator suggested the Japanese would probably stick with the English term so we got “akai, midori, Refactor.”

At which point Kevin asked: “akai, midori, kaizen?”

Its taken me a few weeks to have an in-depth conversation about this with my translator but it seems Kevlin is right.

Kaizen is actually two Japanese words:
Kai: change
Zen: good

So the net effect is: “to change [improve] for the good”

Which seems to sum up refactoring perfectly.

Now, any other Japanese speakers out there like to refactor?

And just a gentle reminder that Changing Software Development is now available in Japanese. My personal Japanese translator didn’t do this translation, it was handled by the Japanese publisher.

An Agile Reader - buy it now!

2010-10-24T21:41:00.002+01:00

When I deliver training courses I like to give participants something to take away to read, something that is more readable than PowerPoint slides. Sometimes participants get a copy of my book, Changing Software Development, but that is expensive to me and not always appropriate. (These days I tend to tell people CSD (as I call it in private) is an advanced Agile text, not an introductory one.)

A few months ago I decided to put together “An Agile Reader.” This is a mini-book, its a collection of articles which are interesting for someone new to Agile and decidedly shorter to read than Changing Software Development.

Anyone who’s been on a course with me in the last couple of months will probably have seen a copy of this. The feedback on the mini-book has been good and at recent conferences I’ve given away some spare copies.

So I’ve decided to make it public.

If you’d like a copy you can buy a physical copy here, or download the complete PDF. Price is £4.99 for the physical copy, currently discounted to £3.99 or £1 for the electronic.

I don’t see this as a new book, I prefer to call it a mini-book. Its a collection of essays, most of which you can get for free from my website or this blog. Its self-published using Lulu, its not been professionally copy edited and its not got an ISBN number.

Let me know what you think.

Cranfield study Supporting the alignment trap

2010-10-21T17:05:00.001+01:00

I’ve blogged before about “The Alignment Trap” (Originally in December 2007 and after my piece in The Agile Journal in February 2009). This is the counter-intuitive evidence that sometimes doing things right is more beneficial than doing the right think. I often discuss this in my presentations and there are two standard reactions: for some this research rings true while others question it.

Those who question it often ask: is there any other evidence?

Well, there just might be.

Delivering Value from Information Systems and Technology Investments: Learning from success is a survey report from Cranfield School of Management in the UK. I only heard about this report a few months ago and its is now four years old but I see little reason to suspect things have changed since 2006.

The researchers define four levels of maturity in organisations developing systems. The first level is project delivery with the focus on the supply side. In order to reach the higher levels organisations must first be able to successfully deliver the IT projects they undertake.

In other words: doing things right is a prerequisite to success.

(I should point out that this report has nothing to do with Agile, Waterfall or any other method. Its about the ability to realise benefits from IS/IT projects whatever the approach.)

The second level of maturity concerns realising the expected benefits from projects. Here attention expands from the supply side to look more at the demand side and whether the business cases proposed are realistic. It is here that “doing the right thing” starts to be the issue.

Levels 3 and 4 concern realising benefits consistently across the whole portfolio.

The report contains some other nuggets of information which are with reporting:

73% of survey respondents said significant improvements were needed to get satisfactory value from IS/IT. (This 73% is curiously close to the 71% of companies in the “Maintenance zone” in the Alignment trap report.)
38% of businesses openly admit benefits are overstates in business cases in order to obtain project funding
80% report that review of work against the original objectives is inadequate

Finally, the other statement is worth analysing from this report is:

“The data also indicates that organisations have undue faith in business cases and that the deployment of formal methodologies gives managers a false sense of security, perhaps an excuse for not becoming sufficiently involved.”

If we take that statement apart it supports the Agile approach:

“Undue faith in business cases”: Implies more tools are needed to tackle value. Agile would suggest a try, experiment, fail-fast, fail-cheap approach

“Formal methodologies ... false sense of security”: That could be PRINCE2, Scrum or something else. The methodology doesn’t have the answers. You need to be pro-active and think for yourself.

“Not becoming sufficiently involved”: Yet again we hear that lack of customer/user/manager involvement is a cause for failure. We have a few tools for working around these problems but nothing beats having actual involvement.

Cambridge edition of Objective Agility now online

2010-10-14T12:01:00.000+01:00

Slides from my Agile Cambridge 2010 Objective Agility: What does it take to be an Agile company? online now.

Reuse Myth - can you afford reusable code?

2010-10-12T21:16:00.001+01:00

In my Agile Business Conference present (“How much quality can we afford?”) I talked about the Reuse Myth, this is something always touch on when I deliver a training course but I’ve never taken time to write it down. Until now.

Lets take as our starting point Kevlin Henney’s observation that “there is no such thing as reusable code, only code that is reused.” Kevlin (given the opportunity!) goes on to examine what constitutes “reuse” over simple “use.” A good discussion itself but right now the I want to suggest that an awful lot of code which is “designed for reuse” is never actually re-used.

In effect that design effort is over engineering, waste in other words. One of the reasons developers want to “design for reuse” is not so much because the code will be reused but rather because they desire a set of properties (modularity, high cohesion, low coupling, etc.) which are desirable engineering properties but sound a bit abstract.

In other words, striving for “re-usability” is a developers way of striving for well engineered code. Unfortunately in striving for re-usability we loose focus which brings us to the second consideration.... cost of re-usability.

In Mythical Man Month (1974) Fred Brooks suggests that re-usable code costs three times as much to develop as single use code. I haven’t seen any better estimates so I tend to go with this one. (If anyone has any better estimates please send them over.)

Think about this. This means that you have to use your “reusable” code three times before you break even. And it means you only see a profit (saving) on the fourth reuse.

How much code which is built for reuse is reused four times?

I would suggest the answer to this question is: very little.

Consequently development teams which write a lot of reusable code are costing their organisations a lot of time and money. Waste.

Reusable code is not the solution to any of our IT problems. It is a supply side only solution and not a very good one at that. While it may reduce the amount of code that is written it reduces it by artificially constructing supply rather.

Thus, reuse, as a solution to software supply problems, is a myth.

This leave two questions we need to answer.

First: how are we to get those desirably engineering properties if we can’t/don’t push reuse?

The good news here is that Test Driven Development, i.e. writing code that can be automatically tested with other code we write, also has the effect of promoting modularity, high cohesion, low coupling, etc. Thus the answer to this question is: Make your code testable.

This approach will retain focus and deliver worthwhile benefits.

The second question: How should we manage reuse? After all, there are some genuine situations were reuse is the right answer.

Here there are two answers, a micro and a macro answer.

The micro answer, when your working in a development team is: look for emergent reuse.

Don’t plan for reuse but look for opportunities to reuse something that has gone before. When you see the opportunity take the previous work and enhance it just enough to reuse it in your case. This way you only pay the price for what you actually need when you need it.

If you later find the same code is useful again then repeat the process. Improve it just enough to use it a third time.

Remember: you have the tests from the earlier answer to make it safe to do this. Without the tests things get difficult.

Now you’ve reused your code three times, you’ve pay the price a bit every time, and you have the tests to show it still works. By now the code is going to be getting pretty close to generically reusable.

Still, maybe you go round this look a third or fourth time. It doesn’t matter is making one piece of code reusable costs more in the long run than it would have done if you did it the first time because... You haven’t spent money making a lot more code reusable that never needed it.

Telling the future is hard. It is difficult to know whether code will be reused or not, so default to Not and save the money. Occasionally you will be wrong and it will cost more, but overall you will be right and you will save money.

The macro answer: if this code really is widely reusable by many people then, go into business. Market this as a library, a plug in, an application what ever. Let the market decide the economics of reusability.

Agile Eastern Europe slides online

2010-10-09T14:48:00.000+01:00

My conference overdose continues with Agile Eastern Europe in Kiev this week. This was going to be a simple repeat of my Business Analyst in Agile talk which I’ve given a few times before.

But somewhere over Germany I started playing with the slides. I pulled in a bit of “Objective Agility”, took a bit out, changed other bits. So the slides are different: The BA role in Agile Development (Kiev edition.)

If I’m being honest, I don’t feel this presentation worked as well as I’d like. Maybe because I changed it, maybe because I’ve delivered it too many times before, maybe because it was 3pm.

Time for reflection. Sorry Kiev, will try harder next time.

Quality - How much can we afford?

2010-10-06T14:18:00.000+01:00

My presentation from the Agile Business Conference (ABC) 2010 is now online: Quality - How much can we afford?

Goal driven projects

2010-09-29T08:08:00.000+01:00

With great timing (coinciding with my presentation to the BA conference and Jax London) the RQNG (Requirements Network Group) newsletter has published a little piece by me entitled:

Time for Goal Driven Projects.

Jax and BA conference presentations

2010-09-28T14:06:00.000+01:00

Two new presentations from the two conferences I’ve been to this week.

From Jax London, Return to Requirements, this includes my “Agile 10 Step” requirements model. This is something I’m increasingly using and I’ll be writing (and presenting) more about in the near future.

And from the IIBA Business Analysis conference: Objective Agility - what does it take to be an Agile company?

Let me know what you think.

Moving pictures (of me!)

2010-09-23T21:51:00.000+01:00

The next few weeks contains more conference for me than is healthy. While most of these are in London (Jax and IIBA BA conferences next week and Agile Business Conference the week after), or just up the road (Agile Cambridge), the interesting one is Agile Eastern Europe in Kiev the week after next.

Alexey and the other cunning minds in Kiev have decided to put the speakers on video. He’s my bit, explaining why you should come to my talk on the role of the Business Analyst.

This is going to be fun, while I’ve been to Russia and Moscow a few times, this will be the first time I’ve been to Ukraine and Kiev. Should be interesting. Interestna, as they say.

Unfortunately, in this blaze of conferences I’m not going to get to go to LESS 2010 (Lean Enterprise Software and Systems) in Helsinki. Which is a shame, not just because it promises to be an interesting conference but because I quite like Helsinki.

Now, can someone explain why there are so many conferences in the late-September early-October period?

For a bonus point, please explain why this period is also extremely busy with work for me: Agile coaching in Cornwall this week, several training days between the conferences and then back to Cornwall to give more coaching before the month ends?

Factory Physics and software factories

2010-09-19T13:04:00.000+01:00

For the last few months I’ve been wading through Factory Physics. That might sound very negative but actually this book is highly recommended. The reason its difficult to read is nothing to do with the writing - which is actually quite easy going - but more down to the size and the maths. It is over 600 pages long and was originally written as a text book so can be a little tedious at times. Add in a bunch of mathematical proofs - essential - which even if you skim can be hard going and you get the picture.

That said its a great book to read with lots of important facts and implications. I have been reading it because I had some loose ends. The operations management module in my MBA all those years gave lots of suggestions and advice but left me wanting to know more. I finally found what I was looking for in Factory Physics. (Specifically queuing, variation and optimisation, but that will wait for another post.)

Now, to get back to the title of this blog post.

Every so often I come across managers who liken their software development teams to factories. They talk about the “software factory” or what happens “in the engine room.”

I think this analogy shows a fundamental misunderstanding of what goes on in software development groups. The idea that you can liken the development of software to a factory line process, with repeatable inputs, defined outputs, known processes, and repeatable activities just isn’t the case.

The difference shouldn’t need pointing but I will: as commonly understood the factory production line repeats again and again, it constructs one (or a limited number of) product(s) many times. There is little innovation or problem solving in the process (as commonly understood).

Whereas, software development constructs one product, for the first time ever; entailing a lot of innovation and problem solving in the first place; and never constructs it again. (After the first time a simply digital copy suffices.)

Now, having digested a lot of Factory Physics there is another reason I dislike this metaphor. Now I have a better understanding of the complications of making a factory efficient it is clear to me that making a factory effective is damn hard work.

Software managers who like to think of their development group as a “software factory” not only have no real understanding of the development process, neither do they have an understanding of how to organise and manage a real factory producing physical things.

In fact, one of the arguments in Factory Physics is that most factory managers don’t have such an understanding either. The authors argue, convincingly, that a lot of factory organisation and management is fashion, or fad driven and not based on a good understanding or science principles.

So I suppose in that way it is like managing software development.

Now before anyone rushes recommend, or attack, Fredrick Taylor and the ideas of Scientific Management the authors of Factory Physics are clear. Yes we should apply scientific thinking to management but, they point out, Fredrick Taylor was not scientific. His studies - from a scientific point of view - do not stand up. They are full of holes.

Just in case I’ve not been clear, lets repeat:

It is a mistake to think of software development as a factory line process.
Anyone who insists on using the metaphor should find out what is actually involved in running a factory.
There are insights from running a factory - to be found in Factory Physics - which apply to software development and many people will benefit from learning them. That doesn’t mean you should run your software development process like a factory though.

Japanese translation of Changing Software Development out now

2010-09-08T14:26:00.003+01:00

The Japanese translation of “Changing Software Development: Learning to be Agile” is now out and you can buy it from Amazon Japan right now.

And just in case you don’t believe me here is the cover:

Agile is like...

2010-09-07T09:45:00.001+01:00

In the software development world there are, broadly speaking, two groups of people: those who create the software (coders, testers, etc.) and those who manage the process (project managers, development managers, etc.). When discussing “Agile” I find that both sides think the problem is with the other.

To put it another way, if I’m talking to developers they think its managers who are the block to adopting more Agile techniques. But when I’m talking to managers they say its the developers who resist Agile.

This always reminds me of the old Philip Crosby quote:

“Quality has much in common with sex. Everyone is for it. (Under certain conditions of, course.) Everyone feels they understand it. (Even though they wouldn't want to explain it.) Everyone thinks execution is only a matter of following natural inclinations. (After all, we do get along somehow.) And, of course, most people feel that all problems in these areas are caused by other people.”

(OK, so this blog just got filtered out of lots of feeds and stopped by lots of firewalls but lets continue.)

Lets bring it up to date and make it Agile specific, substitute the work ‘Agile’ for ‘Quality’:

“Agile has much in common with sex. Everyone is for it. (Under certain conditions of, course.) Everyone feels they understand it. (Even though they wouldn't want to explain it.) Everyone thinks execution is only a matter of following natural inclinations. (After all, we do get along somehow.) And, of course, most people feel that all problems in these areas are caused by other people.”

Study on benefits of TDD

2010-08-30T10:17:00.000+01:00

OK, this isn’t news, this study came out a couple of years ago and was covered by many people then. But, I find myself regularly referring to it trying to find the link. So I’m going to blog about it then I’ll always be able to find the link.

The study is by Nagappan, Maximilien, Bhat and Williams and is entitled: Realizing quality improvement through test driven

development: results and experiences of four industrial

teams and is freely downloadable from Microsoft.

This is the key findings are summarised in this table:

Team / Metric	IBM drivers	Microsoft Windows	MS MSN	MS Visual Studio
Defect density of comparable team not using TDD	W	X	Y	Z
Defect density of team using TDD	0.61W	0.38X	0.24Y	0.09Z
Increase in time taken because of TDD	15-20%	25-25%	15%	25-20%

The way to read this is: the researches looked at two Microsoft MSN teams, one team did not use TDD and had a defect density of Y. The second MSN team had a defect density less than a quarter of Y but took 15% longer.

To my mind that proves that which was to be proven, i.e. TDD reduced bugs. But, I’m also aware that other writers have disputed this and I’ve heard of studies which disprove it. (Anyone got a link? Thanks)

Most people who I’ve met, and who have practices, or understand TDD agree it is effective. However there are those who don’t believe it. It reminds me of the episode of Black Adder where Rowan Atkinson’s Black Adder hires a ship Captained by Tom Baker. When there is a problem it plays out like this:

Black Adder: “Someone in the crew will know... you do have a crew don’t you?”

Captain: “Arh, opinion is divided on the subject... all the other captains say you need a crew, and I say You Don’t”

At the end of the day Confirmation Bias will probably decide which set of results you choose to believe.

Ed Yourdon on Agile

2010-08-26T09:21:00.001+01:00

Ed Yourdon seems to have fallen off my radar so far this century. Last century I read a lot of his stuff and came to respect him as a man who knows what he’s talking about when it comes to IT and software development.

(If you haven’t heard of him, or don’t believe me just look at the list of books he’s written.)

I recently discovered that he has a blog, and from there that he has recently been to the Agile 2010 conference. In fact it appears that the whole Agile thing has, to a large part, passed him by.

The really interesting thing are some of his comments on Agile from this blog posting:

‘My overall impression is that “agile” (and its variations, such as scrum and XP) are now entering the “mainstream” of computer systems development’
‘there’s still a lot of hype and exaggeration, along with a non-trivial amount of myth and folklore and general silliness. But when you strip away all of this, there is a very solid, and extremely well-documented, core of practical system development guidelines, concepts, strategies, and hard-won lessons that every practitioner in our field needs to know about.’

I’m happy to find myself agreeing with Ed, and even happier that the voice more experience sees it the same way I do!

Objective Agility - what does it take to be an Agile company?

2010-08-25T09:41:00.000+01:00

Modern Analyst has published my latest piece about Agile at the company level: Objective Agility - what does it take to be an Agile company?

This is actually a bit of a taster for a presentation of with the same title I’ll be doing at the IIBA Business Analysis conference in a few weeks.

And talking of businesses analysis... I am running an ‘Essential Agile for Business Analysts’ at Skills Matter in a few weeks. Many of these themes come up in that course. I believe there is still space available.

And if you miss all that, I’ll be sticking with this theme for the Agile Cambridge conference in October.

The No Business Case Myth

2010-08-16T19:48:00.001+01:00

Once in a while I run across individuals, or even teams, who still think Agile is about just getting on and doing it. Well it is, good for them, but, that doesn’t mean there isn’t a reason for doing it.

There seems to be a myth in some circles that work done using Agile techniques doesn’t require a business case. Lets get this clear: Agile does not excuse you from having a business case for your work.

Of course there are instances were a business case might not exist. Some companies, for better or worse, work without them; those companies aiming at innovation may allow work to proceed to a more advanced stage before asking for some rationale; and products which are in a steady state may just tick over without too much attention to a business case.

But in each one of these cases the lack of a business case has nothing to do with Agile. (Actually, each of the cases does have a business case, its either a tacit business case or a part of a much bigger business case.)

Many development efforts which lack a visible business case can benefit from demanding a business case. If you are not sure why your company is doing some piece of work then maybe the company needs to be clearer about the reasoning.

However, a business case might look a little different on a Agile project. It may well be shorter than a traditional business case, it may lack some of the detail traditionally found in a business case, and it may describe much more a trial-and-error approach.

What Agile projects don’t need are in-depth business plans. Here its a question of detail, a strategic business plan makes plenty of sense. But a business plan which lists many many features to be build and a detailed schedule of when they will be built is little more than an illusion. Such plans give the appearance of certainty but certainly don’t provide it. Indeed, too many plans can actively hinder a project - plans are not benign, they are dangerous. (See An alternative view of design (and planning) for more discussion here.)

That said, there is a very good case for writing a business plan - indeed most forms of plans actually: they are rehearsal time for the real thing. They help you explore and learn about the thing you want to build.

Still, sometime you might skip the business plan. Maybe there isn’t time for a rehearsal, and sometimes learning can be maximised but just getting stuck in .

But, if you do write one, then once that exercise is over, once you start doing it for real, don’t try and hold yourself to a plan. Put the plan in the draw and don’t look at it. The value of planning is not the plan you produce at the end of planning; the value it is the knowledge you acquire in your head.

For that reason you want to maximise the number of people involved in the planning rehearsal so you can maximise the learning.

Off topic moan - Price Promise that isn't

2010-08-10T15:56:00.002+01:00

Although I try to keep this blog within its very loose boundary of software and business there are times when you want to say something that doesn’t fit.

I could try and justify the following story on the basis that it concerns my upcoming trip to Agile Eastern Europe. Or I could justify it as an example of poor system thinking, but I think it is really a moan.

It also shows how I was fooled. I’m not as bright as I like to think I am, for once I believed what the corporation told me despite my natural cynicism.

It all began when I set out to book my flights to Agile Eastern Europe in October. As usual I did a quick check on Opodo to see what the market fare was and who had flight. Turns out British Airways had a direct flight, was fairly competitive and since they are my usual airline I decided to book with them.

As usual I then went to the BA website to see if it was any cheaper. It wasn’t indeed it was more expensive, but since I was now logged into the BA system and all my details were on file I decided to finish the booking there.

However, BA was more expensive. That’s OK I thought, BA make a big thing of their “Price Promise.” I’ll just keep a record of what Opodo offer and they’ll give me the money back, about £25.

Wrong.

British Airways won’t pay up. Despite sending them the PDF I took of the Opodo offer they say its a different itineracy. Erh? Same flight out, same slight back. But it seems, Opodo was offering me two single tickets and BA were offering a return ticket.

Wonderful. British Airways are hiding behind word play. I want a flight to Kiev and a flight back. Do I care about the niceties of ticketing? No. The difference is lost on me.

So there you have it. BA have made an extra £25 out of me. Clever, pat yourself on the back guys.

Except, I’ll never book with BA.com direct again. That means 20%-30% of all the ticket revenue will go to Opodo in future.

I’ve been avoiding BA for much of this year because of the strikes. This means I’ve rediscovered Heathrow Terminal 1 and flying with BMI and Lufthansa. Looks like I’ll be using them more.

Advice to companies: if you are going to make these kind of price or quality promises don’t hide behind word play when people try to claim. It annoys them even more.

I have only myself to blame, a quick bit of Googling shows other people with similar complaints - here and here.

Sorry to my regular readers for a well off topic piece.

I'm a BA get me out of here! - the BA role on an Agile Team

2010-08-10T15:56:00.001+01:00

In the last couple of years I’ve given my “BA role in Agile” (PDF of the Nottingham version, or Slideshare of the Skills Matter version) talk a number of times at a number of different venues. A couple of months ago I got around to writing up the talk and it has now been published in ACCU Overload.

If you’d like to read this you can read it in Overload 98, or take the PDF from my website, “The BA role on Agile teams”.

Shorter versions of this piece have, and probably will be again, published elsewhere. The Overload piece is the full version. Thats the nice thing about Overload, it has the space to explore a topic more fully.

I’m just now putting the finishing touches to the 3-day training course that grew out of this talk and which will be running in September at Skills Matter.

Conferences, Conferences, Training - a busy autumn

2010-08-08T11:45:00.002+01:00

I’m speaking at a bunch of conferences this autumn so if any reader out there would like to hear me speak, or ask a questions you might get yourself a ticket to one of these:

Jax London / DevCon, 27-29 September, London: Return to Requirements, 27 September
IRM IIBA Business Analysis Conference, 27-29 September, London: Objective Agility, what does it take to be an Agile Company?, 28 September
Agile Business Conference, 5-6 October, London: “Quality – How much quality can we afford?”, 6 October
Agile Eastern Europe, 8-9 October, Kiev, “The role of the BA in Agile teams”, 9 October
Agile Cambridge, 14-15 October, Cambridge: “What does it take to be an Agile company?”, 14 October

In between times, I’m also delivering several public training courses - not to mention private ones but there is no point in me telling you about those.

What’s particularly interesting here is the new “Agile for BAs” courses that I’ve put together with Chris Matts. The first of these ran last week in association with BA Solutions and was well received. A different, longer, version of this, is running in association with Skills Matter twice this autumn:

13-15 September, Skills Matter, Essential Agile for Business Analysis, London
18-20 October, DevelopMentor, Foundations of Agile Development using Scrum Training and User Stories, London
1-3 November, ProgramUtVikling, Applying Lean Thinking for Software Development, Oslo
24-26 November, Skills Matter, Essential Agile for Business Analysis, London

The train metaphor of software development

2010-07-28T18:18:00.001+01:00

I’m sitting on a train from York so it seems a good time to share my train-leaving-the-station metaphor with the world. In truth, if you’ve worked with me in the last few years, or heard me speak at a conference I may already have shared it with you. But for the rest of the world, and with full embellishments....

Traditional software projects are like a train leaving the station. There is a big train sitting at Platform 9, we know its due to leave soon, but, well, you know what big long distance trains are like, it may well leave a little bit late. Still, to get a seat we need to be early so we are all rushing to the train.

Since we don’t know when the next train to this destination will go - actually, it is far from certain there will ever be another train - we want to be sure everything we need is on board. So we make sure we put everything we might need on the train. (Think of our requirements document.)

Eventually the train lumbers out of the station, overloaded. Quite possibly it leaves late because we were so busy loading the train, maybe one or two people even argued with the guard to delay departure a little bit so we could get more people and things on the train.

The train - our project - is like one of those trains you see in pictures of the Indian rail network with people crowded on and hanging out of the doors.

We all know the train is overloaded, we all know its going to arrive late, we even think it might miss the destination and arrive someplace else. But nobody wants to admit this.

(Particularly true if you have American management and British engineers because the Yanks view the Brits as being overly negative.)

At some point, beyond the half way mark it can no longer be denied that the train will arrive late. So action is taken. Things are thrown off the train to make it go faster.

By throwing things off the train we hope the train will arrive closer to the scheduled arrival time and the intended destination. However, in so much as there was rationale for putting all the things on the train there is less rationale applied to removing them. Things are thrown off the moving train because we can. (Somethings we can’t throw off because they are too connected to other things.)

In the extreme, those who put things on the train, and really know what is needed for the destination (BAs, Product Managers, etc.) never actually got on the train. Once they put the bits on the train they handed over responsibility to Project Managers to get it delivered. These guys are primarily concerned with dates and since they charged with delivering everything they don’t want to throw anything off.

Eventually, the train lumbers into the final stop - which may, or may not, be the intended destinations - with some random collection of things on board. Everyone gets off and breaths a big sigh of relief. Thank God that is over.

Then the news comes: there will be another train. We begin again.

This time though, we have all been burnt. We are going to make sure we have everything we need on the train, plus all the things we threw off the last train, and some more besides because we now understand the need for bargaining and we need levers.

Thats the traditional view. Now for the Agile view.

Instead of big trains we have a metro system - think of London tube, or better still Glasgow’s Clockwork Orange.

I leave the office at 6pm, go to the tube station and wait for a train. The sign tells me there will be one in 2 minutes, and another 2 minutes after that, and so on.

The train arrives and it is full. I have an option: Do I get on the packed, sweaty train and have an uncomfortable journey? Or do I wait 2 minutes for the next one?

I choose to wait. And while I’m waiting my phone rings. Some friends are in a pub nearby and ask if I would like a drink. I have an option: do I go and have a nice beer (valuable) with friends? Or do I value getting home more?

I go for the beer, something I would never do if there was only one train today. But knowing there will be another train, and another train, and another, means that I can go to the pub safe in the knowledge that I will still get home.

Need I say is? We need software development to be like the metro/tube system and not like the big occasional train.

Slides from Skills Matter - BA Role in Agile Software Development

2010-07-23T12:50:00.000+01:00

I presented my “BA Role in Agile Software Development” talk at Skills Matter last night. There was a good turnout and the folks who came to the pub afterwards were all very positive about the talk.

The talk had a few small updates from its previous deliveries so if you’ve heard or seen it before there won’t be anything new for you.

Slides are now online on SlideShare and Skills Matter have a video of the event online too.

The Limits of Agile on InfoQ

2010-07-19T14:27:00.001+01:00

There is a piece by me on todaysInfoQ - The Limits of Agile.

I’m interested to here your comments.

The Business Analysts role @ Skills Matter

2010-07-15T08:54:00.000+01:00

I was down at Heathrow earlier this week, not taking a plane but giving my “The Role of Business Analysts in Agile Teams” talk again. It gets another outing in a couple of weeks time (22 July) at Skills Matter.

So, if you’ve not seen the talk, or know someone (a BA perhaps?) who would like it register on the Skills Matter website for “The Role of the BA in Agile Teams” talk. (Its in the evening and lasts a little over an hour, time well spent.)

http://skillsmatter.com/event/agile-scrum/the-business-analysts-role-on-agile-projects/rl-311

How to improve a team's velocity?

2010-07-01T09:08:00.000+01:00

By way of wrapping up my velocity mini-series (Two ways to fill and iteration, Filling an iteration too well, and Velocity Targeting and Velocity Inflation) I’m going to end with some advice on how to improve a team’s velocity.

Bad news first: there are no silver bullets, there are no easy answers, there is no quick way of doing this.

There is no big fix, there are many, many, thousands, of little fixes which cumulatively add up. Each little fix improve your productivity (velocity) a little bit. Over time these add up to big improvements.

To use economic logic: this is about improving the supply-side. The supply-side argument (largely monetarist) suggests the way to solve unemployment is not to increase demand (Keynes style) but to loosen and liberalise the labour market. There is no single action which can do this, instead there are many small changes that need to be made to make the labour market more flexible.

But back to software.... how might you improve you velocity?

Well, Things to do to improve code quality is a good list to start with. Improving code quality makes teams more productive because they spend less time wading through swamp and scratching their heads.

Of these the thing that comes closed to a silver bullet is: Test Driven Development. TDD is something of a silver bullet, it does improve things BUT (big BUT) it takes time to learn to do it properly. Expect to take time, don’t expect it to happen over night, spend the time and money on training, coaching, setting up a continuous integration server and such. It will pay back, sooner than you think.

How on the heals of TDD is refactoring. In fact the two go together.

Adopting TDD and pursuing refactoring may throw up another problem which people would rather keep quiet about: developer skills levels. Some developers just don’t have the mastery of their tools required. A friend of mine who does TDD coaching tells me it usually ends up as OO coaching more than TDD coaching. So, invest in developer training, buy them books, send them on courses, bring in coaches, set up book study groups and other exchanges were developers can learn to do things better.

I would avoid adding more people to a team. We know, from Brooks Law, that at least the short run that will slow things down. But you can give existing people more time to actually do work. Try reducing the number of meetings you hold. If you have a regular planning meeting and daily stand-up meetings there shouldn’t be a lot of other meetings you need. Certainly there should be very few “all team” meetings.

And if your stand-up meetings are taking more than 15 minutes a day then look to shorten them. Any size team should be able to complete a meeting in 15 minutes if you approach it in the right way - see Three Ways to Run a Stand-Up meeting.

I often meet teams who feel that two weeks is too short a time to do anything useful, and they often question that frequency takes up too much meeting time. Rather than length an iteration it is better to shorten iterations. Teams get to review work in progress and take corrective action more often. There is less time for changes to disrupt planned work. It is good practice at both fitting work in short periods and at making planning meeting more effective.

In time, when you are good at iteration planning I might lengthen them, but it is rare I would let iteration last more than two weeks.

Next get serious about removing impediments. Too many teams live with impediment, accept them as a fact of life, something they can’t do anything about. Each impediment (or block if you prefer that name) reduces velocity a bit. If you want a higher velocity fix your impediments.

I don’t need to say it but I will: retrospectives. Do them, and action the ideas that come out of them.

Finally, just: Concentrate on doing a better job and the velocity, productivity and points will follow. The numbers are there to provide feedback, show you are going in the right direction. Don’t worry about the actual numbers, just keep improving.

Velocity targeting and velocity inflation

2010-06-25T09:40:00.001+01:00

Continuing my mini-series on filling an iteration, velocity and all that I want to flag up a big big mistake: Velocity Targeting. Which leads to Velocity Inflation.

Velocity targeting happens when someone says: “We did 15 points last iteration, lets aim for 20 this iteration”. And when the team fails to meet 20 they say something like: “What happened? We didn’t meet our target?” - or perhaps they start assuming that because the target is 20 the can adjust the plans and message to stakeholders accordingly.

We can all fall into this trap: its called Hope. We hope for a better world. When it gets dangerous is when the person issuing such statements is in some position of authority, e.g. the word “manager” or “leader” is in their title, and they start issuing communications with the target as reality.

Given a few iterations the team will meet the target. However the means they use to meet the target may not be what is expected. And these may well create problems later on.

For example, the team might skip on testing or skip on refactoring. This results in a short term speed up which sacrifices long term maintainability and flexibility. You might actually choose to do this, with the agreement of the authority person, but this should be a conscious decision and you accept the long term slow down.

A more subtle but systematic problem is Velocity Inflation. In this case the team start giving larger estimates, so when work is done the amount done is greater, so velocity rises. The same amount of work is done but the point value is higher. (This can be a conscious or sub-conscious thing, I expect it is more often sub-conscious.)

In some ways this too is natural. Team members want to appear more successful, they want to achieve more, they want to please people - especially those who are in authority and set targets. But, and this is the real danger of Velocity Inflation, it undermines your ability to predict the future work capacity of the team because yesterday’s values can’t be compared to tomorrows.

Velocity inflation is just like financial inflation rational expectations and the Lucas critique need to be considered. It should come as no surprise that I’m going to quote Goodhart’s Law again:

“Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.”

Targets are good when they help people to stretch and reach goals but in setting them you need to be aware of the side-effects. Simply to advocate targets is violation of Deming’s eleventh principle of management:

“11 a. Eliminate work standards (quotas) on the factory floor. Substitute leadership.
11 b. Eliminate management by objective. Eliminate management by numbers, numerical goals. Substitute leadership.”

The solution is simple: don’t do it.

If you (quite naturally) want velocity to rise you have look elsewhere. That will be the subject of my next blog entry.

Filling an iteration too well

2010-06-17T09:38:00.001+01:00

I want to stick with the theme of “how do I fill an iteration?” for a couple more entries. There are a lot of little nuances here, and what works for one team at one time might not be the best thing for another team, or even the same team at a different time.

I appreciated Ed’s comments on my last entry, I think they go to show how small variations work well for individual teams. However, sometimes variations hold problems.

Sometimes you come across a team that completes exactly the amount of work (measured in abstract points) during an iteration as they forecast they would at the start. For example: a team says it will do 10 points of work in the next iteration, and two weeks later they count up and they did 10 points of work.

Occasionally this happens, thats not unusual. Over time I’d expect it to happen more often but I don’t expect it to happen every iteration. When it does then something is probably wrong.

Statistically this is just very unlikely to happen. Yes a team will do roughly the same amount of work but exactly No. They are not doing the same work, the same tasks, the same events will not occur, the same people won’t work on the same things - and if they did we would expect them to get more done.

So when a team regularly scores the same points, week after week, as they expect to (and by implication the same number they did the previous iteration) then some force is making it happen. The real number should be higher or lower.

If the real number should be lower it means the team is busting a gut to do the work. Maybe they are working long hours, or maybe they are cutting on quality. Either way the work pace is unsustainable and problems are being stored up.

If the real number should be higher it means the team is satisficing. That is, they have the capacity to do more work but they are not taking it on so there is spare capacity. Sustainable yes, but not as productive as they could be.

I recently worked with a team led by a manager who would not allow the team to take on more work than they could guarantee doing. He did this because he didn’t want to explain to the project manager running the project that the team hadn’t done as much as they had scheduled.

The project managers in this company wanted predictability. If that is important to you then that’s a reasonable thing to do. But this company was trying to “go faster”. The managers were making a trade off: less work for more consistency week-on-week.

My preferred method is to always schedule slightly more work than the team expects to do. This way there is more work to do if the team find things go well. And if they don’t go well, or if they go badly, then nobody should have been expecting everything to get done anyway.

Two ways to fill an iterations

2010-06-13T12:42:00.001+01:00

The fixed length iteration is a key part of most Agile methods. But the question is: how do you know what (i.e. how much) to put in the iteration?

There are two ways to determine how much is enough but what might be less obvious is that they are alternatives. It is easy to outline both and pretend they work together but really they don’t fit together.

The first way, lets call it the Scrum way, is to set a Goal. The team then commit to making this Goal. The objective is to find a Goal which is both achievable and challenging, and useful but small enough to fit in an iteration.

In this model the Product Owner comes along with some idea of what they want, the team talk about it with the Product Owner and, through conversation, come to an agreement on what can be achieved. The team then go for it.

The team “do what it takes” - if they need to work long hours they do; if they need to bang-heads together, they do; if they need to spend their own money, they do. Given this, one would assume that in those iterations where the team don’t have to move heaven-and-earth they could take things easy. If one iteration they work 14 hour days a reasonable quid-pro-quo seems that in those when they can finish a day early they do.

If at some point the team decide they can’t make the Goal, or the Product Owner says the Goal is compromised - things have changed and it is no longer wanted - then the team declare an Abnormal Termination of Sprint and everything starts over.

The problem with this way of doing things is around sizing the Goal. According to the Scrum literature, by aiming for a Goal the team rally round and choose to stretch themselves. The danger is that the development team start satisficing, that is: they promise enough to keep people happy but not so much that they are ever in danger of failing to meet the Goal.

Those who worry about satisficing probably also worry about what happens if the team meet the Goal early. This isn’t really explained in the Scrum literature I’ve looked at but I’m told that there should be a quick, mini-planning meeting with the Product Owner and some new work accepted. At which point I wonder: what happened to fixed iterations?

Conversely, looked at form the opposite point of view: what is to stop the business side putting on the developers and exploiting the situation to set difficult, time consuming, Goals?

There are plenty of developers in the world who have been bullied by their business partners into giving estimates which meet the business desire but have no real relation to the amount of time and effort it will really take.

Either way, the fundamental problem remains: how do you know how big to make the Goal?

Unless both sides change their mindset then the Goal driven model doesn’t really change anything. And changing mindset on both sides is a big task. One which doesn’t fit well with a gradual adoption approach.

Actually, I don’t think many people use the Goal driven approach to filling an iteration. If they did then I would expect to here about teams having spare time more often, and I would expect to hear about more Abnormal Termination of Sprints. I don’t hear of either so I don’t think this technique is used very often.

The second way of deciding how much to put in an iteration is to use an empirical measurement, i.e. velocity, lets call this the XP way.

In the model the Product Owner proposes some work as before. The development team do some quick estimates on how much effort is involved, the work is prioritised and work begins. The first time you do this you don’t know how much work to put in the iteration - how could you? You’ve not done this before. But the second time you can count how much work you did before and use this as a guide.

Iteration on iteration this count becomes more accurate and its what we call velocity. The technique of using the previous velocity to project the amount of work in the next iteration is called yesterday’s weather.

I prefer to use this technique and when I do I always slightly overload the team in the next iteration. That is, I schedule slightly more work than I expect them to do and everyone knows there is slightly more work than is expected to get done. In other words we expect something not to be done.

There is no point in scheduling even more work because the team isn’t expected to do it. There is no point in scheduling less work because it might be that the team has spare capacity.

If we get luck all the work we scheduled gets done, and if it doesn’t... well nobody should be expecting to get everything done, its just a question of how much.

I don’t update the estimates with actuals because that would be mixing apples and oranges. Estimate counts go in, estimates counts come out.

There are several problems with this technique. You cant be sure what will get done and what won’t, for some people that’s a big issue. Secondly, people like to relate the estimated effort levels to hours or days. When that happens the estimates become less accurate.

More importantly, the Goal driven method aims to stretch the team by challenging them to do something bold. Using velocity and yesterdays weather approach doesn’t even start to do this. The team are immediately satisficing.

Both these techniques have been advocated and used but what isn’t pointed out very often is that they are alternatives. If you use them together you definitely are satisficing to meet a Goal. Using commitment to stretch the team goes out the window. Unless of course you set stretch goals, in which case you are ignoring velocity.

Things to do to improve code quality

2010-06-03T10:32:00.003+01:00

As I mentioned a couple of posts ago, I was recently out in Oslo teaching a course on Lean software development. One of the points I make is: Quality is free (or at least cheaper) provided you invest in improving quality.

This section of the course included an exercise were I ask the participants to think of things they could do to improve code quality. On this occasion the exercise went particularly well and resulted in the list in the picture below:

Lets run through these one by one - not necessarily in the order on the sheet:

Test Driven Development: if there is one practice above all others which contributes to better code quality and fewer bugs it is TDD. On the plus side it can be used on any type of project, Agile, Waterfall or other. Its roots go back a long way but it was a forgotten practice until XP resurrected it. When run as part of a continuous integration cycle with frequent automated builds and tests the practice is Unit Testing on steroids.

However it doesn’t just happen by mandating it so. Most developers don’t know how to do it, they need training and help (coaching) to do it. Even then it is going to be a learning experience, don’t expect it to become prevalent overnight.

(And before you say “But we have a legacy system with 1 million lines of code so it won’t work for us” please read my implications of the power law.)

Acceptance Test Driven Development (ATDD) is the next level up from unit test based TDD. Here those making the requests for development not only specify their acceptance criteria but do so before any development happens, and do so in a way that they can be automatically executed. In many cases professional Testers need to work with the “Customers” to create such tests.

Continuos Integration (CI): This is a valuable practices on its own - making sure code builds and new code doesn’t break anything that already exists. When coupled with TDD and ATDD to created automated, repeatable, test suites, it is an order of magnitude more valuable.

Pair Programming: The controversy over pair programming seems to have died down, but then so too have examples of people actually doing it. A shame really. It is instant code review, it is two-heads better than one (think of commercial pilots or surgical teams). It also allows developers to focus intensely on the work in hand - few distractions from telephone calls, e-mails, SMS, and all the other rubbish that distracts us so easily.

Code review: The next best thing to pair programming. If people won’t pair then at least code review. Put in place a light weight process which happens as soon after the code is written as possible. The big formal process so many of us learnt about in school aren’t practical - only NASA can really afford them anyway. Instead use a lightweight process, you will get 80% of the benefit for 20% of the cost.

Static analysis tools: in the past static analysis tools have gotten a bad name for themselves. The current generation are a lot better and while they are not a true substitute for a code review (because in a code review both reviewer and reviewee learn) they are very cheap to use. Sure you might have to buy a license but once you’ve done that and set them up in the build system they run every time code is checked in and can highlight potential issues very quickly.

Coding standards: Traditionally I’m not a fan of coding standards. In my experience too many teams waste to much time debating and arguing over coding standards and when they are put in place they can be used as a tool for some developers to bully others. However, if you can overcome those problems then they have a valuable contribution to make.

Start by having a group discussion - face-to-face, not over e-mail or on a mailing list - about what could be in a coding standard. Find the areas of agreement and have three or four categories: a very few items as “mandatory”, more items as “recommended” and more as “candidates.” This third group are possible candidates for inclusion in recommended or mandatory but need some consideration. The fourth group for things you agree not to standardise on.

Then review these guidelines every three or four months. Promote some from candidate to recommended, and from recommended to mandatory, and if some aren’t working then remove them or demote them. (This recommendation is broadly in line with Les Hatton’s suggestions in Safer C.)

Then, don’t use coding standards as part of your review. Developers should follow them out of honour. But just in case you miss one, automate them. Set your static analysis tools up to run your coding standards against code which is checked in. Remove the human from the loop and remove the bullying.

Automate: In case it hasn’t sunk in yet, most of the suggestions so far can be automated, and should be automated. Not automating them means they take time to do and are therefore expensive in the long run. Automation might cost in the short run but it makes things cheaper overall.

Refactoring (& refactoring tools): The whole point of refactoring is to improve the code quality and, more importantly, the overall design. If it isn’t then something is wrong. You can, and people do, refactor without automated unit tests but this is equivalent to a high-wire act without a safety net. With the safety net in place refactoring should be a frequent activity and one which doesn’t take up lots of time.

As an old C++ hand I’m always impressed by the refactoring tools available to Java and C# developers. These should lead to more frequent, quicker and safer refactorings.

Hopefully it is immediately obvious how the above can lead to better code. Some of the other items on the list aren’t so obvious but I think they are worth including.

Show and Tell (early): maybe not immediately obvious why this one should lead to better code but it will. By regularly showing potential customers of the software what they are getting developers need to keep their code near to release state. This forces more, smaller, steps in development.

The second reason why this helps is that feedback comes more regularly. This provides positive guidance on what is going right and will point out when things are going in the wrong direction.

Finally, if developers are scared to show their work in progress to users and customers then its time to run up the red flags and look for trouble.

User Tests extends this reasoning. User tests provide another line of testing which helps detect problems early.

Similarly, working on smaller pieces/projects provides for more small steps. Before each step there is an opportunity for re-adjustment and course correction both at the work planning level and at the code level.

Finally, Team cohesion is important because without it the team are running in different directions and doing different things with the code. Part of team cohesion must be a shared view on the development objectives, the design ideas in the code and what makes for good code.

This isn’t an exhaustive list, just the ones my students in Oslo came up with; if you have any more suggestions please add a comment, thanks.

Falling off a log slides online

2010-05-21T09:22:00.000+01:00

Slides from my “Falling off a log theory of software development” talk to Software East are online for download.

These slides only give a small part of the talk, it was a real talk with the slides for illustration only. If you want to get a flavour of what I talked about see these past blog entries:

'In search of mediocracy' - The falling off a log theory of business
and
The lining up your ducks theory of software development

On reflection I think I probably tried to cover too many ideas in the talk. As well as my trade mark Agile Pyramid and Alignment Trap (which make it into just about every presentation) I took in a bunch of other stuff. I probably need to refactor this as two or three talks!

Cambridge, Nottingham and Business Analysts

2010-05-17T09:50:00.001+01:00

My talk at the first meeting of the IIBA in Nottingham last week went well - don’t just take my word for it Alex Papworth says so too.

The slides for the talk are now online “The role of the Business Analyst in Agile”. If you saw my presentation to the IIBA BA conference last September you will recognise the presentation, only minor changes. I’m giving another repeat in July, this time privately to a big British airline near Heathrow.

What has become clear to me is: a lot of Business Analysts are wondering where they fit in the Agile world. To help with this I’ve collected some resources together about the role of BAs, and Product Owners generally, in Agile projects.

I hope thats helpful. If you are a BA, and you have this question then think about attending one of the two courses I’m running later this year. The first is in August and is specifically for BAs, Agile Foundations for Business Analysts. The second is in October and focuses more on specific Agile techniques, Agile Development and Requirements with Scrum and User Stories.

Anyone who followed my last set of links on A3 report templates may have noticed my website has sprung a new tab “Resources”. The intention is to make material more accessible. So far there are Resources for BAs, Resources for Lean and Resources for Patterns (writers).

Before then, I’m off to Cambridge on Thursday to speak to Software East, The Falling off a Log Theory. So still time to book.