News from the Cornish Software Mines

My work in Cornwall, helping companies adopt Agile approaches to software development, is set to continue into the new year - so more jokey tweets about the Cornish Software Mines. I hope to write more about this work - the benefits and what we’re learning in future.

Right now there are two pieces of news I’d like to share:

• At the start of December the author of PHPUnit, Sebastian Bergmann, joined in me in Cornwall to provide training and coaching to teams using PHP. Oxford Innovation (who run the Grow Cornwall programme) have put out a press release about this.
• Oxford Innovation have also produced a Video Case Study of our work with Research Instruments in Falmouth. RI are a great success story and the case study is worth the two and a half minutes it takes to watch.

I first met the team at RI in October 2010 when I provided Agile foundations training to development team. Since then I’ve been in many times to provide follow up coaching.

About six month after the Agile foundations course Jon Jagger delivered a Test Driven Development (TDD) in C# course for the team and has also been visiting to provide follow up coaching. All in all a great success, as the video shows.

Xanpan, ToC, Guilt, plagiarism and sorry

When I was doing my Masters degree I had it hammered into me: credit your sources, don’t plagiarise. To this day I take pride in trying to credit the source of my thoughts, back-up my ideas with research and apportion credit. I won’t always succeed but I think I do a reasonable job. Heck, I’ll go further, I think I do a better job than most.

So image how I felt after my last blog post, about Xanpan, when Benjamin Mitchell tweeted:

“@allankellynet I'm unclear why you don't mention Theory of Constraints 5 Focussing Steps. What's your thinking on this?”

I was mortified because, well, he was right. My blog post discussing how to improve throughput on a Xanpan board was pretty much a Theory of Constraints description.

I had plagiarised. I had failed to credit. And I had written something that other people have written before - probably better than me.

I apologise, I give due credit: these ideas are inspired by the Theory of Constraints - or ToC for short.

So how did I come to make this mistake?

Well, I think the answer is in two parts:

First, I have ToC thinking to heart and it now seems so obvious I’d forgotten it needed crediting

Second, the blog entry was written a day after I’d had a similar conversation with a client. ToC hadn’t been mentioned in that conversation so it wasn’t mentioned here.

So maybe the plagiarism occurred in my conversation with the client. Well yes, but.... my first point still applied and second I tend to avoid crediting with clients. Now I’d better explain that second point some more.

In conversation with clients I tend to avoid saying things like “This comes from....” or “I suggest you go and read....”. Yes I do say those things but every time I say it I believe there are two ways for a client to take it: they may take it as “good, Allan is giving me the original source” or they may take it as “Hmmm, this consultant is name dropping, doesn’t have ideas himself and is very academic.” (Plus, they are paying for my time so perhaps I should just explain things to them.)

In short I don’t think it always helps.

I also remember all those Professors at Business School who uttered “Smith Smith & Smith, 1904” (or whatever) at the end of every sentence. And those turgid academic journal articles where every paragraph was credited and thus become very very boring to read.

I could go on but you get the message.

Now, one more thing, this blog entry is really really defensive. I feel a little attacked by Benjamin - I’m sure he didn’t mean to attack me, its just in 140 characters people come to the point. Maybe he feels attacked by this, don’t know we could get deeper and deeper.

For anyone still reading, I’m sorry for droning on. However I think this does raise an interesting point: in trying to explain things, in writing and in conversation making things accessible and easy to understand is - in my humble opinion - at odds with academic integrity and full disclosure.

As someone who makes a living doing this I need to be more aware of it than most.

Xanpan: refining the process

In my last blog entry I discussed a Xanpan board layout which allows for planned and unplanned work to co-exist in one workflow. Here is a picture of the board I had just put together with a team in Cornwall:



This might be the perfect board layout for the team for ever. More likely it can be improved over time - in the words of Kevlin Henney, it is a Stable Intermediary Form.

I think of Xanpan boards like the Pompido centre in Paris, they externalise the plumbing. In this case the workflow and work in progress. You can see the stuff that is normally hidden inside.

My rules are quite straight forward for Xanpan boards:

Stage 1:

1. Model the current workflow on the board: avoid changing it too much to “how it should be” but you will probably need to modify the workflow a little in order to be able to model it.
2. Make sure you talk through various scenarios for work to make sure you have a workflow that will, well, work. As I said in the original Xanpan blog post columns normally come in pairs: queue then work.
3. Operate the board for at least one iteration (several weeks) and learn from it.
4. Reflect and refine the board: in doing so you will modify the workflow, this should be an improvement.
5. Operate the board some more an start gathering data, qualitative and quantitative about the flow of work.

This alone might improve your teams performance: you have visibility of the work in progress, you can reason about it, people are better informed, you will make better decisions. But, this will not solve or your problems, nor is it the end state, it is just the beginning.

Stage 2: (Assuming you want to improve the throughput)

• Think of the board as a pipeline: there are two ways to improve throughput, make the pipe bigger or make things move through the pipe faster.

First seek to move things through the pipe faster:

1. Look at the blocks: why are they blocking or impeding? What can you do about removing them, not just on this occasion but permanently
2. Where are the queues building up? Can you rebalance the board, or rather how your people (and possibly other resources) are deployed to even this out

Continuing reviewing the board and looking for improvements for ever. But if the time comes when you want a bigger pipe:

• Add people to the team gradually over time: adding people slows down the existing team, the bigger the team the smaller the proportional slow down, above all else: avoid foie gras recruitment. Again this is likely to be a long term activity.

Finally:

1. Look to level workflow: reduce the peaks and move work to the lows.
2. Look to increase the value of the items moving through the pipeline - more and better business analysis and product management

Those are the rules of thumb. Of course there are exceptions, of course there are other issues, of course they don’t cover every eventuality.

Xanpan: planned and unplanned work

I’m on my way back from another visit to the Cornish Software Mines. I’ve been with four different companies this week, laid the first plans for Agile on the Beach 2012 conference and got to know Sebastian Bergmann who has been visiting the mines this week working with the PHP teams.

One of these companies is new to the Agile programme, we did training for the entire company last month and this week I was there to get them started for real. This meant we had to design their board layout and talk about workflow. Once again I found myself advocating Xanpan, the cross between Extreme Programming and Kanban.

The board layout - effectively workflow - is on which reoccurs in Xanpan systems and I would like to discuss here. (I’ll explain the layout here and in a second blog entry I’ll post a picture and talk about the future.)

The problem companies/teams face is: they have planned and unplanned work to do. Typically this is some project they are working to deliver but they need to work on support issues - bug fixes - too. I say that, but it is not just support issues, sometimes it is clients who shouts very loud, or managers who can’t say no, or customers who refuse to wait a few days - indeed sometimes to do so would be the wrong thing to do.

The Scrum answer to this is to stamp on the unplanned work. The team are locked - remember commitment and Sprint-goal. This unplanned work is a problem the Scrum Master should stop - or declare Abnormal Termination of Sprint. Particularly in small companies this isn’t a very useful answer, the truth is, things are more complicated than that, planet-Scrum is too simple.

The ultimate solution is to look at the reason these unplanned items are arising and try to address the root course. In time it might well be possible to do that but right now we don’t want to block the team’s transition. Even in the long term this might be the way of the world for such a team.

The Kanban answer would be to go in the other direction: run all work as “unplanned” and just restock the to-do queue as needed. However this robs the team of the rhythmic certainty of a Sprint. Many of the teams I see need to move away from seat-of-the-pants decision making and knee-jerk reactions. Adopting regular iterations builds in decision making points and helps individuals understand their roles.

So: Xanpan, keep iterations/sprints but allow for unplanned work. To do this we design a board which begins with the three columns:

• Planned
• Unplanned
• Prioritised (usually this queue is limited)

The next column is usually something like:

• WIP: In development (usually WIP limited)

The team start Springing as usual: they hold a planning meeting, review Blue Cards (Business requests, User Stories) and if necessary break them down to White Cards (tasks). These then populate the Planned column. These blues are probably drawn from a traditional product backlog.

At any time work can be added to the Unplanned column. The column is unlimited.

A few minutes before the morning stand-up meeting around the board the person with authority populated the Prioritised column reviews and populates it. This could be the Project Manager, Business Analyst, Team Leader or someone else, the role and title doesn’t matter, what is important is that this person has the authority to do this. On one team this was the joint responsibility of the Project Manager and Business Analyst.

This person looks at the work still in the Prioritised queue, and they review the Planned Work and Unplanned work. From these sources they decide the priorities for today. When the team gather they can see what the priorities are. For the sake of simplicity it normally makes sense for the Prioritised queue to be limited and for the priorities to be ordered: 1, 2, 3, up to the limit.

If need be priorities can be changed during the day: maybe something even more urgent arrives - or goes away, maybe the prioritised queue empties, or some such.

Effort estimates are normally assigned to planned work during the planning meeting. For unplanned work some teams don’t both adding estimates, some added a quick estimate from the person who picks up the card when they pick it up and some put an effort estimate on when it is complete.

Importantly, the original source (planned or unplanned) is recorded on the card - maybe a different colour card, maybe a dot, a word, whatever. At the end of the Spring the team can review how much planned and how much unplanned work was undertaken.

Thats it really: planned, unplanned and prioritised. The first two are effectively queues for the third. Somebody, or bodies, are responsible for balancing priorities.

It is possible that this workflow/board layout is actually a transitional layout. After running with this flow for a while there will be data on how much unplanned versus planned work actually occurs.

Dialogue sheets: more sheets & pricing

A quick follow up to my last two entries about Dialogue Sheets (Retrospective Dialogue Sheets: feedback & updates 1 of 2 and Retrospective Dialogue Sheets: feedback & updates 2 of 2).

Several people who provided feedback said they found the cost of the sheets to be off putting. Since the soft copies are fee the true cost of the sheets depends on where you live and what access you have to large printers, I have little control over that. However, I did set up a print-on-demand service for getting pre-printed sheets.

I’ve looked over the settings and I’ve managed to reduce the price on most of the sheets - while I was there I’ve also uploaded some new sheets - a general discussion sheet, a Sprint level retrospective sheet and a Kick-Off sheet.

The cheapest sheet is now priced at $13.75, or £8.74, €10.20. Since the printer is based in the US the exact price depends on your currency.

Unfortunately, shipping needs to be added to that. Within the US the cheapest sheet should total about $30 - if you order multiple sheets the shipping costs doesn’t rise proportionately so it gets cheaper per sheet.

Apologies to everyone in Europe - myself included, or Asia, or elsewhere - shipping something across the Atlantic costs more. Again, if you buy several sheets at once its cheaper - and is what I do, I have a stack in my office.

I would be interested to know from people who do find cost a barrier just how much they could pay. If I know the restraints I can look into other options.

However, the skeptic in me thinks that the actual price is not important: $13 might as well be $3, $30 or $300. It is the fact that any money needs to be spent that is the barrier.

Which is silly really because sitting half a dozen software developers in a room for an hour isn’t cheap: say $100,000 per developer per year means $50 x 6 = $300, the cost of the sheet is trivial. (This might be why so few teams do retrospectives in the first place.)

Look at it another way: A dialogue sheet costs about $30, which means you save $50 (at least) it would have cost for a facilitator.

Cost arguments always loose so look at it another way: if as a result of the exercise the team save more than one hour it has paid for itself.

As I said, if you have any information that can help me find the right answer please e-mail me.

Retrospective Dialogue Sheets: feedback & updates 2 of 2

This is the second of two blog entries about the use of dialogue sheets. The first entry contain some simple statistics and findings from recent experiences and feedback from users. This entries discusses a few findings which deserve longer comment.

Those who use the sheets consistently report that they engage more team members (everyone gets to speak). Several people report that quieter team members are more likely to contribute in a dialogue retrospective than in a regular retrospective.

Another common finding is that energy levels are higher, there is more discussion and removing the facilitator changes the focus of the retrospective.

These are the results I hoped for when I created the sheets. Yet, to some degree I think these findings are simply the result of changing the format - doing something different. Perhaps a team that uses dialogue sheets for every retrospective will find that in time some of the energy is lost.

Right now I don’t know, I don’t have enough not enough data. However, I have always said these are one technique and I encourage you to use them as one for several retrospective approaches.

A few people report that they don’t like the idea of removing the facilitator. This dislike has stopped them from trying the dialogue sheet. While I understand this fear I hope they will try them all the same. After all, it is the nature of Agile to experiment.

My advice: try a dialogue retrospectives and see what happens, then decide if you want to try again.

One person did report that they felt that without a facilitator some issues were not explored as they might be. I fully understand this finding although it does cause me to ask: were some areas explored which might not have been explored? Again this could be a argument for having some facilitator-less retrospectives and some with.

There is a strange case reported from a company I will not name. Here the management vetoed facilitator-less retrospective. My correspondent doesn’t understand why and I am a little baffled. This gets curious when you know the name of the company concerned. Someone else from the same company replied independently (different country) and has had success with the sheets. I’ve put them in touch and hope this will be resolved.

Interesting, a previous correspondent told me that they kept a facilitator even when using the retrospective dialogue sheet. This hybrid could address both management concerns and the fear of missing topics. I think there is more experimentation to be done here, perhaps using the dialogue sheet to start the retrospectives and then having a facilitator follow up. Or using a dialogue sheet retrospective to identify issues and in following retrospectives deal with them in detail.

Finally, two unexpected findings which carry lessons for those doing regular retrospectives.

All the retrospective sheets include Norm Kerth’s Prime Directive:

“Regardless of what we discover, we must understand and truly believe that everyone did the best job he or she could, given what was known at the time, his or her skills and abilities, the resources available, and the situation at hand.”

To retrospective facilitators this is a holy commandment, something we hold close to our hearts. Yet left alone it is interesting to see how groups react to it.

While one corespondent reported “Kerth's prime directive is also in our company's culture, therefore there is no need to remind people of it, it comes natural for us.” Another reported:
“One group poked fun at the "prime directive", but the other didn’t”.

My own experience is similar. Some teams - usually those who are familiar with the directive - just accept it. One team commented that although they knew it they tended to forget it and it was a good reminder. However, several teams have agonised about the directive, they stall at what should be a routine reminder. Sometimes it is obvious that team members implicitly do blame individuals for problems they face.

I think this behaviour says something about the company, the culture and the teams familiarity with conducting retrospectives. It is, perhaps, the one place where I would like a facilitator on hand. However intervening at such an early stage in the sheet would compromise the approach.

Perhaps the lesson here is that the dialogue sheets work best when teams have trust, and accept the Prime Directive. But then, isn’t that true for all retrospectives?

The other lesson is: while retrospective facilitators hold Kerth’s directive close to their heart the same isn’t true for many others.

The second unexpected finding came at a company in the North West of England. I visited and observed a retrospective using dialogue sheets. There were over a dozen people so we divided into two groups each with a retrospective dialogue sheet.

Unusually the two teams used different sheets - I think one had a T1 and one had a T3, so a similar format but slightly different questions. The first, finding here was that using different dialogue sheets not only does work but can help add insights.

The second, much more significant and unexpected finding was a result of who joined each group. I thought I had divided the groups randomly but as it happens the majority in one group were developers and the majority in the other group were Business Analysts.

At the end of the exercise one of the Business Analysts commented: In a normal retrospective the developers dominate (there are more of them) and so the retrospective usually talks about issues of concern to developers, the issues of concern to BAs are not usually discussed.

It sounds obvious once you see it: the largest group will tend to dominate a retrospective.

However, separate them out, as we did here, gives both groups an opportunity to talk about their concerns. Now the really interesting bit.

The two groups were retrospecting the same time period, however they approached it from different points of view. Some of their conclusions were different but some were the same. To my thinking, when different people, coming from different starting points come to the same conclusions, then those conclusions have added weight.

Overall then: Retrospective Dialogue Sheets are a success.

Retrospective Dialogue Sheets: feedback & updates 1 of 2

Every couple of months I e-mail everyone who has downloaded one or more of my Dialogue Sheets to get some feedback. Getting feedback is why I make people register to download a Dialogue Sheet - sorry, I know some people don’t like doing this, and I know some people fake their e-mail addresses but unless I do this I get very little feedback.

I’ve blogged about my Retrospective Dialogue Sheets before but I thought this would be a good opportunity to update readers with some feedback and findings. This is the first of two posts on this subject, the second post will examine a few findings in more depth.

First simple facts and comments:

• 40% of people who download Retrospective Dialogue Sheets do retrospectives rarely or at best every six months.
• Many who download them and reply to my requests for feedback say “Sorry we haven’t used them” and I guess that many of those who don’t reply haven’t used them either. Given that I expect those who are interested in retrospectives will download, and given that downloaders are measured in hundreds, not thousands, this indicates that actually doing retrospectives is rare.
• 46% of downloaders do them once a month or more frequently, while 15% of downloaders classify themselves as Retrospective Facilitators and perform any retrospectives.
• 25% of downloaders are in the USA, 9.5% in the UK, just ahead of Germany (9.09%). Otherwise only India (5.79%) tops 5%. Though Holland, Canada and France all score 4.5%. I don’t find this surprising given the spread of software development internationally and the spread of dialogue sheet publicity.

• A few people find the sheets too complicated, a few people find them too simple. I had hoped that a variety of sheets would address both ends of the spectrum: the lesson for me is to create some even simpler sheets.
• A couple of people have reported that they don’t feel their team is open enough to use the sheets. My guess is this will present difficulties for any type of retrospective.
• A few people say the sheets are too expensive to print: the print on demand services charges about $40 dollars, the expense is in the postage rather than the printing, if you order a lot the price per sheet is a lot lower. (I must find a European print on demand provider.). However 40% of people say that they have a suitable printer, this surprises me a bit, I never expected large printers were so common.
• To my surprise a couple of people have said there is not enough space to write on the sheets! I will create some new sheets with fewer questions and more space.
• A few replies say that the teams are now using the sheets regularly, and in some cases they have spread from Agile to non-Agile teams.

As I said, one common response is that downloaders have “not had a chance to use the sheets yet.” On the whole I assumed this was down simply not having a retrospective. However a few correspondents have hinted that their company has a prescribed retrospective process. This worries me, if the retrospective approach is prescribed in such detail I wonder how much else is prescribed and how much opportunity there is to change things.

I’ve also been asked if it is possible to customise the sheets. The short answer is No, because they are PDFs its kind of difficult - although I’m sure you could with the right tools. Still there is no reason why not to create your own - a few correspondents say they are thinking of this. (Anyone thinking of creating their own sheets might find this guide for teachers creating dialogue sheets useful.)

Finally for this entry, I’m also hoping to provide translations of the sheets shortly. I have offers of translations into Spanish, German and Finnish - plus I hope to persuade a friendly Russian to help. Main problem at the moment is getting the raw files out of OmniGraffle into Visio.

You are not Steve Jobs (and don't try to be him)

Many people better than I have commented on the passing of Steve Jobs a few weeks ago, I too was sorry to see him go. He touched my life (indirectly): I’m writing this on a Mac, we have an iPad in the house, I might not like iPhones but my new Android phone clearly owes a lot to Jobs vision (i.e. it is a copy).

Reading, watching and listening to the many obituaries about Jobs I started to get worried. My concern is simply that Jobs was not a good role model. His phenomenal success guarantees that more than a few people will attempt to emulate him, in so doing they will sow the seeds of their own failure and make the lives of their employees miserable in doing so.

Here are a few examples.

Jobs was a perfectionist: products didn’t get launched unless he approved of them. However, being a perfectionist is something of a luxury, Apple existed, had revenue, had a brand. If you are a start-up perfection is just too expense.

Jobs started the MacIntosh project because he had Lisa taken from him: MacIntosh and Lisa could only exist because Apple had revenue from the Apple II line.

The first Macs were far from perfect; the team undoubtedly learned from Lisa - what we might call a pivot in Lean Start-Up terms but not quite that clean cut.

Jobs demanded the first Macs would work in 64k, the team knew it needed 128k and Jobs, late in the day gave in. Secretly the team engineered the Mac to accept 512k which is what it really needed.

The moral of the story: there is more to it than being a perfectionist. Even perfectionists need to compromise sometimes. Surrounding yourself with good people helps.

More recently the original iPhone was launched as a 2G GSM phone, without features like cut and paste. It seems incredible now. Apple launched an under-spec phone and refined it in the market. Which leads us to....

Jobs would spurn products/employees who he didn’t think were up to scratch: if employees are loyal to the company and to you, and if you have a deep talent pool, and (perhaps) the stock-options are worth a lot you might get away with this. Otherwise its a sure fire route to staff turn-over.

Jobs may have been a perfectionist but, it seems to me, he wasn’t a perfectionist about quantity of features, quite the opposite. Apple products are simple because they lack so much, once launched they are refined and elaborated in the market.

Jobs didn’t use customer understanding or focus groups to tell him what the public needed. He was a visionary who just knew. This is perhaps my greatest fear: to read some of the obituaries is to condemn the whole discipline of Product Management to the dustbin in favour of senior managers who know what is right for customers - a disease too many companies suffer from.

This argument doesn’t stand up. Jobs was an, perhaps unique, visionary who did understand what customers wanted. He had an ability to see what technology could do and how people could use it. Few of us are blessed with this ability. Thus we have the discipline of Product Management to help us.

Jobs may have shunned market and customer research but his Product teams didn’t. In some cases Apple conducted internal, closed, demonstrations of new products; they used their own staff for research. And, as I just said: Apple launched products and rapidly refined them in the market.

Apple also had many products which were not hits, with and without Jobs: Apple III, Lisa, Newton, Mac portable, Pippin, Apple TV, A/UX, Pink, MobileMe - the iPod took several years to break through and even the Mac came close to failure on several occasions - and iTunes is probably the worst piece of Mac software on the market.

OK, Jobs was not even at the company when several of these products launched but he sowed the seeds and above all it was his company, a culture he had created. And while he wasn’t at Apple he founded NeXT which wasn’t exactly a success either.

But look again: Apple and Jobs were prepared to try things. Some worked, some didn’t. Some took a long time to come to success. Some failed but somewhere, in Jobs head, deep in the bowls of Apple, lessons were learned for future products. We will never know just how much the Mac team learned from Lisa, or how much MacOS X learned from Pink, or even how much the iPad learned from Newton.

Apple and Jobs could do this because they had deep pockets, they had revenue.

Lets put this simply: as much as you might like to think you can emulate Steve Jobs you probably can’t. What worked for Jobs worked for him because he was unique in vision, timing and experience. Just because Jobs could ignore volumes of business advice, armies of experts and accepted wisdom doesn’t mean you can.

The lesson I would take from Jobs and Apple is not about perfectionism, design, arrogance, or the super-CEOs but about the willingness to try something simple, iterate and refine in the market.

Me @ Agile on the Beach

Many of the sessions at Agile on the Beach were recorded and are now available online.

You can find my “Objective Agility” presentation here.

I have a bit part in the The Agile Cornwall case study which is also online. Actually, this video, or rather these videos, are the best version of the Agile Cornwall case study. Although myself and Mike Barritt have given versions of this presentation elsewhere (and probably will do again in time) this version is the best for two reasons.

First, these videos include people on the receiving end of the programme - Research Instruments and Sullivan Cuff Software. They tell about their experiences moving to Agile and the results they have achieved. Second, this version of the case study goes into more depth of about the nature of the programme and how it worked.

Personally I’ve come to see three audiences for the Agile Cornwall case study: anyone who wants to adopt Agile, those who want to know the lessons of our Cornish Agile Laboratory (the Cornish Software Mines), and third, Government agencies who want to see how Government support can help companies adopt Agile.

Finally, I was also part of the panel discussion which is online too - this include Mary and Tom Poppendieck, Toby Parkins and Jason Gorman.

Agile will never work in Investment Banking

In the past I’ve worked for banks, as an employee, as a contractor and as a consultant. And I’ve worked for companies that supply banks in one way or another. And I have lots of friends who work for banks and tell me stories. While I’m not an expert on banks and banking I know a little....

One of the things I know is that modern banking - in all its forms - is highly dependent on IT. Banks spend a lot of money on IT and develop a lot of their own software. As a result they have a lot of software developers. And I also know that quite a few of these developers and their teams have been, and are, experimenting with Agile working in one way or another.

It seems every bank has pockets of Agile, a team here, a team there. And not very often joined up.

Some of these teams are successful, some are less so. They often find themselves battling against big company / bank culture. The banks seem to be very good at killing their Agile teams: they get disbanded, forced bank to traditional (pseudo-Waterfall) approaches by bank procedures, or constrained by “the way we do things here.”

It also seems - notice we get further and further from fact and more into conjecture as I build this argument - that many banks have cottoned on that Agile is good and can help them. To this end many banks (and this is fact) have established Agile support programmes of one form or another. There are people at many banks who are trying to make the bank Agile - at least in software development. For many of them it is an uphill struggle.

At this point we should differentiate between retail banking - what you do on the high street with your bank - and investment banking - or casino banking if you prefer. (Yes there are other divisions but lets limit ourselves to two.)

In retail banking I think there are grounds to be a little optimistic. The stories aI hear are that Agile pockets are having success and things are gradually, slowly, improving and the efforts to link things up are slowly bearing fruit.

In investment banking I’m very pessimistic. I have come to the conclusion that while individual development teams may be successful at Agile for a while inside an investment bank all such teams are ultimately doomed. Your team might be Agile for a few weeks or months or even a year or to, but it will eventually be killed by the bank antibodies.

I have come to the conclusion that investment banking culture is polar to Agile culture. My reasoning is thus:

• Investment banks are extremely hierarchical in nature; Agile is not. Agile can live with some hierarchy but not investment banking extreme.
• Short-term in outlook: investment banks are short-term in the extreme - microseconds in some instances. Agile teams can react in the short-term but only by taking a long term view of engineering and quality.
• Investment bankers are not engineers and do not understand engineering. Nobody from IT ever got to run an investment bank, you don’t get up the ladder from a support function, you get there from selling and banking. A large part of the Agile story is about returning to good engineering principles. The lack of engineering skills, thinking and culture in investment banks undermines Agile.
• Investment bankers think you can solve every problem with money. Throwing money at Agile can work in pockets (by the best team) but doesn’t last or scale.
• Investment banks are individual focused: individual bonuses, the power of the one trader to make money or break the bank (from Kweku Adoboli to Nick Leeson and before.) Agile is team focused.
• Investment bankers think applying more pressure is the way to get results; Agile people now that applying pressure breaks things. A little pressure can help but apply too much and things snap, bankers don’t get engineering so don’t get this.
• Risk aversion: yes, investment banks are highly risk averse when it comes to IT. Perhaps because they take so much “risk” directly with money they very risk averse in their operations.
• Investment banks are contradictory: they say they embrace and manage risk but actually they are risk averse (at least in operations); they say they value the team but their actions say otherwise; they call themselves “financial engineers” but they no nothing about engineering; and so on. Agile is about honesty, facing up to truth and acting on it. Investment banks recent track record shows that honesty is sometimes questionable, to say the least.

I’m sure many of you reading this will say “Rubbish, I know at team at ....”. I am sure you do. While that team might be successful for a while the investment bank culture will eventually kill the team.

So how can we help investment banks overcome this problem?

We can’t. Its the way it is. Take the money, and run, or keep you head down and hope for some more.

Eventually new financial institutions will emerge which get Agile, not just at the software level but elsewhere. These institutions - which might be banks or might take some other form - will eventually replace the investment banks.

This is going to take time. Investment banks are fundamentally a broken business model which are propped up by Government support and market failure. This means normal economic logic does not hold for these institutions.

Cornish Case Study

In the last few weeks Michael Barritt of Oxford Innovation and myself have presented several versions of “Agile in Cornwall” - a case study of our work introducing Agile to companies in Cornwall.

All three versions of this Agile case study presentation are now on my website. I recommend the latest two version - Agile Business Conference and Agile Cambridge. The version from Agile on the Beach in Cornwall was supported with presentations from two of the companies concerned - Research Instruments and Sullivan Cuff Software. In fact, you can find my own case study of the work at Sullivan Cuff on the Software Strategy website. A video version of this will be available in the near future.

If this interest you, or you want to hear it not read it, then I’ll be doing a revised version of the presentation at Skills Matter in a couple of weeks - “Agile Cornwall case study: How the EU is helping Cornish companies get Agile” - a free event, registration required.

Layered burn-down charts

I’d like to conclude this Burn-down chart mini-series (Burn-downs are not just for Sprints and Advice for creating burn-down) by showing a variation on the classical burn-down chart I created with one of my Cornish clients. I’ve recently been advising an SOA project at another client to adopt a similar approach.

In fact, I’m finding this variation makes even more sense then the original, this is the Layered burn-down chart, here is an example:


I’ve redrawn this example to remove client details. What it shows are three things:

• First the classic burn-down, just look at the overall line declining.
• Second the velocity line as described in my earlier blog
• Third the different colours layers show different releases, let me explain...

This company had taken the product backlog and divided it into monthly releases: November, December, January and so on. By the way this was physical on cards and each month’s cards were held together with a bulldog clip.

The contents of a release can - and would - change. But the company felt it knew what it wanted and since the team knew their velocity they could take a stab at which release it would be in. If something new was added the question was simply: when in which release do you want it? And the related stripe on the chart would increase.

That by the way is why the three strips of work yet to do are approximately equal size. If one grew it was obvious, work needed to be removed - either thrown away, or moved to another release, perhaps one that didn’t appear yet.

This was fairly straight forward when the company (thought) it knew everything to be done and ball-pack estimates could be given. With my SOA client they don’t know what the future holds but they can produce a striped chart for the work they do know about. In their case I’m suggesting every stripe represents a requested service rather than a release.

As new services are requested, and estimated, extra stripes can be added to show the total work required. Together with the velocity and slope it should be easy to see when a service will be available to their customers.

Advice for creating burn-down - and other capacity measurements

Continuing from my last blog entry (Burn-downs are not just for Sprints) I’d like to offer some advice on the subject, burn-downs, cumulative flow diagrams, velocity, etc..

1) Don’t track hours. Hours are input and its better to measure outputs. Hours are subjective, see my Humans can't estimate tasks blog. Worse still we have too many psychological hang-ups about hours to make them usable. Instead you want your own currency, call them story points, abstract points, nebulous units of time or what ever you like. Having your own currency allows it to adjust to your team.

Each team has its own currency. Its like having the dollar, euro and pound. They all float independently, there is an exchange rate but it changes.

2) Draw your graphs by hand, process your data by hand. Get a feel for your data. Unless you have a feel for the data and an understanding of how it comes to be you can’t reason about it.

When I say “by hand” I allow you to use Excel (or similar), graph paper is better, but whatever you use don’t use a tool like Rally, Version One, etc. because you won’t have a feel.

Jack Kilby, co-inventor of the silicon chip and pioneer of the electronic calculator, lamented that his invention(s) deskilled engineers. Before the calculator engineers used a slide rule, this meant the engineers needed to know where to put the decimal point. They needed a feel for the data. Kilby lamented that the calculator meant engineers didn’t have the same intuition because of the calculator and led them to make mistakes elsewhere.

4) Use estimated data, forget actuals: estimates and actuals are apples and oranges, or rather, they are future facing estimates and retrospectives estimates.

Estimate tasks just before you do them (e.g. iteration planning meeting), put ball-park estimate on stories which you aren’t going to do any time soon (i.e. later than this iteration), and use averages for really long term stuff.

When you break the stories into tasks, or epics into stories, don’t expect the numbers to match. In fact, you might want to track the variation for longer range forecasting.

5) From 2 & 4: if you have a time-tracking/time-sheeting system leave it to one side, let it exist in its own little fantasy bubble. Don’t use the data, its toxic - see my entries on Capers Jones work. Live by your velocity measurements and charts.

Ultimately, burn-downs are about capacity measurement: John Seddon in Freedom from Command & Control talks about creating capacity measurement charts within organisations. The various graphs we use in Agile are our version of capacity measurement. Accepting its about capacity measurement means its not about targeting. Targeting is wrong see my entry form last year - Velocity targeting and velocity inflation - for more about this.

Burn-downs are not just for Sprints

One of the key tools in Agile is: make things visible. When you can see the thing you can share the thinking and reason able it.

Capacity charts - usually burn-down, burn-up or their cousins Cumulative Flow Diagrams - are amazingly useful things. They show you the state of a development effort. They show this in data. Data by itself is hard to reason with but put it in a graph and wonderful things happen.

It has recently come to my attention that what I thought was obvious about these charts isn’t. So this blog entry, and maybe a few more to come, I plan to discuss charts, and specifically burn-downs, in a bit more depth,

I always encourage, sometime coerce, teams into produce some graphical tracking of their work. For someone like me who’s favourite tool is an spreadsheet this is a piece of cake. But have to remind myself that not everyone finds this so easy.

Importantly there are two burn-down charts you might want to create: a Sprint burn-down and A Product Burn-down. It should be immediately obvious that these correspond to the Sprint Backlog and the Product Backlog.

Just to be clear:

• Sprint based burn-down chart which shows the progress in the current sprint towards completion of all scheduled work and shows work on a day-to-day basis.
• Product based burn-down chart which shows progress through some larger quantity of work, e.g. project, release, milestone. These are updated on a sprint to sprint basis.

Superficially both types of chart look the same, the difference is in the X-axis - Sprint based charts have days, product based ones have iterations. An example will help:



Now I’ve never found sprint based burn-down charts very helpful. Perhaps because my background is working on large development efforts which take months or even years to deliver I rarely see an actual delivery at the end of a sprint. We may have something to show but its only a step on the way to something bigger.

Or perhaps its the economist or statistician in me, trained never to read too much into one set of data figures, and I know we can all have bad days, so I don’t see the point of acting when yesterdays velocity falls.

But I know some people do find value in sprint burn-down so I don’t object if you want one.

However, I always see Product Burn-down charts as vital.

Except on the smallest pieces of work there will be something coming next and a product burn-down can give a good sense of context, overall progress and, most importantly, when you could be finished - especially important on large pieces of work. Armed with this information you can construct sensible release plans.

I recently came across two teams at different companies who had burn-down charts but the charts only covered the iteration (sprint). This seemed wrong to me, it doesn’t tell me much. On closer examination the explanation become clear: the product backlog for both teams wasn’t in a state you could use for a burn-down.

Now if you are taking a truly evolutionary approach this isn’t a problem, you re-evaluate iteration-to-iteration. But actually, few teams take a completely evolutionary approach and these two teams where far closer to the iterative style (i.e. they had original requirements documents to work from).

The lack of a longer-term burn-down was a problem and a sign of a bigger problem.

One enhancement I like to make is adding velocity to the burn down chart. Take this one for example:



This adds a second axis (in Excel you have to hunt around for this option) which allows you to see the team velocity sprint-on-sprint.

The key thing is, without this data, velocity and burn-down - and the graphs to understand it - each sprint exists in isolation. That might be OK if you are delivering every sprint and you don’t need to peek into the future. But the kind of work I see does need to look forward and this information is essential.

Finally, I’ve most of what I’ve said about burn-downs applies equally well to cumulative-flow-diagrams (CFDs). In fact I prefer CfDs and readily use them over burn-downs because they help show how work increases. However, I have also discovered that CFDs are not only harder to construct and update but also harder to explain to people. There is something obvious about burn-downs.

Anyway, you pays your money, you takes your choice. Its up to you.

Agile Cambridge: Cornwall Case Study online

The slides from my presentation at Agile Cambridge are now on my website: Agile County, Making Cornwall Agile (a case study)

Propogation delays

Propagation delays occur when there is a chain reaction, a sequence of reactions which take time to work down the chain. Think of a line of traffic waiting at the red light. The light turns green: the driver of the first car doesn’t move instantaneously, it takes a moment to registers the change, perhaps move the car into gear or take the hand break off, moment to press the accelerator and a moment for the engine to start moving the car.

The second car in the line might be watching the traffic light but they are also watching the car in front. Only when the break lights (if on) go off and the car begins to move can they move. And so on.

It takes time for the change to work its way down the line.

The same thing happens in integrated circuits, silicon chips. Even electrons take time to move. Reducing propagation delays is one of the ways chips get to go faster.

Once you know about propagation delays you start seeing them everywhere. Increasingly I not only see them in software development teams but I talk about them. But not everyone knows the concept, hence this blog.

Propagation delays occur when one team, or one developer, changes some code, or perhaps a database schema, and it takes time for all the other team members to update their code base and make any resulting changes.

Propagation delays occur when a company decides to start doing something, or stop doing something, and it takes time to work down the chain and to actually stop or start.

Propagation delays cause tension because different people, or teams, are working to different assumptions.

As with chips and traffic reducing propagation delays can speed things up.

Updates for Dialogue Sheets

Two dialogue sheet updates today.

Firstly I’ve created a mailing list to make announcements about dialogue sheets and for discussions about the use of dialogue sheets. This list is open to all and is hosted on Google groups (so you need a Google account).

Second, I’ve uploaded a new dialogue sheet. Another retrospective sheet this one is designed for use at the end of training course. I’ve used it at the end of several Software Strategy Agile training course and it has been well received. This sheet fulfils three purposes:

• It allows participants to experience a retrospective
• It provides for a retrospective on a training course at the end of the course
• It introduces dialogue sheets as a technique for retrospectives

Although the sheet was intended specifically for use at the end of an Agile training course there is little that is specific to Agile in it. I’d really appreciate feedback from other trainers who try this sheet.

As usual, all this is at www.dialoguesheets.com.

Cover for Business Patterns for Software

My new book, Business Patterns for Software Developers, now has a cover. Here it is:



The Business Patterns is available for pre-order, it will be published until early in the new year, I’m guessing February but it could move either way.

Dialogue Sheets in Methods & Tools

The latest issue of Methods and Tool (Fall 2011) contains an article about Retrospective Dialogue Sheets. If you are new to Dialogue Sheets it will make a good introduction, and if you’ve used them already thenI’d love to hear your feedback.

Dear client, the truth about software development

Dear Customer,
I think its time we in the IT industry came clean about how we charge you and why our bills are sometimes a bit higher than you might expect.

The truth is, when we start and IT project we just don’t know how much time and effort it will take. Consequently we don’t know how much it will cost. This isn’t a message you like to hear, particularly since you can be absolutely certain that you know what you want.

Here in lies another truth, I’ll try and put it as politely as I can, you are, after all, a customer and really I shouldn’t offend you. The thing is, you don’t know what you want. O yerh, you know in general (usually) but the devil is in the detail. And the more you try and give us the detail before hand the more likely it is to change. Each time you give us more detail you are offering more hostages to fortune.

Just to complicate matters the world in uncertain. Things change, companies go out of business, customer tastes change, fashion changes, Governments change and competitors do their damnedest to make life hard.

Yes it is true we can sometimes gold plate things. Yes some of our engineers like to do more than needed, and yes we have vested interest in getting things added so we can charge you more.

It is also true that you have add things: you quite legitimately think of things after we’ve begun, you quite naturally assume something is “in” when we assumed it was “out” and you also try and put one over too.

(Lets not even talk about bugs right now, it just complicates everything.)

Frankly, given all of this it is touching that you have so much faith in IT to deliver. But then, when IT does deliver, boy, does it deliver big. Look what it did for Bill Gates and Larry Page, or Jeff Bezos of Amazon, or FedEx, the list goes on.

So how do we ever talk you into this?

Well, we package up this unsightly mess and try and sell it to you. To do this we have to hide all this unpleasantness. We estimate how much time we think the work will take, actually these “estimates” are little better than guesses; human’s can’t estimate, most companies don’t have historical data and for those that do have data most of it is worse than useless.

So we get this guess, sorry estimate, and double it. Well maybe we double it, we might treble it, and when we see the new number, if it looks too much we might reduce it. And then we work out whether you will pay it.

That might sound awful but remember: we could have guessed higher in the first place.

Anyway, we don’t know which number is “right” but to make it acceptable to you we pretend it is certain and we take on the risk. We can only do this if the number is sufficiently padded. And even then we go wrong.

If the risk doesn’t happen then we get a fat profit, if the risk comes to pass we don’t get any profit, maybe we make a loss, and if its really bad you don’t get anything and we end up in court.

Really its all a question of risk: we try to present a clean, low-risk solution to your problem, and in doing so we take on a lot of risk. The alternative is that you take on the risk - and the mess - and do it yourself. (Unfortunately, another sad truth, is that as far as anyone can tell in-house IT is generally not as good as specialist providers so if you do decide to take on the risk you actually increase it.)

There really isn’t a nice simple solution to this, you have to work with us on this.

Thats the first part of the solution: you, the customer, have to be prepared to work with us, the supplier, and work with us again and again. This will reduce the risk.

The second part of the solution is: you need to be prepared to accept the mess and share the risk with us. If you aren’t prepared to take on any risk then we will take it on, and we will charge you for it, and since the risk is a lot we will charge you a lot.

Sharing the risk also has the effect of reducing the risk. Once you share the risk you have a motivation to reduce it.

If you are prepared to accept those two suggestions then lets press reset on our relationship and talk some more.

Business Patterns for Software Developers on Amazon for pre-order

Milestones in the writing on Business Patterns for Software Developers are coming think and fast this week.

Monday saw me reviewing book covers - the proposals look really good. Like many other patterns books the cover will feature architecturally significant buildings. Unlike most other patterns books these three are also a World Heritage site.

Now the book is listed on Amazon for pre-order - order now!

Which means, it also has an ISBN number - 978-1119999249 is Business Patterns for Software Developers - or if you like the old style number 1119999243 - spot the difference :)

(Amazon are listing the release date as May but I’m pretty confident we’ll better that, I also suspect the page count and price will be higher than currently listed.)

I’m pretty close to the final text, mainly images I need to fix now.

The keen eyed among my readers will notice the words “Agile” and “Lean” are absent from the book title. Indeed, the words are largely absent from the text. They are in there, somewhere, they are difficult to ignore, but this isn’t a book about Lean or Agile. Yes it contains lots of advice for Lean and Agile companies but it isn’t just about them.

Read my lips: Its not about Agile, its about Business. Thats all it has ever been about, really.

In the meantime, the business patterns which form the book are still available on my website and will continue to be so. The versions which will appear in the book are a lot more polished and professionally edited, plus the book will have a lot of other new material.

'Objective Agility' & 'What is Agile' presentations online now

Last week I presented a revised version of “Objective Agility: What does it take to be an Agile company?” at Skills Matter in London. The presentation is now on my website (Objective Agility: What does it take to be an Agile company?) and Skills Matter have a PodCast available too (Objective Agility PodCast), so you can here my words too!

By the way, I will be teaching my Agile for Business Analysts course at Skills Matter in November if you like what you see and would like to see more of this.

I’ll be presenting Objective Agility again at Agile On The Beach in Falmouth in a couple of weeks. There are still a few tickets available for the conference but the early bird rate is about to expire.

As part of the build up to Agile On The Beach I did a webcast a few weeks ago entitled “What is Agile?”. The slides for this are also on my website (previous link) and the AgileOTB folks hope to have a podcast available soon.

Two more business patterns: Customisable Product, Customer Understanding

At this year’s EuroPLoP conference I presented two more business patterns for workshop review. There are:

• Customisable Product
• Customer Understanding

I have now updated the paper with the workshop comments and the patterns can now be downloaded from my website. The other patterns in the Business Patterns series can also be downloaded.

These are the final two patterns for my forthcoming book Business Patterns for Software Developers. The text of the book is just about finished and it will be going to the publisher soon. It should be available early in the new year.

Is "Faster Better Cheaper" axiomatic in Agile?

“Faster Better Cheaper” - a cry heard often from management types!
Or at least engineers think thats the sort of thing managers say. Personally I don’t think I’ve ever actually heard a manager say it but I have heard engineers say managers want it “Faster Better Cheaper” with scorn in their voice.

The idea that you could have it (whatever it is) “Faster Better Cheaper” seems to have gained popularity when Daniel Goldin was the head of NASA. As many engineers, including myself, are quick to point out the original quote was “Faster Better Cheaper, choose two”. I say “original quote” but I have no idea who said it first, WikiQuote doesn’t help, and Google doesn’t shed much clear light. (Anyone out there know?)

Whatever, last week, I heard another engineer-type say that he believed management wanted “Agile” because it was “Faster Better Cheaper” and, not for the first time, it got me thinking. Is Agile Faster Better Cheaper?

My knee-jerk reaction is “No!” but the more I think about it the more I think it might actually be so.

Firstly, Faster than What? Better than What? and Cheaper than What?

Presumably “Faster than traditional Waterfall development”. Or at least, what-ever-it-is-you-are-doing-now, which is probably to some degree based on waterfall.

In the short run, when a team are first changing the way they work thy are quite likely to go slower than they would do otherwise simply because they are learning something new and not using a set of practiced reflexes.

Agile most definitely promises to deliver something sooner - if you want it, plenty of business folk seem unhappy with getting things too soon or too often. But Agile will NOT deliver the whole thing Faster, it can only promise to deliver a part sooner.

The logic of Agile recognises “You will change your mind when you get something”, meaning, if the business/customer sees part of the final thing they will add, remove and change the thing they originally asked for. In some cases, and I have seen this happen, they remove work. Lots of work.

Thus, the whole might come faster, but the whole is smaller than the thing you thought it was.

So, Faster - in the short run no, in the medium run you should get something sooner, and in the long run, Yes, it will look like Faster.

What about Better? - how are we to define better: better quality (fewer bugs), better suited to need, more fully filling the initial request.

Yes: Agile promises better quality (fewer bugs). Indeed if you don’t improve quality and remove bugs you are going to have problems doing Agile. If you do improve quality - fewer bugs - then the evidence suggests you will have shorter schedules, i.e. you will go faster.

Better suited to need: again, Yes. If those who made the requests are seeing something sooner, and having their views on functionality taken into consideration then one would expect the final product to be better suited to need.

More fully filling initial request: No, complying with “better suited to need” means we are not aiming to build the thing we first thought of but rather build the thing we need to satisfy the need.

So, if your criteria for success, for better, means complying with a requirements document that was frozen at some arbitrary point in time then No, Agile falls at this hurdle.

Cheaper: again there is a short run and a long run dimension here.

In the short run the development team need training, coaching and time to practice. Of course you could skip the training and coaching (plenty of teams do) but that means they will need more time to practice (make more mistakes, go down a few dead ends, etc.) Practice time costs.

In the short run I don’t think Agile is cheaper. Indeed, in the short run, if you are doing it properly you will see expenses rise as you have the likes of me come to train and coach your teams. (If you are not doing it properly you probably won’t see your costs rise but they will all the same as teams (slowly) learn by themselves.)

In the longer run, when you’ve finished the training, coaching and when the team are proficient then those extra costs will be gone and you should be cost neutral as least. But if you are not writing, finding and fixing so many bugs, and if parts of the requirements are being left undone - while satisfying your business customer - then costs should fall because you are expending less effort and doing less work.

Even if you are not doing less work costs should fall because on a software project costs are dominated by the number of people you have multiplied by the time you have them. If you are going faster then time is reduced and costs come down.

Therefore the project will be cheaper.

Summary: Better (fewer bugs) provides for Faster (delivery) which results in Cheaper (software).

Agile is “Faster Better Cheaper”, QED.

When you put it like that “Faster Better Cheaper” looks axiomatic over the long run.

The engineer in me hates this conclusion, I’m not prepared to accept it but if I follow the logic it is true. (The engineer in me would love someone to find a flaw in my logic so please go for it! Shoot me down!)

Note the three assumptions in this logic:

• The reference point is the “Waterfall” model of development
• Your definition of better considers a low-bug count important and emphasis fitness for purpose over conformance with initial requirements
• Your are prepared to spend in the short run for quality (defect prevention) to save in the long run

How can we explain this? (Notice the change from ‘I’ to ‘We’, I’m pulling you into the conspiracy.)

Well, if this is true then its not so much that Agile is “Faster Better Cheaper” but that the model we are comparing it with - the traditional (“waterfall”) model - is so utterly broken.

Waterfall breaks Faster Better Cheaper everywhere:

• Waterfall leads to large batch sizes and economies of scale thinking: in software development there are dis-economies of scale so large batch sizes makes everything slower
• Waterfall project managers believe quality (bugs) can be traded off against time but actually the opposite is true. Reducing quality reduces “better” and slows a work down. The same project managers are trained to resist requirements change so almost guaranteeing business customers will be dissatisfied with what is delivered and consider it “worse.”
• Slowing software development down makes it more expensive, remember: Cost = People x Time

Its not so much that Agile is a good model but that the competition is hopeless. It isn’t even a fair comparison. My belief is that Waterfall never really worked anyway, so comparing Agile to Waterfall is a false comparison - bang goes a big assumption.

In which case the, returning to the original question: “Is Agile Faster Better Cheaper?” the answer is really “Depends on what you are comparing it to.”

Dialogue sheet observations

Regular readers will know I’ve been pushing Dialogue Sheets as a new retrospective technique. I have observed a number of dialogue sheet sessions - both the retrospective sheets I’ve made public plus some other sheets I’m still testing. Someone from Motorola Mobility recently asked for my observations on using dialogue sheets and I thought it would be a good opportunity to make my observations public.

As a facilitator I find there is very little for me to do, the sheets are designed to be used without a facilitator. As an observer I have been asked to intervene occasionally. Usually I try to say nothing, if the group is willing they can usually work out what to do themselves. When people in the group are either suspicious (something new) or unsure about the retrospective they usually need a few words of explanation.

The most common sticking point is Kerth's Prime Directive. Some people seem unwilling to accept it for the duration of the retrospective. They start making comments about how they don't think people did not do their best.

I don't like stepping in when this happens because I think the group needs to come to an understanding themselves. This eventually happens even if it is something like "well we kind of ignore that" but it can take a while.

The second thing I have noticed is time: It is not how many questions there are on the sheet that determine how long the retrospective will take but the number of people involved. When there are 4 people it is quite fast, when it is 8 it takes longer. So far I haven't found a good rule of thumb for determining just how long the sheet will take.

Sometimes I intervene to just remind people of the time and move them along.

The other bit which can be difficult is the end questions. They are intended to bring the group to set of conclusions which they can act on. Still sometimes the items are "Better communication" which while right isn't very specific or actionable.

There have been plenty of dialogue sheets downloads now and I occasionally e-mail downloaders for feedback. Of those those who have used the sheets “energized” is the most common comment.

I’m planning to revamp the three existing sheets and add one for project start-up. I’ve also had a request to translate them into other languages - specifically German. Once the revamp is complete I’ll start on some translations. I’m also thinking of creating a mailing list for discussion dialogue sheets.

If you haven’t looked at the dialogue sheets they are free for download, and you can buy pre-printed dialogue sheets.

Skills Matter talk postponed

Apologies to anyone who was planning on attending my “Objective Agility: What does it take to be an Agile company?” talk at Skills Matter tonight. The presentation has been postponed until 1 September.

Paul from Skills Matter and myself agonised this morning about postponing the talk. We both wanted to go ahead but with recent events in London we thought it best to postponed until things were a little quieter.

Quite a few people were booked for the event, I hope you can all make it for September, and those of you who couldn’t make it, we’ll you can come too!

Xanpan

For a little while now I’ve been quietly talking about my new Agile method. The clue is in the name: Xanpan - pronounced “Zanpan”. Most obviously Kanban and XP (Extreme Programming), its a fusion.

Its not so much than Xanpan has any new radical ideas in it, its more than it is a synthesis of ideas from several Agile methods plus my own. It contains a fair chunk of what I would call “product management ideas”.


Xanpan is a little more than that, in takes from Scrum as well as XP, and from Lean as well as Kanban, plus there are other ideas in too.



I’ll write more about Xanpan in future but for now here are the essential features, and where it steals them from. A little rough and ready but I wanted to share this and get some feedback.

From Extreme Programming: The technical practices

• Test Driven Development plus Automated Test Driven Development. If you want to adopt BDD then go right ahead
• Lightweight, near time code review, better still pair programming if developers are happy with it
• Continuous integration: with tests and static analysis tools as part of the build system
• Rough up front design: no up front design is wrong, but so too is design that takes weeks or months. If you need a design session it is probably part of the planning meeting with all the developers around the whiteboard for an hour or maybe two
• Refactoring to keep the code soft - its software, emphasis the soft
• Shared code ownership
• Minimise documentation, accept tacit knowledge as part of the development process.
• Velocity measurement and “yesterday’s weather” approach to deciding what to put in the next iteration

From Kanban: Process flow

• Have a visual board to track progress
• Divide the board into columns which represent you process - probably more than just: Todo, In Progress and Done. Columns are usually come in pairs: a queue then a action which draws from the queue. Work to improve the flow and change the columns as you go
• Work in progress limits - usually on action queues, occasionally queues may be limited too
• Improvement comes from improving flow
• Minimise the time spent estimating, if possible abolish estimates and use statistically derived approximations, i.e. averages
• Cumulative flow charts

From XP / Scrum: Process rhythm

• Use iterations to establish a regular rhythm to the team
• Iterations start with closing the previous one: review work done, update statistics, conduct a retrospective. This is followed by a planning meeting: work is presented, estimated (if need be) and scheduled
• Stand-up meetings where appropriate
• Burn-down/up charts where appropriate
• User Stories are the default requirements capture tool although Use Cases, Planguage and more informal approaches are acceptable too. If using User Stories then do not use more than three levels of division

For task management Xanpan builds on Blue-White-Red, which is itself a XP/Scrum fusion.

From Lean: Culture of improvement, Kaizen and Learning

• Retrospectives are not enough: Leaders should undertake personal reflection
• Rehearsal: Teams should undertake formal and informal training together; deliberate practice as Kevlin Henney and Jon Jagger like to say
• Team Coach: Each team should have a coach, different coaches may focus on different things so there may be more than one coach. Except in large teams (over 12 people) and in the early days of adoption the coach is usually not full time on a team. The team is there to help the team adopt practices, learn and reflect.
• Individuals have multiple skills and should all muck in, but there are specialists in some areas.

Somethings Xanpan doesn’t take:

• Kanban’s stop the line quality control: sometimes it is appropriate, sometimes it isn’t. Keep regular retrospectives, part of your rhythm
• Scrum Master: teams will have a Manager or Project Managers, or may be lead by an non-commissioned manager, e.g. Team Leader, Technical Lead, or similar
• Anti-manager ethos: Management is part of the solution, not the problem. Unfortunately the quality of IT and development managers is general is poor; good management can be really powerful

Something Xanpan doesn’t like but doesn’t outlaw (because it can’t):

• Matrix management
• Distributed teams, particularly teams in different timezones

Something I find missing in all Agile methods to date is the right balance between engineering and requirements. Therefore....

Xanpan broadly follows the 10 Step Requirements Model but recognises this model is not broad enough to cope with all scenarios. No model ever will be.

On requirements Xanpan believes:

• There should be a dedicated requirements role staffed by a trained/experienced Product Manager or Business Analyst; both is probably overkill
• There should be a clear business case setting out why a team or project exists. The business case is not a shopping list of features, nor is it a large document but it should allow someone to decide the work is finished
• Customers are not the only stakeholders. Each stakeholder will make their own judgement about the success or failure of work therefore each stakeholder needs to be considered
• Evaluating the benefits of completed work is as important as deciding what work should be undertaken next. There is no point to developing more software if the software that has been delivered so far is not valuable.

And some more:

• If the team has an architect they should also code; the architects role is to help create a shared understanding of the architecture and educate less experienced team members.
• Teams should be kept together and not broken up when work finishes or changes; bring the work to the team, not the team to the work
• Block and impediments should be tracked to see which ones occur repeatedly. If a team cannot resolve an impediment themselves then the leader or manager steps in, in effect it is the start of an escalation
• There are three version of Xanpan: Evolutionary, Incremental and Iterative - these build on the ideas in my Agile Spectrum article. Iterative is the most compatible with existing corporate culture and project management methods - salami slice requirements; Evolutionary is the extreme - goal directed working
• Xanpan adopts the three planning layers described in my Three Plans for Agile article and encourages Goal Driven Projects.
• Xanpan believe management has an important role to play in the development process (but a lot of development managers are not up to the job)
• Whenever possible put bug-fixing in a separate stream of work; but staff it with the same people, rotate people
• Keep teams together: bring the work to the team, rather than the team to the work

Underlying it the Xanpan philosophy is about:

• Quality is free: automate as much testing as you can, although you probably can’t automate 100%


• Prioritisation is vital and omnipresent: there is no such thing as too little time, just mis-understood priorities


• People are key to good software development but there aren’t enough good people to go around, so we need to improve the people we have and help them work more effectively. And we need to grow more good people


• Agile is not about what you do but what you achieve; measure outputs not inputs


• You can always improve


• Customer/End user involvement is key to building a successful system.


• Xanpan is a pick-n-mix of bits of various development methods and adopters are encouraged to continue the approach


• No process or Methodology can cope with all situations, and if it could it would be too big to write down or learn, there adaptation is always required and people need to think


• The process should be changing, if you are doing Xanpan the same in six months as you do today you aren’t doing Xanpan

Cornwall nominated fror Agile awards

I’ve tweeted and blogged about the Cornwall Agile Programme before - something I like to call the ‘Cornish Software Mines’. So it is with great pride that I’m please to announce that the programme has been nominated for an award.

The Agile awards are presented as part of the Agile Business Conference and ‘Agile Cornwall’ has been nominated under the ‘Best use of Agile in the Public Sector’ - officially the programme is part of the ‘Convergence Programme for Cornwall & Isles of Scilly’ so the nomination is under this name.


This autumn I’ll be delivering a couple of case studies of the programme and the companies which have benefitted. The first of these will be, fittingly, at the Agile on the Beach conference in Falmouth. This isn’t a coincidence, the conference is a spin-off from the programme. This will be the most comprehensive case study as it will include presentations from several of the companies who have benefitted from the programme.

A few weeks later a cut down version of the case study will be appearing at the Agile Business Conference. If you miss it there I’m sure it will have another outing before long.

In the meantime, I’ve posted a case study of the work I’ve done with Sullivan Cuff Software on the Software Strategy website.

Retrospectives - common or not, a small survey

I’m still experimenting with Dialogue Sheets for Retrospectives (the download page, and my earlier blog entry) and so are other people. Of the feedback I’ve had so far it has been overwhelmingly positive. Perhaps people who aren’t positive don’t bother trying them.

The sheets are free to download but I do request people register. My objective here is to obtain more feedback. Periodically, like today, I get the e-mail addresses of those who have downloaded and I send a polite note saying “Any feedback?”.

In registering I ask a couple of other questions. One of these questions is: “How often do you hold a retrospective?”. I thought it would be interesting to share the results of this data so far:

How often do you do a retrospective?

38%	Every two weeks or more often
21%	Never
16%	At least once a month
15%	I am a retrospective facilitator and so hold many
6%	Rarely
3%	At least once a quarter
1%	About every six months

This is good to see, about 54% of people are holding retrospectives with the frequency you would expect from a Scrum, XP or other type of Agile team. But sadly the second biggest group is never holding retrospectives, 21% of people. And 10% are holding them rarely or very occasionally.

Now think again, this data is not representative. 15% of people are not retrospective facilitators (e.g. Scrum Masters, Agile Coach, etc.). The people who download these sheets have an interest in retrospectives, this group is self-selecting.

The implications of this are that an awful lot less than 54% of people are doing retrospectives with anything near the frequency one should expect from an Agile team. Given that retrospectives are the primary means of learning in an Agile team I suspect that means that an awful lot of teams are are not really practicing Agile as described in the books.

I have long suspected that retrospectives are actually one of the more advanced Agile techniques and are far from common. I think this data supports that argument, but at the same time I think they are more common than I tended to think, maybe thats the progress of 3 years.

Abandon Hope all ye who enter Agile

Earlier this week I had a conversation with Benjamin Mitchell about an entrepreneur he’d done some work with. Benjamin had been suggesting a lean start-up approach to the business in which the entrepreneur worked to validate his business idea early and gradually build the product one validated step at a time.

Things didn’t go well, the understanding I came to from Benjamin’s story - although his might be different - was that the entrepreneur was very attached his business idea. It was pretty well formed and he could imagine it working. In popular culture entrepreneurs are often seen as iconoclasts who battle doubters, naysayers and banks to bring their ideas to life against the odds. Benjamin was just another doubter.

Now I tend to agree with Benjamin’s approach because it is very difficult to tell if a new business idea, indeed any project, will work until you try to build it. But building it is expensive so I advocate a “Try, fail fast, fail cheap, move on to the next thing” approach - I don’t think this is that different to lean start-up thinking but as I’ve not read the lean start-up book I don’t really know.

I don’t just advocate this approach for new businesses, I advise it for established businesses embarking on a project. My logic is: you can’t be sure a product will work in the market till you try it, nor can you plan a project until you know how fast (velocity) the team will work. In fact, until the team trys to work together and build something you don’t have any data on which to project plans. Making estimates without data is little better than guessing.

Thus, I advise a Mao like approach to corporate projects, “let a thousand flowers bloom” - start multiple small projects, small teams and use portfolio review to remove those that don’t seem promising and reallocate resources to those which do seem promising - or launch more experiments.

Unfortunately, like the entrepreneur, people get very attached to the idea of a project so it gets momentum. Big corporations respond by only starting projects in which they have a high degree of confidence. That means they do a lot of pre-project work - planning, designing, etc., this in turn makes it difficult and expensive to start projects, which in turn means that any project that does get started already has momentum. And it means that once underway the project finds it difficult to cope with change.

It seems to me that successful projects in corporations are often the result of one person who wants the project to happen and does what is necessary, not unlike entrepreneurs. In both cases a strong willed, capable, person makes something happen. That person needs to be motivated, they need hope, they need ambition and hope. Such people are going to kind who ignore naysayers and doubters.

The Agile approach - try, gather data, evaluate, or my “fail fast, fail cheap, retry” approach are not going to go down well with the kind of pig-headed, obstinate, passionate, do-anything-it-takes, approach which has made many successful projects a success. Quite the opposite.

Interestingly Benjamin had another example of a company which does take the “fail fast, fail cheap, try again” approach and was staffed by passionate, hopeful, people. From what he told me these people were passionate about the process as much as they were passionate about the products they tried. They could accept regular failure of good ideas because they saw the process working, and perhaps more importantly, could transfer their hope to the next idea. There was always a next idea.

What I am saying is: Don’t rely on hope to get your projects done, use data - be empirical not emptional.

Finally, I should say, that while I talked much of this over with Benjamin he would be the first to point out unvalidated steps in my thinking, and ungrounded assumptions I’ve made. So maybe this entire entry is really a hypotheses. The hypothesis is:

“The trial and error approach of Agile is unlikely to please those who get passionate about an idea and want to move heaven and earth to make it happen.”

I’ll let you, dear reader, decide for yourself on that hypotheses. If you have any evidence - to support or disprove - my hypothesis please add a comment below.

When did Scrum start loving project managers?

One of the things I’ve always found paradoxical about Scrum (specifically ScrumTM) is its position on management. On the one hand, Scrum is very management friendly - see my Scrum has Three Advantages over XP post. Basically Scrum has done a very good job of marketing itself to managers.

But Scrum is a little like a Monty Python Spring Surprise - “that's our speciality - covered with darkest creamy chocolate. When you pop it in your mouth steel bolts spring out and plunge straight through-both cheeks.”

Hidden inside the tasty Scrum case is a sometimes evangelical dislike of managers, and in particular project managers. Take these excerpts from the Scrum Primer (Deemer and Benefield, some versions have Larman as a co-author too):

• “The ScrumMaster is not the manager of the team or a project manager; instead, the ScrumMaster serves the team, protects them from outside interference, and educates and guides the Product Owner and the team in the skilful use of Scrum.”
• “unlike a project manager, the ScrumMaster does not tell people what to do or assign tasks – they facilitate the process, supporting the team as it organizes and manages itself. If the ScrumMaster was previously in a position managing the team, they will need to significantly change their mindset and style of interaction for the team to be successful with Scrum. In the case that an ex-manager transitions to the role of ScrumMaster, it is best to serve a team other than the one that previously reported to the manager, otherwise the social or power dynamics are in potential conflict.”
• “Note there is no role of project manager in Scrum. Sometimes an (ex-)project manager can step into the role of ScrumMaster, but this has a mixed record of success – there is a
• fundamental difference between the two roles, both in day-to-day responsibilities and in the mindset required to be successful. ”

Indeed I once sat in on a course entitled “Agile Project Management” by a well known Scrum trainer. In response to the a question from a project manager “What does a project manager do in Scrum?” the answer was “If you have a project manager in Scrum you aren’t doing Scrum.”

Take another example, Bas Vodde’s Nokia Test, the final question asks “are project managers (or anyone else) disrupting the work of the team?” Every time I show that question to a project manager we have to discuss it, project managers don’t generally believe they disrupt the team unreasonably. It certainly looks like Bas doesn’t like Project Managers.

A few years ago when Jeff Sutherland spoke in London I recall him saying there was little future for project manager. Many needed to revert to programming (which they had done before project management), a few would become Scrum Masters, a few Product Owners and a very few could continue being project managers on the largest projects.

Sutherland’s own Scrum Handbook seems pretty clear: “there is no team manager or project manager in Scrum.” (Actually, if you read what else the handbook says about project manager it looks like the text is taken directly from the Scrum Primer, Sutherland seems to feel the same way as Deemer, Benefield and Larman.)

Given all the anti-manager, specifically anti-project manager project manager noise from some in the Scrum camp I found it surprising a couple of months ago when I noticed that Scrum Master Certificate now come with the implicit endorsement of the Project Management Institute.

For example, take Martine Devos’ London Scrum Master Course, it is worth 14 PMI Professional Development units. Jeff Sutherland’s own Scrum Master course boast 16 PDUs (one assume these are PMI units although he doesn’t state so explicitly.)

So, if the PMI are now crediting Scrum Master Courses, and the Scrum folks are making their courses compliant with PMI rules then one assumes that the two are reconciled. Why would a project manager who is intent on being a Scrum Master, and therefore no longer being a project manager, want credits?

Maybe Turkey’s are voting for Christmas. It looks odd for me.

Actually, to be fair to Scrum, the situation is a little more complex than I’ve laid out here. Looking a little bit further into Scrum history we find the following.

Schwaber and Beedle in Agile Software Development with Scrum say “The Scrum Master is a new management role introduced by Scrum” and a little later “The team leader, project leader or project manager often assume the Scrum Master role.” This final statement describes what I have seen happen most often in practices.

In Agile Project Management withScrum Schwaber says “The Scrum Master fills the position normally occupied by the project manager. I’ve taken the liberty of redefining the role.”

One explanation might be that the view of the Project Manager has changed over time. Initially the Scrum originators saw project managers as candidates for filling the Scrum Master role - or at least not the source of problems. As Schwaber almost says: it is the same role filled in a different way.

Later Project Managers, and maybe all managers, came to be seen as a problem, and, most recently as it becomes clear that Project Managers can be Scrum Masters and can be a force for good on a project the position has returned to the original view.

Alternatively it might be there are multiple opinions on how Scrum and the Scrum Master relates to the traditional Project Manager role. It benefits some people, at some times, to claim the Scrum Master is not a Project Manager. And it benefits some people at other times to reconcile the two roles.

One of the advantages of Scrum, specifically ScrumTM, is that is defines what is it, and is not, much more clearly than “Agile”. However with time this picture is becoming muddied.

Finally, this entry is a bit critical of Scrum, I won’t pretend it isn’t, but really, I’d like to understand what is going on here. If anyone can shed some light on the thinking, whether it changed or not, please add a comment.