ACCU conference 2 of many (Thursday)

Thursday kicked off with a keynote from Linda Rising in which she described her personal journey of discover of, and with Patterns. I’m lucky enough to count Linda as a friend, and I already knew much of the patterns story she told. Still, I had fresh insights and understanding.

For many of those in the room Linda’s ideas and thoughts were completely new. She finished by presenting us with something of a challenge - albeit on originally thrown down by Christopher Alexander - What are you doing to make a better world?

Next I went along to Phil Nash’s talk on Objective-C. He tried to teach Objective-C in 90 minutes. He almost managed it too! I don’t really code anymore but I enjoyed this insight into another language and how Mac’s and iPhones are programmed.

Mark Delgarno of Software Acumen and Helen Sharp of the Open University workshop on “What motivates software developers?” provided plenty of food for thought.

Helen reported on her research which can be summarised as: “Nobody knows” - basically there are many studies, but none of them actually bothered to define what is a “software developer” so they covered project managers, coders, testers, and possibly many other people who are involved one way or another with creating software but have very different approaches, skills and motivation.

For me Thursday’s sessons finished as they had begun with Linda. The good news from this session is that we, as human beings, are hard wired to be optimistic. The bad news is that consequently that means we cannot estimate work and even when presented with evidence we make the same mistakes. I’ve mentioned our confirmation bias before in this blog, this presentation was a though examination of the issue.

A new innovation at ACCU conference this year was the introduction of lightening talks. After the main sessions on Thursday and Friday an hour was given over to short (5 minute), talks and presentation from various people. The speakers and subjects were only agree on the day and several of the talks were only written in the hours before.

I got several nuggets of information from these talks. Two in particular stick on my mind: the Google Web Toolkit as an alternative to Selenium (Paul Grenyer); and the advantages of giving specifications by example rather than by rule (Keith Braithwaite),

ACCU conference 1 of many (Wednesday)

Another year, another great ACCU conference. I counted 9 languages on the conference program, everything from Scala to C++ - yes there was more C++ that anything else but it was far from the only thing going on. Languages plus sessions on design, pattern, management, Agile and neuroscience.

Normally some of the most insightful comments come not in the scheduled sessions but in the discussions over coffee and in the bar afterwards. Sometimes you don’t know these are insightful until after the event when you’ve left and had time to think it all through.

This will always be a special ACCU conference for me because I was the keynote speaker on Saturday. My keynote was well received and the slides ....

So here is a brain dump.

The opening keynote on Wednesday was from Bob Martin - “Uncle Bob” - about software craftsmanship. (I’m watching the software craftsmanship movement with interest and will write a blog about it in future.) Bob suggested that the most successful Agile method - Scrum - was the accidental “winner” of the Snowbird summit in 2001. This was because the Agile manifesto and Agile values said nothing about technical practices.

Scrum, as is well know, does not contain any technical practices. Teams often borrow these from XP. Because of this Bob believes Scrum is a subset of XP - a subset which just deals with project management. Without the technical practices productivity falls off and teams make a “big mess faster.”

Bob’s answer is a renewed emphasis on technical practices - hence software craftsmanship.

Bob is a great showman and his talk was very entraining, if you get the chance to see him do so. Along the way made several Scrum jokes and was quite critical of Scrum Master Certification.

Someone like Bob can do this, Bob has the reputation and standing to be able to take pot shots from a safe(ish) position. This is good because someone needs act as a foil.

Jutta Eckstein’s session on transparency in Agile was interesting. It reminded me of the session I ran here a few years ago called “Exposing problems / Creating awareness.” The these was similar: “Agile as a problem detector.” The session was largely a discussion between Jutta and the various people in the room.

Keith Braithwaite presented the latest instalment of his “Measuring the effects of TDD” research project. I’ve seen Keith talk about this subject before and over time his presentation is getting more and more compelling. His sample size in increasing and its looking like a more convincing case.

If you don’t know the work have a look at Keith’s blog. To summarise it: you can measure code complexity using a measure called cyclomatic complexity. Keith’s research shows there is a correlation between complexity and automated until tests. Namely: code with a high level of automated unit tests exhibits lower complexity metrics and people comment they are “happier” with the code.

Having discovered the metric Keith is still trying to understand the cause-and-effect relationship and, perhaps, more importantly, uncover how this relationship can be used to improve matters.

Nicolai Josuttis finished the day with a second keynote. You could call it the antidote to software craftsmanship: Crappy code.

Although Nico made it jovially it had a serious point: there is a lot of bad code out there and it is likely to be around for a while to come. The economics of software development and business tend to perpetuate this situation. Therefore, we need to get used to crappy code and find ways of dealing with it.

While I agree with Nico I can’t help feeling it was a little defeatist. If we accept this poor state of affairs do we still have hope of a better tomorrow?

Between them Bob and Nico certainly set a debate raging - Good code, or Crappy code?

Wednesday finished with 40 of us in Chutney’s curry house.

Impediments - hidden or otherwise

A comment to my post on “Three ways to run a stand-up meeting” asked: “Can you give an example of the ‘hidden’ impediments?” so here goes.

First: know the difference between a block and an impediment. A block stops all work, they are seldom hidden because people can’t get on with what they are doing. A impediment is something which gets in the way, slows you down, saps your morale.

Fortunately, or maybe unfortunately, software people are very good at routing around blocks. They find ways to “get stuff done” even if it is not the best way, the fastest way, or the “right way”. So blocks often become impediments. Unless we strive to impediments our teams will not perform to their maximum.

One nasty form of impediment is the “I’m blocked so I’ll make work to look busy.” The work which is created is seldom as valuable as that you are blocked from doing and even worthless.

Anyway, examples of impediments, especially hidden one:

• Vendor is not responsive: someone is waiting on a vendor to deliver something and it isn’t happening; or you are waiting on a vendors technical support, or perhaps you need information from the vendor before deciding whether to use their product or someone elses.
• Build problems: the build is broken, or you need someone else to fix their bit, or you are waiting for access rights, or the compiler seems to have a bug, or ... the list goes on
• Waiting on another team: particularly test, source control, user interface, tech support, etc. Idealy we want each team to be vertical, that its, to be self sufficience, to contain the skills and authority to get a job done without needing action from others. But this doesn’t happen as much as it should so you need another team to do something. At this point problems occur because the other team doesn’t have the same prioirities as your team
• Upgrading compiler, OS, app server, etc: some piece of software (or perhaps hardware) needs to be upgraded. So instead of getting on with the task in hand someone is diverted to upgrade the system. And when this takes long than expected, or hits problems, then it becomes an impedimant and even a block.
• Bug - typically in a library: you can’t get on with your own work because of someone elses bug. Ideally you can just jump into their code and fix it but.... if the bug is in a third party library then it not quite so easy. And if the problem is with the compiler, or a vendor you get to one of the problems above.
• Database changes: databases teams seem to work in a completely different temporal dimension . Too often I meet teams who can’t progress because a DB schema change is needed, or the meta-data is a problem, or .... The real solution is two fold: simplify designed and reduce the need for complex databases, and ensure the team has database skills itself.
• Waiting on a management decision: all too common, management can’t make their minds up whether to do it like X or like Y. A particularly nasty version is when you’ve asked for something “Can we buy Z?” and rather than say “No” you are kept hanging on. If they just said “No because...” you could address the reason or find another way of doing it.

When team members can’t report real achievements in a Scrum meetings they usually report ”what they have done“ which includes ”why there isn’t an achievement“. Thats the point at which these kind of impediments are mentioned. A good Scrum Master / Meeting Leader / Agile Coach will spot these issues and move them to impediments.

If you have any more examples of impediments please add them as comments below.

Russian translations

Only a few weeks ago I was pleased Mark Nakamura translated some of my blog entries to Japanese. Now I’m being translated to Russian.

Ivan Doroshenko has translated Three ways to run a stand up meeting and Lean & Agile the same but different. Hopefully Ivan will be translating some more, so Russian speakers keep an eye on his blog.

New writing and events: Downturn & Product Management

Its been a busy few weeks with presentations to BCS Bristol, BCS Kingston & Croydon, Hewlett-Packard and a trip to Switzerland to deliver some Scrum training and coaching - not to mention a family holiday.

The couple of weeks is also busy with some public events coming up:
• ACCU Conference: I’m delivering a keynote, a regular session and co-ordinating a set of pattern talks with Kevlin Henney. (This starts on Wednesday next week and I believe there are a few places remaining.)
• Brighton Agile Forum have invited me to speak on 11 May
• ACCU London is hosting Jeff Sutherland on 21 May

I’ve had some new articles published elsewhere recently:

• Agile in the Downturn is in this months Agile Journal
• ACCU Overload 90 April 2009 (whole journal download) continues my On Management series with a look at the Product Manager role (article download)
• A New Approach to Software Project Management in CEO Today: this is one of those virtual magazine formats so you have to flick through the pages. And for reasons I don’t entirely understand my company name (Software Strategy) and not my name has been used as the author. Can’t say I’m entirely happy the way this piece has worked out, its an experiment in reaching a different audience.

Actually, if anything I’ve been writing less recently but these three have been in the works for a while.

Three ways to run a stand up meeting

Stand up meetings - those short meeting that a team holds every day to update each other. Also called daily meetings, Scrum meetings and probably some other things. The basic format is well known: three questions -
• What have you achieved since we last met?
• What are you working on next?
• Do you have any blocks or impediments?

Yet despite this simple sounding format there is still a lot of subtle detail and variation possible.

Take the questions for example:
• “since we last met” as opposed to “yesterday”: if the meeting happens every day there may be little difference. But there are weekends, people have holidays and occasionally meetings are skipped
• “what have you achieved” as opposed to “what are you working on”: this makes the questions results oriented and many people find it hard to answer because while they have worked on stuff, there is nothing they can yet say they have “achieved”. (I don’t usually enforce this distinction until the team and the developers are experienced in answering the question and actually delivering stuff. To enforce it early can make people negative as everyone reports “nothing”.)
• “blocks or impediments”: blocks might limit discussion to just those things that had stopped work, including impediments means we ask about things which are slowing work down

Still asking the same three questions every day can be boring and sometimes you want to vary the questions as long as you stick to the same key elements I don’t mind.

What is less well know is that there are - at least - three different ways of running the meeting to ask the questions.

For years I asked the three questions in three rounds. With everyone circled around the whiteboard (or if you prefer Kanban board or information radiator or just task board) I would ask the first person “What have you achieved?”, then the second person, then the third person and so on.

Once I completed that round I would start over a second time with “What are you working on today?”. Again, first person, second person, etc..

And then another round of “Any blocks to report?”

One modification was to ask the third question as an open question to the whole group at once. That can speed things up a little although it might lead to one person’s block or impediment monopolising discussion.

The second way to ask the questions - and the one I think most teams use - is a single round asking all three questions at once. So the first developer reports achievements, current work and impediments in one monologue, then the next developer and so on.

I’ve recently started to favour this format simply because it is a little faster so easier to keep a meeting short. But, this week I realised it might be a false economy.

I did some Scrum training for a team in Switzerland and stayed on to do a little coaching. One of the problems the Scrum Master had identified was failure to identify impediments. I stood in their Scrum meeting and heard each monologue, within the monologue people talked about impediments but did not explicityly identify them as such. They were “hidden”.

Splitting the three questions up into three rounds encouraged team members to categorise what they are saying into Achievements, Work and Impediments. It is very clear what is Achieved, what is In Progress and what is slowing them down.

The third way of running a to do it from the team board. Rather than ask each team member in turn you point to a task card on the board and say: “Who’s working on this task? - what’s happening here?” If the task is in the done column it is (obviously) an achievement (only point to done cards that become done since the last meeting.) If the task is in the “In Progress” column then its work that is ongoing and deserves discussion.

After the card have been discussed there are two questions to ask:
• “Anyone not reported?” - which may be the case if two people are working on the same card, in which case they just say “Same as <who ever>”. If anyone has much to say at this point then maybe there should be another card on the board. Why is a team member working on something not agreed?
• “Blockers or impediments?” - which needs saying collectively.

This format works well when the whole team is split into several sub-teams who work together or when some people are remote, in which case someone locally can report for them.

I’d never thought of this approach as anything special until few weeks ago. I read something by David Anderson and he said that his Kanban teams had taken this approach. David went on to say that this format allows for bigger teams because one person can report on a group effort. This formay also deals with remote people more easily.

Finally, there is a common problem I see with team reports: lack of focus.

The cause is usually the lack of a board showing the task (the team board, the Kanban board, the information radiator - I’m repeating myself). Without a board with the task cards on it the team lacks a focus point. When someone can point to a card (i.e. piece of work) and say “I’m doing this” it is physical. Without it the team members end up saying something like “I’m working on that feature which does blah blah blah and ...” which takes more time and looses focus.

Lets be clear: a board showing current tasks brings focus to team reporting.

My Swiss team really want to use an electronic tool. I’ve suggested they look at getting a big plasm screen on the wall to show the status, I’ll be interested to know how it works out.

Mr Scrum - Jeff Sutherland in London

On Thursday night I was lucky enough to meet Jeff Sutherland - one of the originators of Scrum - and hear him speak. The event was organised by the Scrum Training Institute and hosted by BT. If you missed this talk I’m please to say ACCU London will be hosting another talk by Jeff in May. Details have yet to be finalised so keep en eye on this blog or join the ACCU London mailing list - although I can say it will be a different talk, so if you did go to this weeks you may well want to go again in May.

So what did Jeff say?

Well there was some drum beating for Scrum. About how Scrum can improve team performance by 200%, 600% or even 1000% in some instances. And about how one venture capital firm in Silicon Valley now only works wit companies who develop software using Scrum.

Jeff also had some notes from Scott Downey on how to get a team started with Scrum. The interesting bit here for me, was that Scott uses some of the same techniques I do when I’m coaching Agile and Scrum teams. For example, I like to run very compressed iterations at first, 1 week instead of 1 month. This helps the team get used to the routine.

Scott is also very keen in “information radiators”, also known as “Kanban boards”, or for the non-techies White Boards. And like me he avoids the use of tools to manage the tasks.

Almost every team I introduce to Scrum or Agile says: “We don’t need a white board, we have a tool”, or “Why should we use a white board when we can get a software tool.” Personally I don’t like tools but I know some people do and some people find them useful. So my answer is: Lets start by not using a tool, using a simple board, and once the team has settled down lets look at whether a tool would be better

A couple of other things Jeff said which I think worth noting.

According to Jeff, Scrum is NOT ...
• a methodology
• a defined process
• a set procedures

One of the questions that often comes up about Agile and in especially Scrum is: “What is the future of the Project Manager?” so I took the opportunity to ask Jeff directly. In summary he said:
• some Project Managers will become Scrum Masters: although the two roles are very different some Project Managers will find the Scrum Master role natural
• some Project Managers will become Product Owners: again, these are different roles but some Project Managers will find the Product Owner role a logical move
• some Project Managers - presumably on large projects - will take on a wider brief, possibly called Programme Manager which involves them less with the development team and more with the groups who are impacted by the work

Jeff was keen to point out that there are still jobs for the those who now hold Project Manager jobs but that those jobs will be different.

Finally, Jeff also named the phenomenon of customers not knowing what they want until they see it. He called it Humphrey’s Law. I’ve not heard it called this before and I can’t find a reference to it so if anyone out there knows it then please send me a link.

For me Jeff’s talk was well timed - I’m off to Switzerland today to teach a Scrum course.

Burn-down charts: The Good, Bad, advice and alternatives

I’ve never completely accepted burn-down charts. I know they are a staple of Agile development and I’ve even used them myself but I’ve never been completely happy with them.

The reason I’ve never been happy with them is that they require and initial figure to burn-down. You need an initial number - in hours, days, points or what ever - to seed the chart and then you burn off some amount and you see the line falling.

The problem is that initial figure, it has to come from somewhere. In generating that figure you need some description of the work to be done, i.e. requirements, and you need some estimation of how long it will take. Both known requirements and estimates have their problems.

Burn-down charts were popularised by Scrum where there are, strictly speaking, two types of burn-down chart: the product burn-down chart and the sprint burn-down chart. Since in Scrum the work to be done is fixed for the iteration (barring a major issue) the requirements are pretty well fixed so that isn’t s a big issue, and the estimates are the best known at the time which isn’t too much of an issue.

In a 4 week iteration, with a Sprint Goal I can see were the Sprint burn down has a place. But if the iteration is 2 weeks or 1 week I’m not sure it is worth the trouble. And if there is no Sprint Goal then what does it show? Progress against a collection of tasks?

My bigger issue lies with the Product Backlog burn-down chart. Because the items haven’t been committed to a Sprint then there is every possibility they will change. Indeed, the more Agile you are the more likely they are to change.

However, once the estimate gets out there, once it becomes known the team estimates the project at 100 points of work people start ask questions when the number changes.

There is also an inbuilt assumption that there will an end to the development work. When the thing under development is a Product or a Programme development work may continue for a long time, perhaps ever. So its wrong to talk about end dates - and also wrong to talk about projects because projects, by definition, have an end date.

Then there is the estimate: it is made when there is little information known about the requirements, the system or how much work is needed on the code. So the estimate is going to be pretty inaccurate. The best that can be said is that its a finger in the air.

But, the estimating process normally takes a lot longer than sticking a finger in the air. If you involve the whole team it takes even longer. If you don’t involve the team then its the estimate of some “leadership” which runs counter to the principle of self-organizing teams.

More detailed estimates also leads you into design decisions - because how can you estimate if you don’t know what you are going to do? However, its best to delay design (and other) decisions until the last possible moment because the information on which they are based can change.

The first Agile teams I coached skipped burn-down charts altogether. Then one day a colleague on another project (yes, you Ed), told me his Project Manager felt a lot happier now there was a burn-down chart in place for his work.

And thats where much of the value of burn-down charts. They give Project Manager types something to stare at. Burn-down charts look enough like a Gantt chart to reassure people.

All that said I use burn-down charts. Yes they are useful. They are genuinely useful when you have a piece of work with a defined end condition, i.e. a genuine project.

Of course you have to overcome the estimation problem. The trick is to accept estimates are just that, estimates, do them fast, don’t worry about them. Then do plenty of statistical tracking. How much new work is found each iteration? How much is removed? Then project forward best and worst case.

I did this with a team last year. At first we had a burn-down chart with but I refused to give a end date. After a few weeks we had data on how fast the team was working, what new work was being found and how much work was being removed. From this I could forecast a best case and an worst case end date.

That’s important: there is no definitive end date, just a range of possibilities. The more data you have the more accurate the estimate should be but its no guarantee. Giving a range of dates might not make you popular but it ensure people remember there is no certainty.

Burn-down charts are not the only tool for tracking team progress. Some people prefer burn-up charts. At first I thought this was for psychological reasons - up is more positive than down - but its also easier to add more work to the top.

So, if a team is doing 10 points a week on a 100 point project its easy add another 20 points to the chart and show the team now have to get to 120.

Taken to the extreme you get cumulative flow charts - often used by Kanban teams - shown in this diagram:



These show both the total work done and the total work to-do as constantly rising lines. If total done ever matches total to-do then its game over. This technique is open ended, the work is ongoing. And it is only necessary to estimate a little bit ahead.

So, if you do use burn-down charts:
• Consider using burn-up charts or cumulative flow diagrams
• Don’t make estimates too detailed; don’t worry about over estimating, it will all come out in the wash anyway
• Certainly don’t spend too long making detailed estimates
• Track work done, work discovered as you go along and work removed; you don’t need to put these on the charts but keep the numbers so that you can...
• Use statistics to project a range of end dates
• Don’t let anyone thing a burn-down chart is a substitute for a Gantt or Pert chart

And when people demand a done date remind them that the old method didn’t give them a done date either. Yes it gave them a date but it was never the actual date.

BCS Kingston & Croydon - Tuesday 24th

Back in February the snow came down and the evening talk I was due to give to BCS Kingston and Croydon was postponed. (The snow closed Kingston University which was due to host the event.)

Well it was rescheduled and is now happening this week on Tuesday 24 - Lean Constraints Action! on the BCS Kingston & Croydon site. Admission is free and there is no need for pre-registration this time.

10 things to know about Kanban software development

1) Kanban software development originated by David Anderson. Many of the practices and heuristics have been seen on other Agile teams before but they were first described as a cohesive whole by David.

David’s innovation was to explicitly limit the work in progress. This had been done by other Agile teams before but in Kanban there is a well-known limit on the number of work items which may be worked on at one time. The limit is usually quite low, in the teams I have worked with the limit is approximately the same as the number of developers on the team or slightly less.

2) Many Agile teams use a white board to track work during an iteration. Typically this is divided in to three sections: “To do”, “In progress” and “Done.” When a physically limit to work in progress is imposed it becomes useful to split this down further. Work in progress could be: in development, in test, in analysis or in other states. Each of these may have its own limit.

Thus, the second innovation in Kanban is to split the board into many other columns. Sometimes buffers are placed between the limited columns, these may be labelled “ready”.

3) The Kanban test: In November 2008 on the Kanban mailing list, I asked: “What is the defining characteristic of Kanban that make”, in response David Anderson said:

“For me it is simple... Are you limiting work-in-progress? Are you signaling to pull work from an upstream process? If it is a WIP limited pull system, it is kanban!”

Which give a test for Kanban and is pretty much points #1 and #2 above.

4) The Kanban approach was summed up by Karl Scotland when he said (on Kanbandev again):

“a subtle difference between kanban and typical agile processes such as Scrum. Scrum focuses on being agile which may (and should) lead to improving. Kanban focuses on improving, which may lead to being agile. However, being agile itself is not important - it just happens to be the best way we (or at least I) know at the moment.”

5) The best place to learn more about Kanban is the Kanban dev Yahoo list.

6) There are currently no books on Kanban software development. There is one that comes close, Corey Ladas ScrumBan - Essays on Kanban Systems for Lean Software Development. I’ve read this book and I’m not rushing to recommend it.

7) Kanban derives more directly from Lean Thinking and Lean software development then many of the previous Agile techniques. Once you apply the innovations in 1 and 2 above just about everything else follows.

8) Kanban teams focus more on work flow, the time it takes to get work from one end of the pile to the other. In the extreme Kanban teams dispense with work estimating (which do not add value), iteration planning meeting (planning is an ongoing process), fixed release dates (they release when it is ready) and scheduled retrospectives (they have a stop-the-line approach to address a problem when it is seen).

9) Originally the Japanese word Kanban is two words kan and ban; kan means visual and ban means card. Like many people I talk about Kanban cards but strictly speaking I’m duplicating myself.

In Lean systems the cards are used to pass information, to signal when limits are reached, to signal an out-of-stock situation or some other trigger for action. Thus Kanban has come to mean more than it literal meant, and now the word is used as the name of a method it means even more.

10) Whenever I blog about Kanban I get more hits on this blog. It seems people want to know about Kanban. Actually, I just discovered that at the moment this blog appears on the first page in Google if you search for “Kanban software development.”

The lesson: Kanban is still new and still in its infancy.

For my previous description of Kanban see Making sense of Kanban (and some doubts), Agile and Lean - the same but different, Notes on a Kanban software development experience and there are some other pieces too if you search the blog.