Agile on the Beach: Falmouth, September 15-16, 2011

I’ve mentioned some of the work I’ve been doing in Cornwall over the last eight or nine months in this blog a few times, and anyone who follows me on Twitter will have seen my Tweets about the “Cornish Software Mines.” Well, one of the spin off of this work is a Cornish Agile Conference - Agile on the Beach.

Agile on the Beach is happening in Falmouth on Thursday 15 and 16 September 2011. Speakers include: Kevlin Henney, Tom and Mary Poppendieck, Rachel Davies, Steve Freeman, Jon Jagger, Jason Gorman and quite a few more - see here. Some more speakers will be announced nearer to the conference.

Tickets went on sale today, early bird price is £230 - if you are quick you might get one of the limited number of £195 tickets available.

The organisers (and yes, I’m one) hope that people attending the conference will take the opportunity to extend their visit to Cornwall into a long weekend. After all, Cornwall is much better know as a holiday destination than a software development centre.

All the details on the Agile on the Beach website. In addition, if you plan to attend sign on to the LinkedIn group and you can see who else will be there. More details are being announced all the time so to stay up with the latest announcements follow the Twitter hash tag #AgileOTB and Twitter User @Agileonthebeach.

Parrot cages, the true story

I heard this story a few months ago from Benjamin Mitchell, it is good and I wanted to believe it so I did. (Benjamin by the way tells it with a good deal more embellishment.)

Anyone who has had a course from me in the last six months has probably heard me retell the story with a large health warning saying “I’ve heard this story, I believe it but I don’t have a source to prove it.”

Now, Toby Parkins has tracked down the link. So here it is: Parrot cages, the true story, care of the BBC - so it must be true!

Dialogue Sheets at Skills Matter - 19 May, 6.30pm, free

I’ve blogged before about the Retrospective Dialogue Sheet technique. Later this month I’m doing a Skills Matter In the Brain Session about this technique. This will last about an hour to an hour and half and is free for all to attend, register at the Skills Matter website.

I won’t actually be doing much talking, I’m bringing along some specially created dialogue sheets which will give everyone an brief experience of holding a retrospective using dialogue sheets.

For more information about dialogue sheets see my previous blog entry, or the www.dialoguesheets.com website.

Light Touch Agile Coaching in the Cornish Software Mines

I’m off to Cornwall again next week for my monthly visit. Anyone who follows me on Twitter (allankellynet) may have noticed that every few weeks I put out a few tweets along the lines of “On my way to the Cornish Software Mines again”. They may not actually dig source code out of the ground in Cornwall but there are some very active software companies there.

Under the auspicious of Grow Cornwall I have been in providing Agile coaching to several companies as part of an Agile programme designed to bring better development practices to multiple companies. To date six companies have been part of the programme to some degree and next week we are adding some more.

In the next few months I hope more of this work will come into the public domain as articles and case studies. There are even plans to hold an Agile conference in Falmouth, to be named Agile on the Beach. (More to follow soon.)

This engagement has really given me a chance to think about my coaching style and develop some techniques and approaches which I’ve thought about in the past but not really codified. One of these I call Light Touch Agile Coaching.

Many description of Agile Coaching assume a full time coach. Scrum specifically just about mandates a full time Scrum Master for each time. The Scrum Master is closest Scrum gets to an Agile Coach, indeed, some people would see them as the same thing. I don’t, and thats worth a full article in its own right but not today.

In Cornwall there are three companies I’ve be working with more than others and a clear pattern to the work has emerged. It goes something like this:

• Companies joining the programme receive a training course over two days, preferably offsite. This has worked best when the net is spread wide and the whole development team and managers are involved.
• The team are allowed a little time to digest this then asked: do you still want to try Agile? The programme is designed with the expectation that they say Yes, in which case the coaching begins. If they say No then fair enough, its is there decision.
• I visit once a month, I spend between half a day and two days with the team. How much time I spend depends on the size of the team and the company, how long I have been working with them, and the particular set of problems they are faced with.
• My visit usually starts with an inspection of the white board - call it a Kanban board or an Information Radiator if you prefer - its the board with all the work on it.
• I read the board, I ask question, I get asked questions, I understand the state of play and get an idea on what needs addressing.
• Sometimes, some places, I sit down with a manager and talk about the last month. What has happened, what progress they have made, what understandings they have reached, what has puzzled them, how the team have performed, and so on.
• Next I sit down with the development team and conduct a similar type of exercise. Sometimes the teams have conducted their own retrospective and I review that, other times this exercise takes on elements of a retrospective.
• (One of my challenges is to make all the teams proficient with retrospectives so when my work is finished they can continue themselves.)
• In some companies I sit with the marketing group for a conversation too. All the companies I’m working with develop software for external customers and some of them have marketing teams. As the development teams have become more effective a lot of my focus has shifted to marrying up marketing with development so the right thing gets built.
• During the course of these meetings I start to construct a list of “Expected next time”: things they managers and teams tell me they will change, or things they will put in place for my next visit. Naturally, as a I talk to people I’m also looking through the last “Expected next time” list and seeing how the team have done.
• I started doing the “Next time” list for my own benefit but I’m open about it with the teams and it has become a useful little tool in both tracking whats changing and getting commitment form the teams.

I like this light touch approach. On the whole it has worked well, I get to help the managers and teams while they keep responsibility and feel ownership. Yes, there are cases were I wish I had been there more frequently, and there are cases where I feel I could just step in and fix it far more quickly, but that is not my role.

While I’ve concentrated on Agile processes and management (both product and more company) coaching we have also introduced more technical coaching. Jon Jagger and Nancy Van Schooenderwoert have helped out here. They have taken a similar light touch approach. In Jon’s case he normally spends his time pairing with the developers on his visit while Nancy works remotely.

There are some variations to the approach outlined above, things haven’t always gone smoothly and one company dropped out of the programme right after the training. But, on the whole it is working and working well.

Some successes to date:

• One company develops a software product for doctors, they recently received ISO-13485 using an Agile process I helped them develop. ISO-13485, for those who don’t know, is a standard based on ISO-9000 which certifies a company to create medical devices.
• The day after completing TDD training with Jon Jagger one team wrote a unit test which found a bug in their live system.
• One manager when asked “So this estimation process and charts are more accurate than the way you used to do it?” replied “This isn’t estimation, that is Mystic Meg stuff, this is science, I know when we will deliver.”
• By adopting Agile one company has won, and is successfully working on, a new website for one of the worlds largest telecom’s companies. The same company has now spawned a spin-off which is born Agile and will work with the same telco in a contract worth over a quarter of a million pounds.
• The three companies I am working with have all been strong enough to hire new staff in this year.

I’m not saying that light touch coaching will work in every case - indeed, at one of the Cornish companies we decided they needed a more intensive programme. I do think that this is a model that can be used elsewhere.

Public training 18-20 May

I’m giving a public training course at Skills Matter in London in a couple of weeks, 18-20 May. This is the Agile Foundations course specifically for Business Analysts. If you are not a BA I’m sure you will find it useful and interesting; you’ll just get a bit more on the requirements and Product Owner side of things than the vanilla version.

As it happens, I don’t give that many public courses, mostly they are private courses for specific companies or invited audiences - like the one I’m doing next week. So this is one of the few occasions to just turn up.

Quiet blog - Business Patterns for Software Developers in progress

Just a brief note to explain why this blog has been a bit quiet lately.

Two reasons.

Anyone who has seen me in the last 5-6 weeks will know we had a big family event four 4 weeks ago. So a lot of my time is going in that directions.

And second, about the time, I finally signed the contract to write “Business Patterns for Software Developers.” The business strategy patterns I have been writing and bringing to conferences for the last six or seven years are finally going to be made into a book.

So, most of my spare time is absorbed by getting the book project rolling. Fortunately most of the text is written but there are still gaps. Plus there is a lot of editing and reviewing to do.

The text of the book is due to be finished in September and the book in the shops in January. Expect to see if on Amazon long before then, once the ISBN number is issued it is in the catalogues and will be picked up by online retailers.

Exciting times.

Slides from ACCU 2011 - 10 Step Model for Agile Requirements

The slides from my presentation at ACCU 2011 conference today are available online, the Agile Requirements 10 Step Model.

Retrospective Dialogue Sheets

I’m at ACCU 2011 this week, in my lightening talk I publicly unveiled, for the first time, Retrospective Dialogue Sheets.

I have been experimenting with dialogue sheets at conferences for a few years now. For a long time my aim has been to use them for team retrospectives. I’ve now produced a few variations and tested them with some teams. Still I need to do more tests, so I’m looking for volunteers.

I’m making these dialogue sheets available as a community resource, without charge. However, there is a catch: you need a large printer (A1 size) to print them. So I’ve set-up a print-on-demand service where you can buy printed versions.

Right now you can:

• See examines of previous dialogue sheets and exercises
• Read more about retrospective dialogue sheets, www.dialoguesheets.com
• Download a retrospective dialogue sheets to print use with you team
• Buy printed dialogue sheets

If you do try one of these sheets please e-mail me and let me know about your experiences.

Finally, I’m still looking for volunteers who will let me watch teams using these sheets. I’m happy to give teams printed dialogue sheets for free if you will let me observe the team doing the exercise. If you can help, please e-mail me.

Strategic Staffing, Or, How many people should I put on this (Agile) project?

One of the questions that comes up again and again in the context of Agile work is “How do I know how many people to put on this project?”

(I’m using the word ‘Project’ in the loosest sense, think “piece of work”)

My usual answer is: Staff Strategically, allocate people to a piece of work in relation to how important this piece of work is relative to everything else you are trying to do. Keep teams together for as long as possible, only make small, gradual changes to teams over time.

This isn’t really a problem when work is underway, you simply sum up the ball-park estimates on the work remaining (in abstract points), measure the velocity (again in abstract points), divide total by the velocity and if you get an answer you like then all is well. If the answer is not soon enough you can (gradually) add more people, and if the answer is very good you might even slow down.

However, the problem really comes when you are starting a piece of work. Particularly if you are bidding on a contract to do work for someone else. The problem occurs because without any data (velocity, sum of outstanding work) you can’t estimate when you will be done, and without doing the work you can’t get any data.

Rule 1: Without data you can’t forecast anything so don’t pretend you can.

So, what do you do?

Option 1 is to pretend you aren’t doing Agile; use whatever methods and techniques you’ve used before, get an estimate (or maybe more of a guess) on how many people you need for how long then get started with those numbers. As soon as you start throw away the plans. Wing it for an couple of iterations, by then you have the data and can re-plan.

Not a perfect solution but one that could work.

Option 2 is to just start. Start as small as you can, I think two people is about as small as you can get - below that you are into the realm of micro-projects which have their own dynamics.

Let this mini-team run for a few iterations and then, as if by magic, you have data.

Once you have data you can: keep as is, grown, shrink or cancel entirely.

Rule 2: Start small, get data, grow later

Starting small is essential because it reduces risk and because it minimises the momentum which can make it difficult to cancel work.

It also minimises the influence of Conway’s Law: “every organization will produce systems which are a copy of the communication paths in the organization.”

If you start with a big team you will get a big project - allocate a C# developer, a DBA and a UI designer and you’ll get a three-tier architecture. If you start small you should get a small project.

In the longer term how you staff a project is less a function of “How many people do I need to do this work?” and more a question of “How many people can I afford to do this work?”

Projects are never really staffed on a “how many people do I need” basis, we just pretend they are. Writing today, March 2011, you will probably put more people on a piece of work than you would 18 months ago, but fewer than you would have three years ago.

Rule 3: Staff work strategically relative to the other work to be done and corporate priorities, rebalance only occasionally

Strategic staffing means looking at how any one piece of work stacks up against the other pieces of work in play - or proposed - and the resources that are available to use. It means considering: what is the benefit of a piece of work? What is the risk? What work absolutely must be done, and what work can be left undone? What work is experimental?

The trick is to arrange work so teams can grown, or shrink, as resources and priorities change.

That is not a recipe for changing team composition every week. Indeed, the general principle should be: keep teams together and consistent for as long as possible. Only change teams occasionally, and only grow teams slowly.

However, things do change and work needs rebalancing. This should be an active, measured, and occasional process. Better to change staff allocations once a quarter in a review meeting than every week as a knee-jerk reaction.

That is what strategic staffing is. If you find yourself changing staff allocations every few weeks in response to events you aren’t working strategically.

As for deciding how many people to put on a piece of work when you are bidding for a contract, well, its another argument for constructing contracts as ongoing, rolling contracts.

Telling your client: “We think we need six people for six months” is little better than lying if you don’t have data to back it up. Have these six people worked together before? How productive will they be with the client’s environment? With the clients requirements? What work might emerge?

Better to tell the client: “We propose to start with three people for three months, at the end of that time we will review progress with you and decide with you, when we have data, whether to increase or decrease staffing. If you want to move faster than we can allocate four people and review after one month.”

It might not be what your client wants to hear but it sure beats guessing, or lying. Involve your customer, give them choices. It might not be a great sales technique but if the client won’t engage in this conversation there are probably other conversations they won’t have either.

Final roundup of facts from Capers Jones

In two previous entries I’ve reported some interesting statistics and findings - possibly facts - from Capers Jones book Applied Software Measurement (see Software Facts - well, numbers at least and More Facts and Figures from Capers Jones). I want to finish off with a few notes from the later chapters of the book.

On packaged software

• Modifications to existing COTS package - I assume he includes SAP and Oracle here - is high-risk with low productivity
• When changes exceed 25% it may be cheaper to write from scratch. (I’m not sure what it is 25% of, total size of the package, 25% of function points, 25% of features?)
• Packages over 100,000 function points (approximately 1m lines of code) usually have poor bug removal rates, below 90%

On management

• Jones supports my frequent comment that comes redundancies and downsizing Test and QA staff are among the first to be let go
• Projects organised using matrix management have a much higher probability of being cancelled or running out of control
• Up to 40%(even 50%) of effort is unrecorded in standard tracking systems

On defects/fault

• “Defect removal for internal software is almost universally inadequate”
• “Defect removal is often the most expensive single software activity”
• WIthout improving software quality it is not possible to make significant improve to productivity
• “Modifying well-structured code is more than twice as productive as modifying older unstructured software” - when he says this I assume he doesn’t mean “structured programming” but rather “well designed code”
• Code complexity is more likely to be because of poorly trained programmers rather than problem complexity
• Errors tend to group together, some modules will be very buggy, others relatively bug free
• Having specialist, dedicated, maintenance developers is more productive than giving general developers maintenance tasks. Interleaving new work and fixes slows things down
• Each round of testing generally finds 30-35% of bugs, design and code reviews often find over 85%
• Unit testing effectiveness is more difficult to measure than other forms of testing because developers perform this themselves before formal testing cuts in. From the studies available it is a less effective form of testing with only about 25% of defects found this way.
• As far as I can tell, the “unit testing” Jones has examined isn’t of the Test Driven Development type supported by continuous integration and automatic test running. Such a low figure doesn’t seem consistent with other studies (e.g. the Nagappan, Maximilien, Bhat and Williams study I discussed in August last year.)
• Formal design and code reviews are cheaper than testing.
• SOA will only work if quality is high (i.e. few bugs)
• Client-server applications have poor quality records, typically 20% more problems than mainframe applications

Documentation

• For a typical in-house development project paperwork will be 25-30% of the project cost, with about 30 word of English for every line of code
• Requirements are one of the chief sources of defects - thus measuring “quality” as conformance to requirements is illogical

Agile/CMM/ISO

• There is no evidence that companies adopting ISO 9000 in software development have improved quality
• Jones considers ISO 9000, 9001, 9002, 9003 and 9004 to be subjective and ambiguous
• Below 1,000 function points (approximately 10,000 lines of code) Agile methods are the most productive
• Above 10,000 function points the CMMI approach seems to be more productive
• I wold suggest that as time goes by Agile is learning to scale and pushing that 1,000 upwards

Jones also makes this comment: “large systems tend to be decomposed into components that match the organizational structures of the developing enterprise rather than components that match the needs of the software itself.”

In other words: Conway’s Law. (See also my own study on Conway’s Law.) Its a shame Jones missed this reference, given how well the book is referenced on the whole I’m surprised.

Elsewhere Jones is supportive of code reuse, he says successful companies can create software with as much as 85% reused code. This surprises me, generally I’m a skeptical of code reuse. I don’t disbelieve Jones, I’d like to know more about what these companies do. As has to be more about the organisational structure than just telling developers: “write reusable code”.

Overall the book is highly recommended although there are several things I would like to see improved for the next revision.

First, Jones does repeat himself frequently - sometimes exactly the same text. Removing some of the duplication would make for a shorter book.

Second, as noted above, Jones has no numbers on how automated unit testing, i.e. Test Driven Development and similar, stacks up against traditional unit testing and reviews. I’d like to see some numbers here. Although to be fair it depends on Jone’s clients asking him to examine TDD.

Finally, Jones is very very keen on function points as a measurement tool. I agree with him, lines of code is silly, the arguments for function points are convincing. But, I’m not convinced his definition of function points is the right one, primarily because it doesn’t account for algorithmic logic.

In my own context, Agile, I’d love to be able to measure function points. Jones rails against Agile teams for not counting function points. However, counting function points is expensive. Until it is cheap and fast Agile teams are unlikely to do it. Again, there is little Jones can do directly to fix this but I’d like him to examine the argument.

I want to finish my notes on Jones book with what I think is his key message:

“Although few managers realize it, reducing the quantity of defects during development is, in fact, the best way to shorten schedules and reduce costs.”