SPA London, Tom Gilb and the doom of Agile

I’m planning on going to SPA London session next week, Tom Gilb is giving a talk entitled “Does real Software Practice Advancement need yet another 'Manifesto'? ” He’s being quite provocative, look at this from the synopsis:

"AGILE HAS DOOMED ITSELF - TO BECOME YET ANOTHER FAD: XP IS ALREADY DEAD. What is Seriously Wrong with Agile practices and interpretations - why AGILE, AS CURRENTLY PRACTICED, is PROJECT-failure-prone as a culture."

I think Tom is both right and wrong. He is right that projects need more of a value driven approach, he is right that Agile is a fad but he is wrong to say XP is dead or Agile is doomed. Personally I’ve been expecting an “Agile backlash” for at least two years now, and to some degree I think it deserves it.

I too have concerns about “Agile” both the marketing of it and what lies below. The term is very vague, covers a multitude of things and is often abused. Its fair to expect it to come down a peg. But, now I look too faced, and you may be asking “Why does Allan talk a lot about Agile?” the answer is simple. Its the best tag we have at the moment, its the best term for grouping all these ideas together.

Lets take the synopsis one part at a time.

“Agile has doomed itself”: perhaps, the term is poorly defined. As a result a lot of bad practice hides behind the Agile label and gives Agile a bad name. Still there is a lot of good in there.

“[Agile] is another Fad”: certainly there is a lot of hype, much undeserved and certainly this will blow over. As bits of Agile become mainstream some of that will go naturally. But some hype is necessary, its called “marketing” and its all around us. The Agile bubble will deflate, it might get pricked but thats not a reason to attack it alone.

“XP is already dead”: I’m sorry to see Tom adopting this view. XP’ers were the storm troopers of the Agile movement. The very word “Extreme” meant it would repulse some people. But XP got people listening, got people excited and now XP takes a back seat.

Look at what XP has left us with: Test Driven Development, Continuous Integration, Refactoring, simplicity in design - the technical practices which underpin Agile development.

XP’s project management side was always a bit weak but it isn’t that different to Scrum. Scrum finesses it a bit, makes it more acceptable to management, adds in some bits that were missing and tightens it up but the two are the same. If XP’s project management is deeply flawed then so too is Scrum.

So XP is still with us, it just split in two. If XP is dead then it died so Agile might live.

Hopefully the talk will explain the rest of the synopsis.

Now why Agile is a success and is here to stay. There are three parts to this, one especially for Tom.

First, as I’ve argued again and again, Agile software development is a branch of Lean thinking, in particular Lean product development. Lean has been around for a long time and for about 20 years has been documented, studied and been something of a “fad” itself. As such I don’t think Agile is going away, but I think we will see more use of the word Lean.

Second is the name factor. “Agile” is a solution, it is a solution to slow software projects, it is a solution to poorly run software projects, it is a solution to poor quality software, it improves team performance and customer satisfaction. Yes “Agile” is a vague term but it is a label for all those things.

Go searching on Google for any of those things. You’ll find what you want, eventually. You will need to dig through lots of stuff first. Yes “Agile” is something of a marketing term but it is a marketing term that labels a whole set of desirable outcomes and ways of achieving them. As such it is short hand for finding ways to make software work better.

Finally, there are lots of different pieces make up the “Agile” toolkit. Scrum brings Project Management. XP brought better quality software - a crown inherited now by the Software Craftsmanship movement. Crystal (remember that one?) and Lean bring continual improvement. Elsewhere I’ve written about Agile governance, and so the list goes.

Tom and Tom’s Evo method brings important tools to that toolkit. Evo pinoneered short iterations. Evo emphasizes value delivery. Which leads us to better requirements understanding and governance. Evo emphasizes measurement. Planguage gives us another way of documenting requirements. And so the list goes on. If you are not familiar with Toms work then its well worth a look, lots of good ideas.

But, Tom’s ideas aren’t the whole toolkit. XP, Scrum, Lean, Kanban, etc. all bring something to the party, its up to you to decide which guests you want.

On the one hand I find it ironic that Tom appears to be attacking Agile. For me Evo is an Agile method. Evo shares common roots with Agile, namely the work of Edwards Deming. On the other hand, I enjoy Tom’s ideas, he’s thought provoking and I always learn something. Maybe he’s just getting some of the Agile hype for himself.

Events: Business Analysts & UK Lean conference

As the summer comes to an end conferences start again.

At the end of September I’m going to be at the UK Lean Conference (27-29 September) on a panel discussing Kanban. A couple of days later I’m going to be across town at the IRM Business Analysis conference (28-30 September).

Monday the 28 is going to be a tough call. Should I go to the BA pre-conference session? Or the second day of the Lean conference?

If your going to be at either conference please say Hello.

Agile failure modes from othe

I have been preparing some training material for a company which wants to know more about how Agile projects fail. Fortunately I’ve made a habit of recording Agile failure modes in this blog so I have much of the material to hand. (Use the search box above and search on “Agile failure modes.”)

But, I’ve not got exclusive rights to this topic and others have done the same thing, often with catchier names. So, for completeness:

• Wagile from Jason Gorman
• ScrummerFall from Mitch Lacey
• ScrumBut from Eric Gunnerson
• WaterScrum from Scrum Community

As far as I can tell these are the original sources of these terms but I may be wrong. Others have also used the term and some have Wikipedia entries so I could be wrong.

The Product Owner role

The Agile Journal this month has an article from me this month entitled Dissecting the Product Owner role.

This complements my pieces on the Business Analyst and Product Manager roles which appeared in ACCU Overload earlier this year.

CMM & Agile

I’ve finally, about 6 months after everyone else, got around to reading the Software Engineering Institutes report on CMMI and Agile. Entitled “CMMI or Agile: Why Not Embrace Both!” the report argues that CMMI and Agile are not only compatible but complement one another. Actually, you can guess at what the report says from the fact that the title ends in a exclamation mark (“!”) rather than the question mark (“?”) you might expect.

The report argues that the two are compatible and complementary. They make a strong case and I for one am glad. CMM initiatives have too often made the words “process improvement” dirty words. They shouldn’t be, it is what we should be doing.

The report argues that CMMI, and its predecessor CMM, have been misunderstood and misused by the industry. Firstly CMM(I) was taken to be a methodology when it was a model. CMM(I) doesn’t tell you what to do, it allows you to evaluate your capabilities.

The second mistake compounds the first by imposing CMM(I) on organizations. Rather that improving an existing process and organization CMM(I) initiatives have replaced what already exists with something else - and in the process destroyed the good that existed. This is the scenario I wrote about a few weeks ago in “A true story about CMM”.

There are a few points were I could take issue with the report.

My first issue is that the report does not define what it means by ”Agile.” I agree this is a problem, and one I’ve been thinking about addressing in this blog. But, CMM and CMMI are very well defined while Agile is not. How can one critique a method without defining what is mean by the term?

Second, the report seems determined to identify an “Agile Institute” to mirror the Software Engineering Institute. It seems to spend a lot of time talking about the authors of the Agile manifesto and founders of the Agile Project Leadership Network. Neither of these groups owns the term “Agile” or speaks for the whole Agile community. There is much under the Agile umbrella which, rightly or wrongly, falls outside these two groups.

Comparing Agile and CMMI is an asymmetrical comparison. One is clearly defined, with a well funded champion to issue standards, revisions and issue certificates. The other is the ill-defined product of an ill-defined community which owes as much to the punk ideal as anything else.

A bigger issue with the report concerns why you might want to bring CMM(I) and Agile together. The report spends several pages early on explaining that the CMM grew out of one type of environment with one set of problems. Namely large projects, often military in nature, often with life at risk, and with a low-trust relationship between supplier and client.

Conversely, Agile came from a different environment: mainly corporate IT groups, small projects (no more than a dozen or two people), with high-trust within the team. Failure to understand these origins and contexts has led to much misunderstanding between advocates of the two.

Given that these two environments still exist, and that Agile has demonstrated success in its native environment and CMM(I) has demonstrated success in its native environment, then why would people in either environment want to adopt elements of the other?

I don’t think this question is really answered. This is a shame because the authors have spent a lot of time setting up a story to explain why we got where we are today but don’t explain how that story ends.

To my mind the story only tells part of the truth. Both CMM(I) and Agile have been sold into many organizations outside their native environments. There are many people who want the respectability of CMMI, and there are others who want the fashion of Agile. Nor am I aware of anyone in the Agile or CMMI camps saying “Don’t do that sort of project with our model.”

I’ll defend Agile here because I believe Agile has, and is, a journey of exploration. The underlying principles are universally applicable but that doesn’t mean all the details are worked out in any given field yet. So if you ask me “Can I do Agile on an X type project?” my answer is “Yes but I may don’t know how immediately.”

However the SEI folks who wrote this report claim CMM(I) has been misapplied. So I ask: which of them was saying “Don’t use CMMI on an X type project” ?

My final niggle is one of clarity. The report has five authors. Several of those authors are quoted and referenced throughout the text to support the arguments made. Yet nowhere in the report, when one of the authors is quoted is there a disclaimer. That is, the report does not say “Smith, one the authors of this report showed the CMMI is effective (Smith 2003).”

The names of the authors aren’t hidden, there is no subterfuge but its not clear and I wish they had been.

On the more positive side there are several quotes in the report worth calling out. About Agile and CMMI:

• “In today’s increasingly dynamic world, CMMI-based organizational process improvement approaches cannot rely exclusively on traditional project management approaches, waterfall-based project lifecycles, and heavyweight analysis methods.”
• “CMMI and Agile can complement each other by creating synergies that benefit the organization using them. Agile methods provide software development how-to’s that are missing from CMMI best practices that work well”

And about CMM(I) and the approach companies have taken in the past:

• “Too often, CMMI has been applied rather than implemented”
• “SCAMPI appraisals are not audits or tests”
• “the complexities of large-scale projects require a more involved infrastructure, this fact is not a license to create unnecessary bureaucracies and unbalanced production at the expense of productivity”
• “Using process experts to create and deploy process improvement activities is not unusual, and often has the advantage of allowing these experts (and the project teams) to focus on their respective tasks. But what makes this approach risky is that project personnel are frequently left out of process design activities and are disinclined or openly skeptical toward the adoption of process improvement activities. This situation is typical of some Six Sigma style approaches to process improvement as well. Quality process teams are created and given the responsibility for quality. As a result, developers abdicate their responsibility for improvement (or worse, for quality).”

Interestingly I also noticed that the SEI’s own Team Software Process (TSP) is now listed as an Agile method. I’ll have to look into this some more.

I am also please that the report highlights the role of organizational learning in software development. Regular readers know this is something I’m very keen on (heck, I wrote a book on the subject!) so I’m pleased to see it explicitly considered.

Even with my reservations I welcome the report. I believe it is a roundabout endorsement of Agile (in its various forms) by the Software Engineering Institute. It is also a very big sign saying “Agile and CMM(I) are compatible.”

A favour to ask of my readers

Dear readers,
There are getting to be quite a few of you. I know who some of you are but not all of you. And I know a few of you have bought, and even read my book, Changing Software Development: Learning to Be Agile.

I have a little request to ask of you: would any of you care to write a review of the book on Amazon?

Yes, I’m looking to improve my sales and the more (positive) reviews the better.

Thank you very much.

allan
ps Little bit of trivia for you. Last year I discovered the book I thought was called “Changing Software Development: Learning to Be Agile” was actually called “Changing Software Development: Learning to Become Agile.”

Well, a few weeks ago I discovered the book has both titles. If you have a copy look at the cover, the title if “Learning to become Agile”, not turn inside and look at the title on the inside page, here it is “Learning to Be Agile” - the way I wanted.