Dear boy, have you ever tried programming?

I sometimes come across teams who are wrapped up in discussing what they should be building. OK, I suppose this is because I hang out with too many management types. Sometimes there are only a couple of programmer, occasionally one, or maybe they haven’t got that far.

Sometimes I come across pre-teams, people who will form a team to do some work but haven’t started yet. I guess the project folk would call this “pre-project phase” and there is a lot of discussion: “Should we do Agile or Waterfall? Scrum or Kanban?” and “Which outsourcer will we use?” and “If we are agile how will we….”

In short these people are wrapped up in agonising about what might happen.

On these occasions I’m reminded of a story I once heard about the actors Laurence Olivier and Dustin Hoffman. In the story Hoffman, who is a famous method actor, says to Olivier something like:

“Mr Olivier, I am honoured to be working with you, but please tell me how do you learn you character? I mean do you use the method or some other approach? What should I do?”

To which Olivier says:

“My dear boy, … have you tried acting?”

This is what I feel, but usually don’t, to these teams, managers, analysts, whoever:

“Have you actually tried developing and delivering some software?”

Some planning, some forethought, some agonising is good. It helps you think things through. But it quickly becomes a game of diminishing returns. Its the old Eisenhower quote again:

“Plans are worthless, but planning is everything.”

Which is a natural follow on from Helmuth von Moltke’s observation:

“No plan survives contact with the enemy.”

Peter Drucker has a similar, though to my mind better, quote too:

“Plans are only good intentions unless they immediately degenerate into hard work.”

So let me say:

“No plan for software development survives contact with the code. (Double so for legacy code.)”

“No requirements roadmap ever survives contact with the market.”

(According to wikipedia the Olivier-Hoffman exchange happened during the filming of Marathon Man and isn’t exactly true but does contain a grain of truth.)

Book of the Year

I’ve been asked by several people lately - mostly for their own blog posts! - for my book of the year. Well thats easy…

From a professional point of view, that is, if you are asking me as someone who advises software developers and teams, and is well know as an “agile supporter” then my unreserved book of the year is… drum roll…

Scarcity: The True Cost of Not Having Enough by Mullainathan and Shafir

I wrote longer review of the book back in June, “Scarcity, not Slack.”

#NoProjects: videos + Q&A

One of the reasons this blog has been quiet recently is that I’ve been putting my efforts into video: The #NoProjects Video series.

This is now available as 16 short episodes on YouTube based on my popular #NoProjects/Beyond Projects conference presentation. If you have seen the conference presentation you know what to expect although there are some changes. My hope is that by making the presentation available as 16 short (I think the longest episode is 5 minutes) pieces the argument will reach a wider audience.

I have also been directing my efforts at another Beyond Projects endeavour: the Managing without Projects one day workshop which I will be holding in London (Bloomsbury) next month (Friday 15 Jan). I’ve got a few people signed up for this and I know a few more bookings are in the works but the more the merrier so please come too!

Blog readers can get a 10% discount on this workshop with the code BlogDec15 which is good for the next week (until 14 December.)

There are a few questions I’ve been meaning to clarify about #NoProjects / Managing Without Projects and now is as good a time as ever:

Q: Isn’t “No” rather negative?

A: Yes it is, I’ve had a go at changing it but attempts so far haven’t caught the same feel. Perhaps this is because “No” is so definite or because “Beyond” is 4 characters more and 4 characters on Twitter is a big deal.

For a while I preferred the name “Beyond Projects”

Lately I’ve been calling it “Managing without Projects”

But perhaps “Software without Projects” might be better still, that would lend itself to the hashtag #SW2Prjs

Q: Is #NoProjects actually #NoEstimates?

A: No

The hashtags and ideas originate with different people. There are parallels and the ideas might be symbiotic but while I know some #NoEstimates people would like to see #NoProjects as part of the same idea I don’t.

Estimates are a complex area and right now its difficult to have a rational conversation about them - perhaps it was ever so. But, I still see value in “estimates” in some context. Its all a question of “What question are you really asking?” using estimates might help or it might now.

While most of the conversation on “estimates” implies “time estimates” there is another form of estimation: “value estimates”. These don’t get anywhere near enough attention or use. I use them and I help my clients use them but I don’t see them elsewhere very often.

I should expand on this another time.

Q: Does #NoProjects mean #NoManagement, aka “Self managing teams” ?

A: No - I have no argument with management itself. I have an argument with the concept of projects which I think leads to poor management of software development.

I see management as a skills, the same as Java, SQL, Testing or Analysts. Unfortunately very few people actively learn about management or try to master it as a skill in the same way they might Python or Java.

When they do try and learn “management skills” people often start with “project management.” I sometimes thing of “project management” as “management lite.”

As a result an awful lot of “management” of technology (specifically software development) is poor. In such conditions removing management (i.e. self management) can actually improve things.

Again, I should really write an extended piece on this subject.

Q: What about Project Managers?

A: I treat each and every Project Manager as an individual. They can bring valuable skills and experience to a team. Unfortunately many of them don’t and add little. In part this is because they are asked - both by their employer and their professional bodies - to manage using a model that is inappropriate to the software domain.

So I don’t see mass redundancies of project managers, rather I see the good ones being allowed to manage properly using a better model.

Q: Isn’t this a pretty radical position when so many companies revolve around the project model?

A: Yes, my hope is to get people to talk about the issues I raise. Even if companies don’t abandon the project model completely I hope they can get better by finding their own ways to address these issues.

Besides, I’m a consultant and I’m only too happy to talk about these issues myself and give advice. Just call me.

As a consultant few of my clients actively set out to become a NoProjects company and I don’t try and make them so. They and I want them to do better. I find that if I think in a NoProjects way I can understand the client better, and if they still have projects I can then map the ideas across.

This blog & Whats next? after Agile?

This blog has been noticeably quieter than usual over the last couple of months, and whats more, when I have posted something up its often been of the “this is what I’m doing” type rather than the “here’s something to think about” type post. Well…

In the last few months I’ve been putting my energy into other mediums, hence A little book of requirements and user stories and the managing beyond projects workshop I’ve mentioned before - the dates for the workshop has moved back to January by the way.

The big news this week is that I am serialising my #NoProjects presentation as mini-videos on YouTube. Various conference videos are already online but this is a series of short videos, check out the Managing without Projects playlist. (There are 7 episodes so far, more coming soon.)

I also added a new newsletter earlier this month and I expect this will in future carry material which would have been in this blog otherwise. So if you want the latest, freshest stuff I suggest you subscribe to the newsletter.

And here is the piece that the newsletter carried at the start of this month….

What's next? - after Agile?

As this is the first of my new, occasional, newsletter it seems a great chance to look to the future. So lets ask that question that comes up regularly:

"What comes after Agile?"

Sometimes only asked rhetorically by agilistas to prove they have the inside track on what is really happening!

I had a go at answering it myself a few years ago with a presentation entitled "The future of Agile". To cut a-long-story-short, I thought the next thing would be Lean. Six and a bit years on I think I was right but perhaps not in the way I thought it might be. I saw Lean displacing Agile as the predominant approach to software development, or at least the buzzword. Well it hasn't, Agile is still the buzzword it was.

But Lean has permeated more and more thinking on Agile. If you look at Agile almost every idea can be traced - in some cases literally in other case philosophically to Lean thinking. Anyone who goes a bit deeper into Agile quickly runs across the Lean roots. Over time we've seen more and more Lean ideas adopted under the Agile umbrella.

My own Xanpan book is an example of that: hard core XP agile infused with Lean Kanban thinking.

Talking of Kanban.... six years Kanban was still new, and it seemed for a while that the Kanban community didn't want to be part of the Agile community. They wanted to be seen as Lean. (One supposes they thought Lean was superior, or at least more marketable.) That seems to have changed too, I now hear expressions like "Kanban is an alternative path to Agile." I feel there is a broader understanding that while Kanban is different to Scrum it is not so different that it isn't part of the Agile movement.

(Perhaps the greatest contribution the Kanban insurrection made has been breaking the Scrum hegemony. Scrum is still often used as a common synonym for Agile but it is no longer seen as the only player. Because of Kanban people are more aware of differences.)

Back to my original question.

When people ask "what is next?"I usually to sense there is something else they are not saying, perhaps what they are saying is: "Agile destroyed the waterfall, what is coming along to destroy Agile?" - after all this is the tech industry were new technologies come along regularly. Cynically I sometimes wonder if these people are thinking "Maybe I can ignore Agile and get with the next thing" or even "Agile is a fad, give it a few months and the wheel will turn, we'll be back to waterfall."

Notwithstanding cynicism, it is wrong to think something will come along and disrupt or destroy Agile, the way Agile did Waterfall. Many more innovations build on what has gone before than destroy what has gone before. The question "What comes after Agile?" is better asked "How will Agile evolve to next?"

That is an easier question to answer because the future is here: *Continuous Delivery, Mob Programming and No Projects*. All three build on Agile and takes it to a higher level

Continuous delivery: Teams delivering not just at the end of the iteration but all the way through it. In some cases many times a day or even many times an hour.

Unfortunately for those people who were hoping to leapfrog Agile and get with the next big thing the first step to doing Continuous Delivery is doing Agile well. If your team can't release a product update at least every two weeks then you aren't even at the start of Continuous Delivery.

The path to the continuous delivery thing lies through Agile.

Then there is Agile's cutting edge, the new ideas which will build on Agile and take it further.

On the technical side mob programming and BDD (behaviour driven development) continue to build.

On the process and organization side "#NoEstimates" continues to stir up controversy although I fear it does as much damage as it good.

And of course there is #NoProjects which I am closely associated with.

My thinking on #NoProjects continues to develop and it continues to be a popular conference talk. Now I want to try something different. Next month I intend to run a one day workshop on the subject to help teams explore how they may get away from project thinking. More details and how to book on EventBrite.

A Little Book about Requirements and User Stories

As mentioned a few weeks ago I’m compiling my recent Agile Connection series on User Stories and Requirements into a LeanPub book.

I’ve now released the first couple of chapters, you can get them from LeanPub right now!

I intend to add more chapters over the coming weeks. The bulk of the material will be straight from the Agile Connection series, perhaps with a little more editing. There will also be some off cuts, bits that didn’t make it to Agile Connection series either because they were blew the word limit on another piece, were too short to stand lone, or didn’t quite fit in. There is stuff you can do in a book because it is large and different.

My intention is to up the price a bit as I add chapters. Its really annoying now that the UK (and I think all of Europe) charge VAT on e-books. Seems a bit unfair that printed book don’t attract VAT but ebooks do.

If you have any comments on the book, the content or things you think are missing please let me know.

Managing beyond projects - a workshop #NoProjects

Regular readers will know I’ve been talking about #NoProjects for a couple of years now, why projects don’t make sense has been a reoccurring theme of mine.

Unfortunately, in the space of these blog posts, and in the time available for conference presentations there is not enough time to go into alternatives and how work should be managed. I feel bad about that because there are solutions.

So now I’ve put together a day long workshop to discuss the problems and solutions in more detail.

I’m running this for the first time in January, 15 January to be exact.

More details on the EventBrite booking page - Managing Beyond Projects.

For my readers I’ve set up a 10% discount code, BlogReader, this is good 12 November.

Numbers are limited, I intend to limit it to about 12 people tops. Tickets are in four price bands of tickets and there is a limited number in each band so if you want the cheapest tickets book early.

(This is the first time I’ve arranged an event with paid for tickets myself so its something of an experiment. I’m doing it to maximise feedback. So if you have any feedback, mail me direct!)

Software has diseconomies of scale - not economies of scale

“Practical men, who believe themselves to be quite exempt from any intellectual influence, are usually the slaves of some defunct economist.” John Maynard Keynes

Most of you are not only familiar with the idea of economies of scale but you expect economies of scale. Much of our market economy operates on the assumption that when you buy/spend more you get more per unit of spending.

At some stage in our education - even if you never studied economies or operational research - you have assimilated the idea that if Henry Ford builds 1,000,000 identical, black, cars and sells 1 million cars, than each car will cost less than if Henry Ford manufactures one car, sells one car, builds another very similar car, sells that car and thus continues. The net result is that Henry Ford produces cars more cheaply and sells more cars more cheaply so buyers benefit.

(Indeed the idea and history of mass production and economies of scale are intertwined. Today I’m not discussing mass production, I’m talking Economies of Scale.)

You expect that if you go to your local supermarket to buy milk then buying one, large - carton of milk - say 4 pints in one go, will be cheaper than buying 4 cartons of milk each holding one pint of milk.

In my #NoProjects talk I use this slide, it always gets a laugh:

Yesterday I put this theory to a test in my local Sainsbury’s, here is the proof:

• 1 pint of milk costs 49p (marginal cost of one more pint 49p)
• 2 pints of milk cost 85p, or 42.5p per pint (marginal cost of one more pint 36p)
• 4 pints of milk cost £1, or 25p per pint (marginal cost of one more pint 7.5p)

(And if you don’t know, the UK is a proudly bi-measurement country. Countries like Canada, The Netherlands and Switzerland teach their people to speak two languages. In the UK we teach our people to use two systems of measurement!)

So ingrained is this idea that when it supermarkets don’t charge less for buying more, complaints are made (see The Guardian from a few months back.)

Buying milk from Sainsbury’s isn’t just about the milk: Sainsbury’s needs the store there, the store needs staffing, it needs products to sell, and they need to get me into the store. That costs the same for one pint as for four. Thats why the marginal costs fall.

Economies of scale are often cited as the reason for corporate mergers: to extract concessions from suppliers, to manufacture more items for lower overall costs. Purchasing departments expect economies of scale.

But…. and this is a big BUT…. get ready….

Software development does not have economies of scale.

In all sorts of ways software development has diseconomies of scale.

If software was sold by the pint then a four pint carton of software would not just cost four times the price of a one pint carton it would cost far far more.

The diseconomies are all around us:

• Small teams frequently outperform large team, five people working as a tight team will be far more productive per person than a team of 50, or even 15. (The Quattro Pro development team in the early 1990s is probably the best documented example of this.)
• The more lines of code a piece of software has the more difficult it is to add an enhancement or fix a bug. Putting a fix into a system with 1 million lines can easily be more than 10 times harder than fixing a system with 100,000 lines.
• Projects which set out to be BIG have far higher costs and lower productivity (per unit of deliverable) than small systems. (Capers Jones' 2008 book contains some tables of productivity per function point which illustrate this. It is worth noting that the biggest systems are usually military and they have an atrocious productivity rate - an F35 or A400 anyone?)
• Waiting longer - and probably writing more code - before you ask for feedback or user validation causes more problems than asking for it sooner when the product is smaller.

The examples could go on.

But the other thing is: working in the large increases risk.

Suppose 100ml of milk is off. If the 100ml is in one small carton then you have lost 1 pint of milk. If the 100ml is in a 4 pint carton you have lost 4 pints.

Suppose your developers write one bug a year which will slip through test and crash the users' machine. Suppose you know this, so in an effort to catch the bug you do more testing. In order to keep costs low on testing you need to test more software, so you do a bigger release with more changes - economies of scale thinking. That actually makes the testing harder but… Suppose you do one release a year. That release blue screens the machine. The user now sees every release you do crashes his machine. 100% of your releases screw up.

If instead you release weekly, one release a year still crashes the machine but the user sees 51 releases a year which don’t. Less than 2% of your releases screw up.

Yes I’m talking about batch size. Software development works best in small batch sizes. (Don Reinertsen has some figures on batch size The Principles of Product Development Flow which also support the diseconomies of scale argument.)

Ok, there are a few places where software development does exhibit economies of scale but on most occasions diseconomies of scale are the norm.

This happens because each time you add to software software work the marginal cost per unit increases:

• Add a fourth team member to a team of three and the communication paths increase from 3 to 6.
• Add one feature to a release and you have one feature to test, add two features and you have 3 tests to run: two features to test plus the interaction between the two.

In part this is because human minds can only hold so much complexity. As the complexity increases (more changes, more code) our cognitive load increases, we slow down, we make mistakes, we take longer.

(Economies of scope and specialisation are also closely related to economies of scale and again on the whole, software development has diseconomies of scope (be more specific) and diseconomies of specialisation (generalists are usually preferable to specialists).)

However be careful: once the software is developed then economies of scale are rampant. The world switches. Software which has been built probably exhibits more economies of scale than any other product known to man. (In economic terms the marginal cost of producing the first instance are extremely high but the marginal costs of producing an identical copy (production) is so close to zero as to be zero, Ctrl-C Ctrl-V.)

What does this all mean?

Firstly you need to rewire your brain, almost everyone in the advanced world has been brought up with economies of scale since school. You need to start thinking diseconomies of scale.

Second, whenever faced with a problem where you feel the urge to go bigger run in the opposite direction, go smaller.

Third, take each and every opportunity to go small.

Four, get good at working in the small, optimise your processes, tools, approaches to do lots of small things rather than a few big things.

Fifth, and this is the killer: know that most people don’t get this at all. In fact it's worse…

In any existing organization, particularly a large corporation, the majority of people who make decisions are out and out economies of scale people. They expect that going big is cheaper than going small and they force this view on others - especially software technology people. (Hence Large companies trying to be Agile remind me of middle aged men buying sports cars.)

Many of these people got to where they are today because of economies of scale, many of these companies exist because of economies of scale; if they are good at economies of scale they are good at doing what they do.

But in the world of software development this mindset is a recipe for failure and under performance. The conflict between economies of scale thinking and diseconomies of scale working will create tension and conflict.

Have I convinced you?

Get small.

Finally, I increasingly wonder where else diseconomies of scale rule? They can’t be unique to software development. In my more fanciful moments I wonder if diseconomies of scale are the norm in all knowledge work.

Even if they aren’t, as more and more work comes to resemble software development - because of the central role of individual knowledge and the use of software tools - then I would expect to see more and more example of diseconomies of scale.

(PS, if you didn’t see my last post I’ve started a newsletter list, please subscribe.)

Big announcement: a newsletter

Announcing, drum roll…. my newsletter!

After years of putting it off I’ve decided to create a newsletter to keep people informed about what I, and my company Software Strategy, are doing - course we are running, presentations we are giving, blog posts I have made, journal articles and, lets see, rather than work out all the details I thought: lets just do it and see what happens.

Do I expect it to become a replacement for this blog? No

Do I expect it to have original content? Yes

Yes, I expect it might preview some material before it appears in the blog.

Yes it will contains links to new blog post, presentations and such.

I’m not sure how often I’m going to publish in the newsletter, every month seems reasonable but since I’ve never done it before who know! Anyway, I promise not to make them too frequent.

As you are already expecting the sign-up is on MailChimp so get over to the Allan Kelly Newsletter list page and sign-up now!

Large companies and fast cars

A few weeks ago I tweeted:

“Large companies trying to be Agile remind me of middle aged men buying sports cars”

I wasn’t saying large companies couldn’t be Agile - heaven knows most are trying and a few have successful software teams but on the whole the successes are few and far between. My thinking has nothing to with whether they can succeed (they clearly can, but most fail), or whether we should try and help them (yes, I’ve helped a few in my time, and I’ve seen a few failures) but rather my thinking comes from Peter Drucker:

“Large organizations cannot be versatile. A large organization is effective through its mass rather than through its Agility.” Peter Drucker, Age of Discontinuity

Peter Drucker wrote that in 1968, long before anyone ever thought of applying the word “agile” to software or business.

What I’m getting at is: for most large businesses the things Agile requires go against the grain of what has traditionally made them successful. For example…

Big businesses use economies of scale to extract favourable terms from suppliers. But software development doesn’t have economies of scale, rather it has diseconomies of scale. Applying economies of scale thinking to software development and agile hinders it.

Big businesses (to achieve economies of scale and management) often favour standardised procedures and processes: these bring some benefits but they reduce variation (not good in software development), reduce experimentation (required if you are going to try something new) and discourage risk taking. Sure standardisation brings benefits but the benefits they bring are opposite of what agile thinking would suggest.

Big businesses have lots of customers who are customers because they are customers - inertia. For example changing your bank is so much trouble and effort very few people do it past the age of 25. All that Agile talk about MVP, product management, customers and so on doesn’t matter when you are dealing with existing customers who stay through inertia. You biggest risk is upsetting them enough to make them move.

Big businesses are inherently risk averse: the risk of upsetting customers, suppliers and shareholders is greater when you have more of them. Yet many agile practices look very risky at first, even if you understand logically that they are risk reducing simply doing something your peers don’t is a risk.

For example, image a manager who buys into the logic of continual delivery (CD) and sees how it reduces the risk of releasing software. If he alone adopts CD for his team - when all the other managers stay with the annual release - and something goes wrong then he will be seen as at fault. It leads to defensive decisions and an aversion to change because change is risk

And big companies have more people so there can be a herd mentality. Consider that example again, it there are only two development managers in the company and one decides to adopt CD then half the managers do CD and half don’t. If however he has 10 peers then there are 10 who take a different decision, i.e. stick with the thing that always “worked” in the past and one, him, who changes.

Big businesses often grow through acquisition. Acquisition is often the modus operandi of big companies: buy similar companies, buy their customers, extract economies of scale, install standardised procedures and remove differences. Thats how you make a merger work if efficiencies are your goal. But those things fly in the face of what an Agile company would do.

Some big businesses have got to be big by squeezing the fat out of processes and inventories, i.e. remove the slack. But without the slack there is no room to manoeuvre - remember queuing theory, once utilisation rises delays increase and variation is bad. That makes change more difficult and also makes practices like reducing WIP difficult to implement.

One way big companies succeed is to define roles and responsibilities, indeed hand-in-hand with economies of scale goes economies of specialisation. That is, as organizations (even teams) get bigger it becomes economic to have specialists, like business analysts, in particular roles. But again this cuts against what an agile company would do. Agile companies value multi-skilled/multi-functional people who can adapt to what is required. This approach also helps deal with bottlenecks; specialisation often creates bottlenecks (even met a test manager who carefully schedules his testers so nobody is ever underused?)

I could go on but hopefully you see what I’m saying here: the things that make a company successful at being big are often the same things that make it difficult for the company to be agile and, importantly, vice versa.

Size is inherently un-agile.

Thats not to say size is bad, size brings many advantages but these are not the advantages of agile. Trying to be big and agile is an example of having your cake and eating it. The few companies manage to achieve both really are the exception that proves the rule.

Hence: a company gets big and successful, like a middle aged man who has a good job, and in an effort to reclaim the nimbleness of youth embarks on a quest for a fast car and younger wife (agile), while forgetting that middle age brings its own benefits, notably in this context knowledge and experience.

Now, the question that lurks here is: if a large company becomes “agile”, that is, if it changes many of the things that have made it successful as a large company in order to achieve agility, then: does make sense to remain a large company?

Possibly by dismantling the things that make big successful and replacing them with practices and processes which allow agile to be successful the company has turned itself into little more than a disjoint conglomerate. In which case it may find better economies, and a better stock market valuation, by splitting itself up.

And that in turn suggests that any framework for “scaling agile” has a number of challenges to overcome if it is to succeed.

Finally, I don’t mean to be critical of people who try to help large companies, in many ways these people are taking on the most difficult tasks and deserve our support.

And for completeness, I should say: if you are a big company out there wrestling with these problems please feel free to call me, my rates are very reasonable!

Little Book of User Stories anyone?

If you follow me on Twitter you will probably have noticed I’ve been writing a series of pieces for The Agile Connection on User Stories. This series began by accident, until recently I didn’t know I knew so much about user stories!

Actually, although the series is based around user stories it is probably better to say its about requirements in an agile setting. (There is a full list of the pieces so far on my website.)

I can now see the end of this series, a few more pieces to go and I’m thinking of rolling all the pieces, plus some offcuts and one or two extras into a LeanPub book. This is tentatively entitled “The Little Book of User Stories.”

Its on LeanPub now and if your interested please register your interest. And tell me how much you think I should charge.

If it looks like more than a few people are interested I’ll put the work in an publish it. If not, well, MVPs should be able to fail.