Scarcity, not Slack

Question: What is the opposite of Slack?

Or, perhaps a better question: What is the problem to which Slack is the answer?

Lots of software people like to advocating the benefits of slack for development teams. I’m not talking about the popular collaboration tool Slack, rather I’m talking about Slack in the sense discussed by Tom DeMarco in his book. DeMarco’s is a good book and has some good arguments but on the whole I’ve never been a fan of introducing random slack into the development process. (Have a look at my comments on Seb Rose’s blog post from a few years back to see what I mean.)

Now I’ve found a book, and two academics, who have done the hard work of creating an anti-dote:

Scarcity: Why having so little matters so much by Sendhil Mullainathan and Eidar Shafir (this is the UK edition, in the US the publishers have a different name slightly, Scarcity: The New Science of Having Less and How It Defines Our Lives)

Scarcity is the opposite of slack, and just maybe, scarcity is the problem which requires slack.

This book is highly recommended. Although it has nothing to do with the software community this book deserves to be essential reading for everyone who claims to know something about software management and in particular Agile.

The authors are ambitious if nothing else: they set out to create a new discipline, the study of scarcity. Although most of their experiments, examples and studies concern scarcity of money and/or time they endeavour to project their arguments into a more general space, scarcity of anything.

As you might expect, I read this as a software engineer. I read the book applying their thinking to the world of software and specifically Agile Software development. And after all, so much of managing software development comes down to the scarcity of time and the scarcity of money even if it manifests itself as a scarcity of people and excessive demands.

Now there is good news and bad news for software people.

First a little theory: the authors suggest that when people encounter scarcity they “tunnel”. In other words their mind starts devoting a lot of capacity to the resource which is in short supply. This has advantages: our focus increases, our productivity increases, some of judgement (yes estimating) skills improve and overall our effectiveness increases. Yes, scarcity can create productivity improvements. Scarcity is not all bad, it can be beneficial.

But there is a downside. For people in the scarcity tunnel their mind devotes a lot of time to thinking about this problem. As a result there are negative effects. Anything that doesn’t effect the short term tunnel is pushed to one side. So, for example, the short term benefits of taking a payday loan displace the long term problems which will be made worse.

This creates what the authors calls a “bandwidth tax”: so much of the mental energy is devoted to the tunnel and scarcity that less capacity is available for other things. For example, parents who have to juggle childcare arrangements for children are less effective at work.

The authors present serious evidence that the bandwidth tax reduces intellectual capacity, i.e. IQ, intelligence quotient. Perhaps controversial this leads them to suggest the poor are poor because they are poor, a vicious circle develops which is difficult to escape and traps people.

OK, enough of the book, read it yourself. What about software development? What does this book mean to software?

I see that tunnel a lot with development teams, they are doing stuff they know they shouldn’t but they feel they can’t afford to do the right thing. As a result they never get out of the mess they are in.

It is immediately obvious that one of the ways Agile Software Development - specifically iteration based Scrum, XP and my own Xanpan - work is by creating a short term scarcity of time (two weeks!) which causes tunnelling. Effectiveness goes up as items outside the tunnel are pushed out.

Work in progress limits (whether flow based or velocity/points based) are another form of induced scarcity. They force managers and product people specifically to tunnel on what is important, what needs going an they push less important items out.

But this comes at a cost.

One clear implication is that activities with benefits which will fall outside the tunnel will be dropped, skipped or minimise. And the obvious thing here is Technical Quality and improving the Technical Infrastructure. For example, adding automated tests, applying test driven development, improving the build system, refactoring, the list could go on.

(Indeed, it was worry about not investing in these which causes Seb Rose to advocate Slack.)

Mullainathan and Shafir give examples of poor farmers who save money by not taking out insurance against crop failure, drought or other possibilities. They also show how the same farmers would benefit and save money by taking out insurance.

They give example of vaccination programmes where those who will benefit - and know they will benefit - do not partake.

When they need the benefits of such insurance it is too late, they are tunnelling and cannot afford to invest. When they have the time or money to invest in insurance it seems unnecessary. People are optimistic and under estimate the risk to small problems occurring.

This is the same as teams who don’t do practices such as Test Driven Development. They are tunnelling.

There is more evidence here about the role of deadlines and routines: deadlines and routines help us manage our time and ensure things happen. The research presented in the book complement my own notes on time estimation (Notes on Estimation and Retrospective Estimation and More notes on Estimation Research). (It is because of such finding that I cling to fixed length iteration in Xanpan rather than going abandoning them like Kanban.)

And those vicious circles which keep poor people poor parallel software teams too. Poor people find problems which can be solved with a little money become very disruptive because even a little money is hard to find, they tunnel more, more problems are ignored. The current fire is extinguished but other problems are worse now.

I see the same thing with software teams. Projects run late and over budget so managers tunnel on the project, while this is happening other problems are building up, quality is neglected, technical debt is incurred, support tickets are left unanswered, changes which could help are put off until the fire is extinguished.

Sometimes I think the answer is simple: choose not to enter the scarcity trap, choose not to firefight, choose to do the right think. It reminds me of the talk JB Rainsberger gave on his 2015 European tour “Just Stop It.” It reminds me of Renton in Train Spotting: Choose Life. It reminds me of Joe Bergin’s (in)famous “Do the Right Thing.”

Perhaps easier said than done, personally I have one piece of advice for these companies: Stop Thinking Projects. Project thinking obscures the problems and solutions.

The authors have some advice, some solutions, but they don’t have a complete solution. I find myself in the same boat: I can see how their arguments play out in software and I have some solutions but I don’t have them all.

Importantly they do identify Slack as part of the answer to Scarcity. Anyone who has studied queuing theory knows this.

Not only does this book deserve to be widely read in the software community - and specifically the Agile community - but I hope to see these two authors appearing at software conferences before long.

Finally, I consider myself lucky to have found this book. After several years where I struggled to find books which said something new I’ve been lucky enough to read three in the last year. (I blogged about Scaling Excellence last year (another must read), I didn’t blog about Joy by Richard Sheridan but his book deserves reading just for the share life affirming enjoyment, a reminder that software development can be and should be joyous! OK, I enjoyed it for another reason, Richard and I both had our lives changed by the same events on 2 January 2001, but that is a long story!)