Path dependency

I’ve been wanting to write about Path Dependency for a while. Its a subject that fascinates me but I’m not quite sure I understand what is means and how it affects us.

The idea comes from economics and at its simplest path dependency says “History matters” - or, how we got here effects the way things are. Things are the way they are because of all the decisions that have been made up to this point. Sometimes this can explain why our theories don’t explain the real world - yes, this does touch on chaos theory too.

There is an interesting paper by Margolis and Liebowitz entitled “Look, I understand too little too late” that makes for a good starting point. One of the classic examples of path dependency is supposed to be the Qwerty keyboard. But the authors demonstrate that this isn’t really the case. They go on to question the whole idea of path dependency as an economic force.

So, maybe in economics the theory doesn’t apply. But I think it does apply to software.

All too often when you look at a piece of software it just doesn’t work as you might expect it too. It is programmed in C++ when Python would be a better language, or it is procedural based when OO would be better, maybe it just implements its own data structures when it should use a library. It seems every programmer when confronted with code he didn’t write says “this is awful... maybe we should rewrite?”

It doesn’t just happen at the code level, look at the whole product: are all the features still needed? Perhaps our product is difficult to use because we needed so many features two years ago but now they get in the way.

And the same thing goes at the process level: we write requirements documents because that’s the way we’ve always done it, we throw code over the wall to tester because that’s what we did at the last place. We are slaves to the past and ignore better ways of doing things.

I think a large part of this is path dependency at work. Not entirely, sometime it is because we now understand what we are trying to do better, and partly it is because we have more options available now, but largely it is because of how we got here.

Every individual decision made sense at the time. However, the cumulative set of decisions isn’t the optimal one now. Yet it is difficult to justify changing it now.

So often our theories, whether in Economics, Software Development or Business assume that should be just are: theory says that supply equals demand so it will, theory says code should be unit tested so it is, theory says that Business should concentrate on core competencies so they do. But, unlike Physics these things don’t happen instantaneously. We might need time to let the system adjust, or we might need to give it a helping hand or remove some blockages.

Path dependency may be a grandiose term but is makes an important point: often how we got here is a better explanation of the current situation than how it should be is.

I think much of what we call “design” is about breaking path dependency. Rather than accepting how things are, rather than doing things the way they always have been done, design is about finding a better way and encouraging people to do it that way instead.

That goes at the code level, the product level, the process level and at the business strategy level. We should respect the ways things are, they are that way for good reason (even if we don’t understand them all) but that doesn’t mean we can’t change and improve.

I’m still working out what path dependency means to me and my world view. I think its important and can explain a lot. As I work it out I’ll update you.