Thoughts on strategy and software development

My last two blog entries, and some recent reading (I’ll write about it soon) have given me a lot of food for thought about business strategy and how it relates to software development. Fundamentally strategy is about knowing what you are doing, articulating what you are trying to do and using this understanding to drive what you do.

All companies have a strategy whether they realise it or not. I don’t expect many local shop owners spend their time thinking about strategy but they have one. For them strategy is mostly location, location, location. Open a shop in a good location and you will get trade. OK there is a little bit more to it, does the shop sell newspapers? fruit? ethnic foods? In the abstract these are issues of strategy but I expect each one is considered without the need to use the s-word.

This is what I think off as implicit strategy, these businesses have a strategy but it is not spoken about. This approach works - well it mostly works, provided little else changes they can carry on as it. However most of these business are small, one shop, maybe two. But, the more these business grows the greater the need to make the strategy explicit.

An explicit strategy is one that is stated and articulated. This means it can be shared and communicated. If you are going to expand beyond the ideas on the owner’s head then it is necessary to communicate so everyone can pursue the same aims.

Moving from implicit strategy to explicit strategy allows three things (at least) to happen:
• Strategy can be communicated and therefore shared so more people can be involved in the business
• Strategy can be analysed, therefore problems can be spotted and new opportunities can be found
• Strategy can be used to drive execution; so structures, processes and products can all be built to ensure the strategy is successful.

In other words strategy is about scale, it is about creating the conditions to grow.

Now we can return to software development... where exactly the same process happens occurs.

A programmer can have a great idea for a program, and like the shop owner, they can build it. They can stick it on the net and may even sell a few copies. As success grows they may want to hire more developers, thats where the software requires design, architecture and so on. Its about scaling. Once again it is about sharing the ideas and communicating them. It is about analysing the design, finding problems and creating the support infrastructure.

Just to be clear, I’ve said it before in other words: Software Design is equivalent to business strategy. It is about scaling up.

And as this software is success, and more programmers work on it money generation becomes more important. The first program sold because, like the local shop, it was in the right place at the right time, but to sell more copies, to pay for more programmers, you have to know what you are trying to do.

So the software is no longer just software, it is a business, and like all businesses it needs a strategy. And since this business is growing it needs to move from an implicit strategy to an explicit strategy - for all the reasons above and more. The software industry is highly competitive and international, things are always changing. If you don’t consider the changes, and how you will responded you won’t survive long.

Strategy matters more for software companies than to many others. This is because software, indeed all IT, is about change. You use IT (and therefore software) to create change. Either in your organisation or in your customers. You need to understand what kind of change you are creating, how you are creating it and why you are creating it. Organisations are in greatest need of strategy when change is in the air.

There are actually two strategy tracks in a software company, one embedded in the software design, and one stated as business strategy. The two are interlinked and can hinder each other. You cannot pursue a business strategy that your software architecture will not support, and you cannot expect a software design to work well if it runs against business strategy. We’re back to Conway’s Law but that is not my primary point here.

There are two problems I want to highlight. The first concerns strategy itself.

The trouble with strategy is that it is grand, it sounds grand and as the economist John Kay puts it strategy is a synonym for expensive. Consequently all too often strategy formulation becomes the work of ‘big brains’ in the company, or even outside consultants.

This excludes the bulk of the employees and this itself has two problems. First you miss out on their thinking and thus potentially miss out on new ideas and emergent strategies. Second once you have decided your strategy you need to communicate it. This isn’t trivial. Communicating a strategy means you have to be able to communicate it, then you need to actually communicate it, then you need to re-inforce the message and keep re-inforcing it. This takes time.

The second problem with strategy is that it doesn’t happen. Maybe we continue thinking like a small shop keeper: people keep buying our products so we don’t seen the need to rethink anything we do. Or perhaps we do rethink our strategy but perhaps we fail to communicate it. Perhaps only the ‘big brains’ know or understand the strategy.

And these problems are important when you are developing software. Because product requirements derive from the company strategy. If you don’t have a strategy, or you don’t have an strategy people know about and understand then you stand little chance of understanding what your software products should do.

Requirements: A dialogue not a document

I’m not a lover of requirements documentation. I’ve discussed this at length before - see Why Do Requirements Change? and The Documentation Myth. However requirements and even requirements documents do have a part to play in the development process. This fact has been brought home to me recently while working with ‘Requirements by Bullet Point’ and ‘Requirements by Project Name.’

These two conditions arise when people try to take short cuts with documentation. In the first case, Requirements by Bullet Point (RBBP), somebody lists ‘the obvious’ requirements, or the ‘things we all know’. In the second, Requirements by Project Name (RBPN), people come to believe the project name conveys all the requirements necessary, unfortunately, the same name tends to mean different things to different people.

What such requirements miss is that the devil is in the detail. Every line of code in a software product contains multiple decisions. A product with 10,000 lines of code contains millions of individual decisions. Some, many, of these decisions relate back to the requirements. When this information is not available the guys writing the code have to guess. When developers have to guess it is the business which suffers.

Agile development lends a whole new twist to the problem because it specifically sets out to deal with changing requirements. But some people believe this means you can start out with no requirements. In traditionally waterfall projects requirements came at the start so projects should not proceed without them. Thats the theory but in practice such projects also fall pray to RBBP or RBPN.

The problem isn’t helped by developers who believe requirements are not necessary, after all, developers know best. Occasionally this is true, normally it occurs with very technical products or where a developer is themselves an expert in the field. The danger is always that developers mistake what they want to do, or what they can do, with what needs doing. What needs doing (the requirements) always needs to be rooted in what the business or market wants and needs.

To make things worse requirements often play multiple roles in a project. Not only are they technical requirements but they are also part of a legal contract and a financial rewards (and penalties) system. When they are called on to fill these multiple roles things just get more complicated.

The problem is that requirements (and I extend this to include specifications here) are just plain difficult for all the reasons I outlined in Why Do Requirements Change? and The Documentation Myth and more.

I think the solution is to recognise that project requirements are an ongoing, evolving, entity. They are not written once and set in stone, they are a continuing process not an event. Or to put it another way, they are a Dialogue not a Document. And to have a dialogue you need multiple people. One of these people needs to be the owner of the requirements, these may be called the Product Manager, the Product Owner, a Manager or a Business Analyst.

The important point is there needs to be someone who is charged with having these dialogues and ensuring a consistent view emerges. Consistent doesn’t mean unchanging but it does mean it remains fairly stable and only changes when new facts or events emerge.

This person needs to be hold an ongoing dialogue with all the stake-holders: the customers, the end-users, the developers, the market, the wider organisation and even the software itself, by looking at it, evaluating it and looking at its competitors. Somebody needs to be charged with knowing what the product needs to be and articulating that view.

Requirements documents have a part to play too. The process of creating the document is an learning exercise for the product owner. In writing the document they are forced to research the subject and pull all their thoughts together in a logical order. For other people the document shows that somebody has done their due diligence and thought about the issues.

Unfortunately too many people in too many companies believe a static document, or RBBP, or RBPN or thought-transference is sufficient to get a software product created. Waterfall had it partly right in this respect, you need requirements to a project, you need to know what you are trying to do, but you need to constantly keep asking and re-asking that question and stating the answer.

Agile: Software and Strategy

One of the reoccurring themes of this blog is my belief that there are strong parallels between the creation of company strategy and the development of software. For a start both are intellectual exercises that produce abstract models. You cannot tell how successful either is until it is tested in use and both require large amount of intellectual thinking.

I also believe that many of the techniques used in software design and strategy formation can be successfully applied to both domains. Both domains can learn from the other.

And, to top it all, because so much business strategy is implemented through software I see software execution as a continuation of the strategy process.

Consequently I’m always on the look out for evidence to support this theory. This month I thought I had found some more evidence. An article in this months issue of the MIT Sloan Management Review entitled Closing the Gap Between Strategy and Execution by Donald Sull seems to support my case. In fact it seems the article has discovered many of the practices of Agile software development in the strategy formation and execution process, e.g.

• Use of iterations: plan some, do some, assess and re-plan;
  
• Use of commitment, or promises, over estimation
  
• A reflection process that sounds a lot like the Retrospectives we practice in software development.
  

So was this the missing link?

Unfortunately not, in this case it is hard to disentangle cause and effect. First of all you have to go back to the roots of the Agile software movement...

Agile development has its root in the Lean manufacturing movement, and in the “Quality if Free” ethos, and in business strategy itself.

The term Agile itself comes from the business domain, tracing backwards – and I haven’t tried very hard – the first reference I know about comes from Alistair Cockburn (Agile Software Development, 2002) who cites Goldman (Agile Competitors and Virtual Organizations, 1995).

Sometime in the 1990’s the idea of ‘Business Agility’ arose. This raises its head from time to time in business literature but only in software development does – as far as I know – it seem to have taken off. (I have a theory or two here and I’ll explore them in a future blog if I get the time.)

So this article builds on research into software development processes which are themselves in partly the result of reading the business literature. You see why I say cause and effect are hard to untangle?

In writing this blog I’ve looked up the Donald Sull’s web site and discovered a lots of mentions of “Strategic agility” and it seems he teaches a course on this at LBS. Which goes to prove that business Agility is alive and well and prospering in at least one business school.

Unfortunately the business agility community and the agile software community do not seem to have much cross over. This is where Sull’s article in the Sloan Review is interesting because he does mention software development. However he doesn’t mention Agile software development, neither does he fully reference the agile software development literature.

Sull cites Scrum as an example of how strategy formation might work, and how the “new rules” of strategy formation might be used. However, given that Scrum is Agile, and Agile has its roots in business is this reference circular?

To be honest, I think Sull has missed some points. As I just said I think he could have referenced the software development literature he cites, for an academic that strikes me as a little sloppy.

Secondly he proposes a strategy formation cycle of four steps: Make sense, Make choices, Make things happen and Make revisions. To me that looks a lot like Deming’s Plan, Do, Check, Act (PDCA) cycle.

I’m probably being picky here. The MIT Sloan Review is towards the popularist end of academic journals. It tries to contain articles that managers might actually read. Had Sull given us the full academic treatment (and he may well have done so elsewhere) the thing would be read a lot less.

So, where does all this leave us?

Well I take it as good news. Someone else is finding links between software development and business strategy, specifically in the Agile field of both subjects. This can only be good news if it brings the two domains a little closer. And personally I think this further validates my theory.

This may also show that the software development community are at the cutting edge of understanding how you actually go about doing agile anything. It is one thing to talk about an Agile strategy but how do you execute one? What do you actually do?

Big MacSwitch

Ten years after I gave up my last one, and after five years of lusting for another, I did it last week. I made the switch to a Mac – or back to back if you prefer.

During that time the Mac has moved from 68020 (I think) to PowerPC to Intel, and completely rewritten its OS so I was expecting something very different from what I last owned. So I’m surprised by how much is actually unchanged, Finder is still there, so is System settings and the short cut keys I learned as a student still work. But so much more has changed...

The challenge now is to adjust to MacSoftware. I ducked and bought Microsoft Office so I still have Word and Excel, I’ve installed Mac Firefox and Thunderbird, and I’ve bought Endnote for reference tracking; and I’ll buy more. But I’ve also given up a lot of software, Visio and BlogJet being the most notable ones.

I don’t know yet if I’ll give in and buy an emulator/virtualisation package, somehow that seems cheating. First I want to see if I can get genuine Mac software. A Mac replacement for BlogJet is top of my list. (Any recommendations?)

But much of this has to wait. Although my book is finished (the publisher even sent me mock ups of the cover this week!) I’m hard at work on the index. This has to be the most boring, uncreative, tedious aspect of writing the whole book.

At the moment doing anything with the Mac takes at least 10% longer than it did with my PC. I’ll hang in there because, and this is the reason I finally decided to move from Windows: there is no point in buying a new PC.

Yes I could buy a smaller machine, that would be good but is not enough alone. My 5 year old Compaq still has all the power I need so there was no point in a faster process or more disc space (plenty left). Although it was showing its age in other ways.

But if I bought a new machine I would either get Windows XP – no change there, and future software compatibility problems I guess – or I get Vista. Thing is Vista represents as big, or even bigger, change to the Windows UI than a Mac does. So I might as well take the pain on a machine I’ve been lusting over!