Developers are not the only fruit

My younger self would be horrified to hear me say this but, when you develop software you need people who are not software engineers.

Don’t get me wrong, coders/engineers/programmers are the most important people because these are the people who actually produce the thing. As such they are central to any development effort, you can’t write software with a collection of managers, analysts and testers any more than you can build a ship without shipwrights.

Coding is where the metal is bent, shaped and welded. When coding there is no room for ambiguity or fudge, the code doesn’t lie and it doesn’t take to ambiguity. Thus there is a lot of details in coding which aren’t obvious before you start. So your developers need to be able to call on Product Managers for more detail and, if you have any architects, you need them at the code face coding with your developers.

See Jim Coplien and Neil Harrison’s patterns Work Flows Inward and Architect Also Implements for more discussion of this topic.

One of the worst projects I ever worked on was completely top heavy with managers (programme, project, customer, delivery, team - you name it we had a manager for it), testers, analysis, admin staff - yes you needed a lot of admin staff when the project was over 120 people. But only about 12 actual coders. Crazy. The overhead was there because... well, the ISO-9000 procedures needed a lot of managers, and it gave the consultancy implementing the project a lot of bodies to bill for, and it gave the Government (the people who were ultimately paying for it all) a lot of reassurance that it would be done one time, to budget and spec. Except, it was late, the spec trimmed left, right and centre, and if there ever was a budget we blew it.

So you see I’m no fan of carry extra weight on a project.

But, you need people who don’t code.

I’ve written before about the role of the Product Manager and why product management is important, and why you need them. So lets say we accept that Product Manager are needed.

In an idea world we wouldn’t need software testers either. And I expect one day to work somewhere where there are no testers. I don’t believe this is an impossible dream. But with the current state of play we need software testers in most organizations.

One role I don’t think we need is Software Architect. On the whole I think most architects are mislabelled. They should be Product Managers, Business Analysts, Senior Software Designers/Engineers or even Project Managers. Sometimes this is because the organization misuses the title, sometimes its because they don’t understand it (after all, what is software architecture?) but most of the time its because of title inflation. You couldn’t give someone a pay rise so you gave them a nice title.

Although I don’t read Joel Spolsky very often I do agree with him about Astronaut Architects. The moment your architects stops coding its time to fire them. Once they stop coding they are a burden not an asset.

Project Managers are another group I don’t think we need. I accept teams need some leadership but Project Managers are completely the wrong people to. Project Managers are trained to analyse, dissect, use whips (badly) and emit hot air. Their are not leaders by nature or training.

The Agile world is re-inventing project management as coaching but unfortunately the Agile world is pretty messed up about coaching. Coaches are useful when you they get it right and they are genuinely improving the team rather than acting as Project Managers with a different title.

But, and this is where my younger self will have a fit: you do need someone to oversee and manage the whole development process. The big problem is very very few people can do this job right.

By way of explanation to my younger self: done badly this role is worse than useless, done right it can really improve things. Unfortunately most people do it badly.

Its done badly for several reasons.

Firstly, most Software Managers have never been trained in the role - or management at all. They ‘make it up as they go along.’

Second, some of these managers don’t understand software development, they come from outside the domain. Such people try and make software development fit their existing model of management but it never will. Therefore they are constantly trying to make a square fit in a round hole.

Third, many of the managers who come from inside the domain, and know how to develop software can’t give up coding software. To be a good software manager you have to stop being a coder. You have to learn to look at the world differently, you have to realise you can’t intervene to fix problems, you can only arrange things so other people can fix them.

When I was younger I once worked for a bank. The code I was working on was terrible and I had no support from the organization. I really really wanted my manager, and his manager, to come and code with me. I thought their big mistake was to stop coding. I was wrong.

Their biggest mistake was not to stop coding but to stop communicating with those who did code. The big mistake was to put too much space between themselves and the building process. Had they involved themselves more they would have better understood the problems and would have been able to arrange things so I could fix them.

If you are managing a software team you should be talking to them every day. You should be improving their working conditions every day. You should be allowing work to flow to them.

When you move from coding to management you have to give up part of yourself, you have to trust the people you manage. It is their code now, not yours.

So, in conclusion: it is wrong to think you only need developers to create software. It is also wrong to think you can code and do other things - you need the separation. But it is also wrong to have too big a separation.

Make any of those mistakes and you just moved one more duck out of line.

The lining up your ducks theory of software development

Software development is easy. Don’t listen to what people say. I was 12 when I started programming and I picked it up no problem. Businesses are full of people who programme without even knowing it: Excel is programming by another name, and how many people write macros?

Yes, programming is really easy. You can buy ‘Programming for Dummies in ...’ - well just about any language you want. People can, and do, create small world changing programs without ever really being taught how to program.

That’s why software development and new businesses go hand in hand. Its one of the reasons why so much innovation today appears as software.

But....

Software development is also one of the hardest things. In fact, developing good software is so complex it might be the most complex thing man has ever attempted.

Moon landing? - it used software, compared to the complexity of the systems in the international space station I’m sure Apollo looks simple.

Nuclear power? - compared to the maths used by the finance industry today it looks simple. And that is all in software.

The thing is, writing a small piece of software can be (but not always) very easy. Problems set in when you want to grow the software, want more people to use it, want to sell it, want to package it, want to re-use the code or ideas, take bugs out of the system, add features, make it run 24x7.

It is very very easy to write simple software and have it do something useful. It is an order of magnitude harder to get it to commercial quality and keep doing it. And, to make it harder, there is no single accepted method for doing this and getting it right every time.

It sometimes amazes me that we get any of this right. More amazing is that software which is fundamentally flawed works, or at least seems to work. Stuff which by any engineering criteria is broken is used by companies every day, they even base their business on it. (There is another blog entry in this.)

Developing good software, delivering on time, producing with sufficient quality, etc. etc. - all the things you expect from ‘professional systems’ is a matter of getting your Ducks in a Row. What I mean is...

Creating good software, meeting expectations, delivering on time and the rest is not just a matter of getting one thing right. It is a matter of getting many things “right” - or at least workable. You can get some of these wrong and still delivery something - maybe a little late, or lacking features but you will do something.

Each time you do something badly, each time one of your ducks is out of position, you won’t break the whole thing. This isn’t a chain, instead of breaking you will reduce your productivity, or move away from the target, or reduce quality. With each duck that moves out of position it gets worse but it still works, somehow.

Put it another way. If creating ‘the best software ever’ means scoring 100%. Each time you move a duck out of position you remove 10%. So:
• don’t co-locate your developers, loose 10%
• start work without talking about design, loose another 10%
• work to a fixed specification or work to no specification loose 10% as well.

Down to 70%.

• Fire developers half way through the project, loose 10% for lost productivity, loose 10% for moral.
• Projects looking late, add more developers, loose 10% for violating Brooks Law.

40% doesn’t look good does it?

But there aren’t many case where you need to score 100%. 90% will do, even 80%, 70% often, some customers will be happy with 40% - or less.

The people who get upset most by this?

Not customers, they get annoyed but quite often there is nothing they can do about it.

It is the developers themselves. They see each loss, they know where all those 10% are being lost. And these guys are problem solvers, they want to fix these problems.

Now go back to my falling off a log theory of business.

I said: you do something right. Not 100%, maybe you just need 10%. But that one thing is about doing something useful, something other people want. Its about benefit and effectiveness.

Viability of your software product/company =
        benefit of your software
                x
        effectiveness of your organization

If your software doesn’t bring many benefits then you need an effective organization to stay in business. You have to be better than the rest.

If your software brings lots of benefits then it doesn’t matter how ineffective your organization - shoot the ducks! - you will stay in business.

If your software has few benefits and you can’t deliver, then Game Over.

Which leaves... lots of benefits and a highly effective organization. Then you have a world beater.

Unfortunately few companies fall into the last category which leave most people working in one of the first three.

'Changing Software Development' is published!

About 5 years after first thinking I could write this book, about four year after I first worked out what the book would look like, and 14 months after signing the contract my book, Changing Software Development is now published.

You can now buy it at Amazon, or from the publishers John Wiley.

'In search of mediocracy' - The falling off a log theory of business

When I was at business school my classmates and I read lots of good stuff about how to run a good company. Books like ‘In Search of Excellence’ and ‘Built to Last’. But I remember some of my classmates asking: ‘Why aren’t the companies I’ve worked for like these?’ To most, if not all, my classmates these books described something that was not the case.

That was five years ago. Since then I have met many people, few if any of them have worked for ‘Excellent’ companies.

Indeed most of the people I meet (particularly in software development) tell me crazy things that have happened at their companies. Decisions to close down offices just before a contract is won. Objective setting sessions were the objectives set are meaningless. Management which does the opposite of what they know is best.

One company I worked with in the last five years came closer to ‘excellence’ than any other company I know. Still, it got into trouble and has had three rounds of downsizing in the last two years. From what I hear things are pretty crazy now. Apparently in the last round they paid off four engineers, soon discovered they needed them, tried to hire them back, HR got involved and slowed the whole thing down during which time three of them got new jobs.

Another company I worked with was perpetually shooting themselves in the foot. So much so that they even let go the one person go who could make sense of the development team. Just this week I learned the same company which is letting one of their key managers go.

I end up hoping the end comes soon but these companies survive. The books say they should go out of business but they don’t.

So it is that I find myself asking ‘where are the excellent companies?’ And I feel pain when someone else tells me another story of the stupid things their employer is doing.

I’ve come up with a theory which explains it. I apply it only to the software industry because this is what I know but I suspect it might be more universal. It goes like this...

Like having a baby there is no law against starting a company. Actually there are a few laws, and in some countries it can be hard, but in most Western countries, provided you do not have a criminal record you have a right to start a company. In the UK it can take less than a day to form a company.

Most of these companies will go out of business in under a year. If I remember the statistics less than 10% survive year 1.

This 10% have done something right. Maybe they hired the right people, maybe there were in the right place at the right time. Maybe their competition was more of a mess than they were. Perhaps they really did have ‘first mover advantage.’ In other words, many of these companies survive purely on luck. They got something right. That doesn’t guarantee they got anything else right.

Some of these companies will go out of business in the years that follow but the longer a company remains in business the greater its chances of surviving even longer.

Over time these companies will gain some customers. Now customers can be surprisingly resilient, not because they are ‘loyal’ but because switching costs are great and because they have to decide to go elsewhere. Remember, these companies may be dysfunctional themselves so changing isn’t easy.

When you have a few customers, particularly big ones, you can make the money go a long way. Software companies which pull in their wings and go into maintenance mode can shed most of their staff and still stay in business for years. In fact, I would argue that almost any company you think is about to fail can last another year.

Notice there is nothing here which says the company will be ‘excellent’, or even that it will do what the management text books say. All I’m saying is that crazy mixed up companies can survive for a long time when they shouldn’t. Many of these companies have little right to exist by business school standards but they do.

As a consequence many, if not most, companies are, to some degree dysfunctional. Many people work for these companies. Many people are abused by these companies. And these companies are enough to put you off companies altogether.

I call this my ‘Falling off a Log Theory of Companies’: Starting a company is as easy as falling off a log. Most companies which come into being will fail quickly, the majority of those fail will be to some degree dysfunctional but will survive. Consequently most of the people I know will work for dysfunctional companies.

I think this theory applies especially to software companies for two reasons. Firstly there are still many many opportunities in software to be found and exploited, i.e. you can still get lucky. Secondly, the IT industry, and those who fund it, are massively optimistic. We accept failure too easily.

A few months ago I explained this theory to someone not in software. He said he’d seen the theory in action too. He also suggested the book these companies need is not ‘In Search of Excellence’ but ‘In Search of Mediocracy.’

New pieces on the website

I’ve been holding out, waiting until I have the time to write a really great start-the-year blog entry. Well I’ve not had the time and I’m about to go on holiday for a week. Still, just time to say I’ve posted some new pieces on my website.

• Session notes from Creating Awareness / Exposing Problems is the result of my ACCU 2007 conference session. These notes were published in ACCU Overload last month and are now on the website.

• My patterns from VikingPLoP 2007 (More) Patterns for Software Companies were uploaded just before Christmas but never announced here.

And what for this year?

Well I think I’m going to try and blog less. Yes less. Two reasons really.

Firstly I have plenty of ideas for blog pieces but quite a few of them would be long entries. Some of these I will write up but I am included to place the majority in ACCU Overload. When a pieces runs to several pages I think this is a better forum. I’ll still upload them to the website and announce them here but by their nature they require more time. More time on these means less time elsewhere.

Mentally I’ve accepted that I am going to write a second book. This will be concerned with, well, Business Patterns and Strategy for Software Companies (umm, that could be a good title.) Don’t expect it much before 2010, I intend to avoid signing a contract for a while. I’ll divert some of my material to this instead of the blog.

Secondly, I’ve spent most of the weeks before and during Christmas researching a new business idea and developing a business plan. Very few people know what I’m planning but this select group increased by one at lunch time.

That also means my consultancy work may have to take a back seat when things get going.

(By the way, if you know anyone who would like to invest please let me know. Current projections suggest I need about £100k).

In the meantime, if your missing this blog you can buy the book. Changing Software Development is out in a few days, I heard today that Amazon are already publicising it in e-mail.

And if that is not enough I’ll be appearing in Cambrigde later this month, at SPA and at ACCU 2008 - events list here.