Low level failure

The last couple of weeks - if not months - have been depressing. I've been hearing far too many stories about IT failures. Not the kind of failures that bring everything crashing down, rather the kind of failures which stop people from being productive and keep people working far harder than they need to on projects that will not live up to their potential benefit.

These are what I call Low Level Failures. Things keep working but they make people's lives miserable and cost us money.

At the risk of depressing you too here's what I've been hearing:

• A major international bank that hires new developers and leaves them for two weeks without a PC. No e-mail, no intranet access, no tool to work with. Pure waste.
  
• A major international ISP which is developing some new software in-house. The team leader has been complaining about internal customers who won't come and look at the software his team are developing. Now he's found out that the company managers want the users kept away from the software. What chance have they of delivering the right thing?
  
• Yet another major international bank (there are a lot of them in London) where the project manager is preventing the team doing the right thing. The code is a mess too but she doesn't want them taking any risks. The project manager knows that if the developers leave the project is lost. What she doesn't know is that the developers see her as the problem and if it wasn't for one of them taking the lead nothing would happen. And if it wasn't for this one developer encouraging the others they would have gone by now. Somebody need to help her.
  
• An investment house in the City were the internal customers who will use a new IT system don't think it important enough to turn up to meetings about the new system. Why bother developing it then?
  
• Another international bank were the project is a mess. The manager in charge got drunk one night and confessed he knows they are in trouble but doesn't know what to do. Good he knows there is a problem but shouldn't he ask for help when he is sober too?
  
• One of the banks mentioned above work to such internal procedures that there is nothing for the developers to do. I'm coming to the conclusion that the reason companies have internal procedure and process standards is not so much to make sure work gets done but to insure that people can't do too much damage. Won't it be cheaper to just not hire them?
  
• An information supplier that decided to launch a new project. Hired an offshore development team and hired people in London for the project then had second thoughts. People started work on the project only to be told there was nothing for them to do and then get laid off. Shouldn't they have thought this through a bit more?
  
• One of the banks already mentioned have a lengthy interviewing process to ensure they get good people. But it hasn't stopped them giving a job to one of the worst developers I have ever met. This person will hold the team back, they will make more work for their colleagues. Anything they do create will be a nightmare to maintain. I can only imagine they were hired to fill head count or because the person doing the interview was incompetent. Fortunately the bank procedures will probably stop them doing any damage.
  

Thing is, although I've heard all these stories in the last few months they aren't really new. Back in 1998 I worked for a company in the City were it took two weeks to get me a PC. When they did it was massively under powered. The support department had better PC in stock but held them in reserve in case another broke.

My point is not that failures happen in IT but that we live with so many low level failure every day. This failure saps our energy, our enthusiasm, it makes work dull and boring. In each case the situation can continue because it doesn't cause anything to really break. No crunch moment occurs. Things are just generally bad.

These examples are mostly taken from the financial markets. There are two explanations for that. The obvious one: I live in London and talk to people who work in the financial sector but I'm sure low level failures happen everywhere.

Second, the financial markets are booming just now, banks are throwing money at projects. Consequently it is easy for a project to burn money without showing a return. And because there are so many projects there is a shortage of developers so the really bad ones get hired. Not only do they get hired but they make things worse.

There is a third explanation which is quite scary but may contain an element of truth. The people I know in IT are in general good. Perhaps they are not just good but are outstandingly good. The things we see as problems are what others see as normal. Could it be that outside of my bubble the rest of the world is used to massively inferior service and IT? I hope I am wrong here.

Its easy to say "Huh, what do I care?" but actually these companies are spending your money. One of the banks not named above may be home to your cheque account, your pension, your savings. You may be a customer of that ISP, or your pension might contain their stocks. Their waste hits you.

Am I the only one to see this? Can't other people see these problems? What are the managers at these companies doing?

It is a double loss: the people on these project have a miserable time and we all lose money.

What now?

One, I feel better having shared the idea of low level failure with you. I will sleep better tonight having got that off my chest.

Two, there is a link between my last post about agile and these projects. Some of these projects were agile but not all of them. I will return to this subject soon.

The start of the end for Agile software development?

On the one hand Agile software development has won the war. Agile (and its cousin Lean) are moving into the mainstream and becoming the accepted norm. But I’m detecting signs that things are not quite what they seem. I’ll write more about this in future as I better understand what is happening. For the moment this is how the argument is shaping up so far...

• Agile (and Lean) have become a bit like apple pie: why wouldn’t you be agile? Therefore nobody wants to admit they are not agile and consequently we are all agile.

• A few agile practises like test first development and stand-up meetings are becoming much more common than the others. Based on the adoption of a few practises some people and organizations believe they are agile. (For me the true test of agile is that the team is learning and changing, if you are not holding retrospectives, if you are working the same way you did 6 months ago then you are not agile.)

• Business does not see agile IT the same way software developers do. The idea of “agile” started in a very very small way in the business world. A few people the software community picked up the term, adopted it and grew the idea. Now the term is moving back to the business world and become much bigger. Businesses need to be agile. Making IT agile has become the mantra of management but they are a long way from YAGNI and pair programming.

• Snake oil salesmen have been at work. Agile IT can mean: SOA and server virtualisation. More and more firms are pushing tools to make you agile but true agility comes from people not tools.

Perhaps more worryingly there may be a resurgence in high ceremony development methods in the offing. IT Governance is getting a lot more attention, partly because of things like Sarbanes-Oxley and Basle 2. The problem is that while IT Governance is itself a good thing, and something agile can support and work with, many people – read auditors – equate compliance with procedures and documentation.

Auditors don’t understand IT but they do understand things like written process manuals. The danger is that we re-run the 1990’s and throw away what agile has bought us. It doesn’t have to be this way, agile software development is compatible with regulation.

The real joker in the pack might be ISO 9000. In the 1990’s ISO 9000 was the epitome of high ceremony projects that didn’t achieve anything. Things have changed in the ISO 9000 (specifically ISO 9000:2000) world and it might, could, just be possible, that if we can work out how to get agile teams through ISO 9000 it could provide the insurance people need.

However that is far from certain. The new ISO 9000 could be the same old hammer used in the same old way by people who don’t understand how software development works.

Unfortunately too many people are concerned with the appearance of control rather than actual control. The illusion of control keeps too many managers in their jobs.

(No I don’t have anyone specifically in mind when I say this - I’ve known a whole bunch of these people. Oddly they usually go by the title of project manager.)

Anyway, that’s a quick overview of why I think Agile development as we know it might be about to end.

Bookings open for ACCU Conference in April

A little bit late, you’ve probably heard the news already. Another year and another ACCU conference rolls around. This year’s line up looks like one of the best yet – but then I’m biased! Keynotes from

• Mary and Tom Poppendieck - of Lean Software Development fame
  
• Mark Shuttleworth – the man behind Ubunto Linux, the Shuttleworth Foundation and the first African in space
  
• Peter Goodliffe – regular ACCU writer and activist, author of Code Craft
  
• Hans Boehm - the man designing garbage collection for C++
  

I’ll be speaking at the conference on Friday afternoon. My subject is “Exposing Problems/Creating Awareness.” As with my other recent sessions at ACCU this will be about one third presentation and two thirds workshop. I expect the audience to help expand on the subject with their own ideas and experience.

In some ways I still can’t believe I’m on a conference programme next to the likes of Jim Coplien, John Lakos, Michael Feathers, the Poppendiecks and others.

Originally this was a C/C++ conference but it has grown, matured, changed and now takes in a range of subjects including C# and Java. Still C/C++ is well represented but so too are Agile and Lean topics. So just to be clear, read my lips: this is more than a C++ conference.

As it happens the C++ standards panel is meeting in the same hotel the week after the conference so a lot of people like Bjarne Stroustrup will probably be hanging around too. I always remember my first ACCU conference, I was in a presentation about some advanced C++ features and the presenter said “I don’t know why this feature is like this, I guess we’d have to ask Bjarne” and this little voice at the back of the room says “I think I did it like that because...” and there was Bjarne just sitting in the session with the rest of us.

Once again this year I’m on the programme organizing committee for this conference. I’ve been on this committee for four years now and I’ve enjoyed it. However this will be my last year on the committee. I’m stepping down to make way for new talent and to give myself a little more time to work on my book and other things.

During this four years the committee has taken the conference from strength to strength both in terms of material and in terms of attendees. One of the strong points of the conference is it has space for the big names of software development – the Stroustrups and Poppendiecks – but it also has space for people who have not presented before or are building their career.
As I described after last years conference.

Last year the conference sold out and we were bursting out of our venue, the Randolph Hotel in Oxford. So this year we’ve moved to the Paramount Oxford Hotel.

This hotel offers great facilities but unfortunately its not in the centre of Oxford, although it isn’t far out and is not in the middle of nowhere. The trouble is: there just aren’t that many big hotels in Oxford with the facilities we need for this conference.

The conference committee has, partly by accident, partly be design, fallen into a pattern of replacing one member per year. Last year Kevlin Henney stepped down and Tim Penhey joined the committee. This year I’ll step down and, well, we don’t know yet who will join the committee.

Anyway, you don’t have to join the committee to come to the conference. If your work means you are involved with software development and you want to advance your knowledge then book yourself a place today.

Good TV for Managers: Robinson at the NHS

I’m not a big TV watcher. Some nights the TV doesn’t go on in our house at all. However for the last three nights I’ve been gripped by a short series on BBC 2 called “Can Gerry Robinson Fix The NHS?”

In this program Gerry Robinson (who you might describe as a celebrity business man, guru, turn-around specialist or something similar) spent six months helping an NHS hospital. His goal was apply some business management ideas to improve the hospital. Perhaps surprisingly it made for gripping TV and was highly education. (Note to aspiring managers: don’t bother doing an MBA, beg, borrow, steal copies of this series.)

What made it so fascinating? Well many things, basically you have management on one side who are trying to, well, manage, and you have a bunch of technical specialist on the other side (doctors, nurses, etc.) and there is a great divide between the two groups. Add in a history of top-down solutions which means nobody feels empowered and you have a recipe for Do Nothing.

More than once I found myself watching thinking: this is just like IT. The medical staff are just like programmers/testers/project managers and the management are just like, well, managers.

Robinson wasn’t given any money to improve the hospital – although he did manage to get some additional money spent. Instead his tools were those of inquiry, empowerment and using his legitimacy to get people to talk to each other.

Time and time again he found people who knew how to improve something but felt nothing could be done. They felt somebody would block any change or that nobody was interested in changing. And even if they did want to change things they could not get the right people to talk together or agree to actual action. About the half way mark this seemed to get Robinson really down.

But then it turned around and things started moving. People started to feel empowered, they started to feel things could change and they did start to change them. And consequently Robinson got much happier!

Of course its difficult to tell how much this chorology actually happened and how much was constructed through judicious editing to make a good story. Either way it certainly seems that people started to feel they could change things and this led to improvements.

I found myself agreeing again and again with Robinson when he said things like “Management isn’t a mystery” and “People doing the work know the answers”. It is all about getting the people who work to actually take power and make things happen. Unfortunately management can become a block to change rather than a catalyst.

There are so many barriers to change. The first barriers are in peoples own minds. They think things can’t be changed, or others won’t agree to them. Several times in the TV series people said “Mr X won’t agree to that” only to find that Mr. X and everyone else did agree.

The next barriers are the problems other people see. They jump to assumptions and see reasons why things won’t work or why you can’t do that. Sometimes there are real reasons and sometimes imagined. Either way these blocks can kill an idea dead. When they are imagined it is a case of checking reality, maybe by asking someone else. When the problem is real it is a trigger to work around the problem, to find an alternative answer or a different way of working.

Barriers of communication existed too. People didn’t talk to each other; they assumed what they thought to be the case was. Managers didn’t talk about problems, they preferred “strategic solutions” (shuffling paper, making plans and changing structures.) Technical people sometimes wanted to do their technical thing rather than engage in conversation and tended to see anything management did as a problem.

There never seemed to be a barrier with authority, only assumed authority. Neither were resources ever a real barriers. Once or twice lots of money or construction was needed so ideas were dropped but usually that was just led to a new idea that achieved much the same success.

In most cases all that was needed was someone to show an interest, someone to lend legitimacy to the improvement ideas, someone to ask for it to be done, someone to inject a sense of urgency and occasionally someone to bang heads together and get people talking to each other. And most of all: someone to appreciate what people had done, someone to say “Thank you.”

Robinson wondered around the hospital spotting opportunities and applying this (his) philosophy to the situation. It wasn’t genius but it got results. The rest of the time he seemed to spend his time trying to persuade the hospital chief executive to do the same thing. This guy was a “big thinker” who wanted “strategic plans” (my terms not his, this is the way he appeared to me.) As a result he was out of touch with what was happening. Staff felt he was remote and nobody (except Robinson) was walking around making people feel useful and conferring legitimacy to change.

I think Robinson’s greatest success was finally persuading the chief executive to try this approach. At first the exec was sceptical (“Why do they need me?”) but once he had one win under his belt the exec was converted and saw the point and was converted. His real power lay not in making plans, re-organizing structures or signing checks but in simply asking people to do what they wanted to do.

At the end of the day it wasn’t a big plan that was required, it wasn’t a new strategy, nor was it more money and resources. It was simply a thousand little improvements and a constant attention to further improvement.

As I said at the top, a very good programme. If there was more TV like this I’d watch TV more often. I’d love to buy the DVD of this series – if anyone out there knows where I can get it let me know.

More trouble with New Blogger

Sigh, I just wrote my first proper blog entry for 2007 – not one of my most exciting I’ll admit! – and when I came to post it I found another little problem with “new blogger.” The Blogger API seems to have changed and now BlogJet can’t post to it.

I’ve been using BlogJet for a while now to write and post my blog entries. I like it, it works for me. Sure it could be better: some short cut keys and spell-check while you type would be the two features I’d most like to see. But it seems that “new blogger” doesn’t work with BlogJet and the BlogJet people haven’t updated their software. Still the same version 1.6.2 as when I started using it in September.

So unless BlogJet gets another release pretty quick I’ll be back to Word and Bloggers own tools. Will this effect my blogging? Who knows. I do know that I have rather a lot of personal projects on at the moment so I should probably do a little less blogging.

Watch this space as they say!

Book review: The Box

A couple of days off work with a stinking cold has given me a lot of time lying on the sofa feeling sorry for myself. Fortunately its also given me time to finish off another book - The Box by Marc Levinson

Yes this book was short-listed for the FT book of the year by that wasn't why I read it. I would have read this book anyway because... well, I've got a thing about ships and even containers. It is hardly surprising, I grew up in Birkenhead, a docks and ship building town just over the water from Liverpool at one time country's biggest docks. And then I have family connections …

I should confess I once crossed the Atlantic in a container ship myself. I was the sole passenger on a trip from Montreal to Thames Port (east of London) on the Canmar Pride nearly 5 years ago now.

The Box is both a story and a business case study. As a story it is well written and easy to read. It tells how the metal container transformed ocean trade, the shipping industry, and if you agree with the author world commerce.

As a case study it is an examination of how technology changes things. A 40 foot long metal container might not look like technology but that is exactly what it is and in 1956 it was cutting edge. The introduction of the container hit many of the same problems and issues that any other new technology hits: trouble agreeing on standards, worker resistance, management failure to understand it and boom-bust economics to name a few.

Like so many technologies many people initially saw it as simply a means of doing the same thing faster and cheaper. Over time it became clear that the real value of the container lay not in moving the same cargos on the same routes more cheaply but in transforming the whole industry. Not only was the old technology superseded but the old business practises were swept away too. However, as is often true of technology, these changes took time to work through the system. In the first few years it wasn't clear that the container would even save money sometimes. Now its hard to imagine a world without containers.

I'd recommend this book for three reasons: its a good story, its an easy read and third its educational. And if your interested in ships then you’ll enjoy it all the more!

Problems with Blogger, Bloglines or me?

Just a quick note to anyone experiencing problems with the RSS feed on this blog. I don’t know how general these problems are or whether they are the result of a mistake by me, Blogger (aka Google) or Bloglines but...

Blogger have undertaken some upgrades to their service. After my last blog entry I decided to switch to the “new blogger” - their description not mine. Except for the fact that I now need to associate my Google account with my Blogger account I didn’t really see any difference. To be fair I didn’t spend long looking so it may well be there have been great changes.

Some while ago I found Bloglines (OK, Jonathon mentioned them and I had to check them out!) At first I didn’t see the point, it was just a web based RSS reader, why use that when I had Thunderbird?

Well now I do see the point, it is easier to use and I can read my feeds from wherever I happen to be; home, office, travelling etc.

Anyway, yesterday morning Bloglines reported a lot of my blog as new entries and gave some twice. Inconvenient but nothing too bad, could be the New Year, the switch to new blogger, or something else.

Then this morning Bloglines reports an error with my blog feed. Now, one of the “new blogger” enhancements is supposed to be improvements to the feed so things are starting to come together.

Still, I’ve managed to re-subscribe to my own blog through Bloglines without any problems and it seems to work fine.

Anyone else out there seeing these problems?

Is it my blog alone? – actually Keith Braithwaite’s blog exhibited similar problems yesterday morning.

Is it the Blogger changes? Bloglines? User error? Something else all together?