Friday, February 23, 2007

Nurnberg funnel and software development

After reading Minimalism beyond the Nurnberg Funnel I have been thinking what implication these theories could have for software development itself.

First for system design. Minimalist documentation aims at being task oriented. The idea is that the readers (users) are trying to do something and need to solve a problem – how do I? – so they turn to the documentation. What if we apply these ideas to system design?

Well I think we would come up with simplified systems that allowed users to perform tasks. The software would treat users as problem solvers and allow them to combine the tasks in what ever way they like to solve their wider problem. Now this starts to sound like Naked Objects.

Second: what if we turn our attention from user documentation to system documentation? Usually the documentation software developers write to describe the insides of the computer system is based on the Nurnberg funnel principle. We assume that is a software designer writes down everything they think they know about the software then someone who reads it will know the same information – open programmer’s head, insert Nurnberg funnel, pour documentation.... programmer can now understand software!

Obviously this isn’t going to work. This isn’t only true about existing software it is also true about systems we are going to write. Traditional software design (specifically the waterfall model) mandates that Architects and Designers – who in the traditional model are different to Programmers – should “design” a system, document the design and hand the design over to the Programmers so they will know how to turn the design into code.

Again, this is the Nurnberg funnel model and it isn’t going to work. When we follow this model we are lying to ourselves. So what are we to do?

Minimalism probably isn’t an answer in this case. Firstly the potential audience is so small it will be impossible to justify the extra cost. Secondly minimalism relies on describing the tasks people will want to perform. In general we don’t know the tasks that future system developers might want to undertake on our software. Yes we can guess what they might want to do but this really is little more than a guess. I have designed and built systems in the past with specific extension points for future developers – just add a new object here. But these are simple changes. The majority of the changes I’ve done to software involve expanding it in a way that wasn’t thought of in advance.

So while the thinking behind documentation minimalism can show the failings of traditional documentation it cannot provide a solution. We need to look elsewhere. I’m not sure I know the solution but I’ll make one suggestion.

The problem of making software developers familiar with a new code base is not a problem of documentation. Rather it is a problem of knowledge management and specifically knowledge transfer. Documentation is one tool in this arena but not the only one. Traditional documentation relies on a just-in-case model. Managers who are worried about risk have developers document the system just in case something needs to be done. This looks like a strategy to mitigate risk because we have “captured the knowledge” but in fact the risk still exists because the documentation is not very useful.

The alternative model is just-in-time knowledge management. Here we ensure that should a new developer need to know about an existing system they can ask someone who knows when they need to. The knowledge transfer then takes place directly from one developer to another.

Instead of the solution being based on something you have it is based on someone you know. On the face of this it this model looks costly. In order to get the knowledge we need a one-to-one relationship rather than the one-to-many relationship documentation provides. But since documentation does not work this argument does not hold.

In fact, writing documentation that might not be read, and if it is will not be understood represents waste. It is a waste of the developer’s time to write the documentation, it is a waste of paper to print the documentation, it is a waste of space to store the documentation and it is a waste of time to read the documentation. The only purpose it serves it to allow a box to be ticked on a risk form.

Using the just in time model all this waste is eliminated. If, and only if, the knowledge is needed then the cost of knowledge transfer will be incurred. And since the cost will be incurred in the future it is cheaper today. (I probably don’t need to spell this out but just in case: this model matches the ideas of Lean manufacturing, product development and software development.)

However, we are still left with the problem of mitigating the risk. To do this we need to keep track of the programmers who developed the system and have made changes since. This potentially changes the nature of the software development model but in today’s Internet connected world should not be insurmountable.

The problem then is that people, even developers, do forget over time. Still, forgetfulness might not be that much of a problem reality.

There are two types of system: those that are used and those that are not. The ones that are used tend to need changes over time; because these systems are living people think of new features and they need to be changed as the environment changes (think Y2K.)

Then there are the systems that don’t get used. Well if these systems are not used it is unlikely they need to be changed. Obviously for those systems that are being changed there are people who know how they work. For those that don’t change there might not be anyone who knows how they work but neither is there anyone who needs to know.

Of course there will always be the odd exception, a program that runs for 10 years without a change and then needs updating; but how many systems are there like this? Does the existence of a few systems that require very occasional changes justify writing all that documentation for all those systems were is isn’t needed, won’t be read and won’t be understood if it is?

Strategy implications from the Nurnberg funnel

I am still thinking through the implication of Minimalism Beyond the Nurnberg Funnel. One of these ideas concerns the benefits to the software producer of supplying minimalist documentation. One of these is the idea of documentation as a product strategy. This is covered by Karl L. Smart in an essay in the book.

Software companies all too often view the production of documentation as a cost, not an opportunity. Documentation is seen as a necessary evil since supplying software without documentation is not acceptable but writing it will push up costs. So companies produce documentation but try to limit the costs and miss the opportunity.

In my experience this usually means that too few technical authors are employed, those that are over worked and their activity is something of an after thought. They are not integrated into the software development regime; they get the product late and are expected to document everything they can. To cap it all technical authors are often among the first to be laid off when a company needs to make redundancies.

(Of course they assume that the role of technical author is even recognised, many producers do not see technical author as a role in its own right. Such companies make developers, managers and particularly testers write documentation in addition to their main job.)

Overall technical documentation is a lose-lose. Companies don’t see the benefits of providing it, users demand it but do not have high expectations and when they do turn to the documentation it is pretty useless.

I have seen a variation on this theme a couple of times, this is the highly complicated product. There is no way the user can learn the product from the documentation so they need training courses. Training courses can be a nice revenue earner for the supplier but they don’t normally don’t manage training very well so don’t make money from it. To the buyer training is yet another hidden cost. To the user there is not enough training, it is not good enough and it is another cost that can’t get signed off. Meanwhile the software continues to get complicated and everyone hopes the training and documentation will act as a sticking plaster.

Smart argues that providing quality documentation is actually a win-win for supplier and customer. Minimalism provides the road to useful quality documentation which benefits the product in a number of ways:

  • Good documentation differentiates the product from its competitors

  • Good documentation is more useful to users who in turn use the product more – and exercise the features - so they extract more value from the product so they are more likely to buy more product.

  • Good documentation means users are more likely to solve their own problems than call the technical support line. Therefore the supplier can save money on technical support.

  • Not only the supplier saves on tech support but so does the customer. Research shows that for every $1 of direct technical support costs a user needs they use $3 of informal support when they ask colleagues to help them.

  • Minimalist manuals are usually shorter than traditional one so they are cheaper to manufacture. Although now documentation is usually supplied in electronic format this argument is reduced.

Producing minimalist documentation might cost more because it requires more time and analysis from writers but this is offset against savings suppliers can increase profits by producing minimalist documentation. (A word of warning: minimalist documentation does not mean producing less of the same documentation, there is more to minimalism in documentation than simply doing less. It means doing things differently.)

Because better documentation makes the product a better product it enhances the sale-ability of the product and therefore improving documentation is a strategic move on the part of the supplier.

This also means that documentation should be an issue for product managers. In addition to specifying features and requirements of the software product managers need to RTFM and find a way of improving the manuals.

Thursday, February 22, 2007

Book review: Minimalism Beyond the "Nurnberg Funnel" and thoughts on documentation

I can’t remember where I came across Minimalism Beyond the Nurnberg Funnel but I’m glad I did. Still I think I may have read the wrong book, a better starting place for this subject would of been the authors first book on the subject The Nurnberg Funnel. Both books discuss the same concept, namely minimalism in technical documentation. However the newer book (Beyond) is not so much an introduction to the subject but a review of how the ideas had developed in 8 years. So I think the earlier book may have been a better starting position. Now, 9 years after the second book I assume things have moved on again.

The authors' (Carroll is the editor of the new book but wrote the original one) concern is how people learn, and specifically how they learn from documentation. To this end the books are about technical communication. I think the ideas suggested would be interesting to user interface designers to look at too. The reference to the Nurnberg funnel comes from mythology.

Apparently – and I’ve not come across this myth elsewhere – the funnel was a device that allowed knowledge to be poured into someone’s mind. Imagine knowledge as some kind of liquid, insert the funnel into the head then pour the liquid into the head. Thats it! You know have the knowledge! This sounds like a great device.

Although I’ve never actually seen such a funnel they must exist because I know plenty of people and corporations who behave as if such a funnel exists. So it must just be that I’ve been unlucky.

Anyway, get back to the book... The originator of minimalism in documentation, John Carroll, suggests that traditional technical documentation is written on this principle. The authors working on a technical product (most of his examples come from word processors) fill the manual with everything they can think of. And over time the manuals grow as new features are added. For the users of the product the manual can be worse than nothing.

Research by Carroll and others shows that users just don’t absorb everything in the manual through Nurnberg like reading. Users find it difficult to find things and when they do they don’t want pages of explanation and technical details, they simply want to be told what to do. Indeed there is a paradox: people don’t sit down and read manuals. They want to accomplish tasks so they just get on with using the software. When they get stuck then they turn to the manual but they still do not want to read the manual, they want to do their task.

At this point a manual that explains technical details and system philosophy is exactly the wrong thing. Users will get frustrated and skip parts of the manual until they find the bit they want – that is, the bit that tells them what to do.

Actually I’m glad to hear this. This is the way I use manuals, I always suspected that other people used them “properly” but now I know: everyone else does the same as me.

The minimalist solution proposed by Carroll is to make the manuals task oriented. Work out what the tasks the users will want to perform and explain how to do these tasks. In the process recognise that different users will require different tasks so document them as such. Manuals should be organized so users can find these tasks rather than by technical logic.

There is more to minimalism than this but I’m not a technical author so I won’t go into detail. The upshot of it all is: users learn how to use the software more quickly, are more satisfied, are more productive and the manuals are often smaller than the non-minimalist equivalents. But what I do recognise here is that a lot of these ideas overlap with the user interface ideas proposed by Alan Cooper in his book The Inmates Are Running the Asylum . And the same idea populates good product management or business analysis:

Identify your customers, understand what they want to do, really understand what they want to do, then cater for that need specifically.

Whether this is in the product you build, the user interface you design or the documentation you provide it all boils down the same thing: understand and be specific. Don't provide technology, provide the means to do something.

(Interesting I was talking to Tom Gilb at XTC on Tuesday. In his opinion one of the main reason projects fail is that they are not specific enough about what they are doing. There is a big conversation to have around the idea but not here, not today.)

Getting back to the book itself. If you are involved in technical authoring then I think it would be well worth reading either The Nurnberg Funnel or Minimalism Beyond the Nurnberg Funnel. If you are not technical writer then its not very clear if you would benefit from reading the book. If you want to know more about how people learn, or you are a product manager looking for ways to improve your products then it is worth familiarising yourself with these ideas.

I didn’t read all of this book, I read enough to understand the key ideas and understand what advantages they brought. This book is quite academic in style so it might be worth seeking out other books on minimalist documentation unfortunately I don’t know any so I can’t recommend any. There are more implications of this work than I have covered here. I hope to come back to these at some later date.

Saturday, February 03, 2007

BlogJet is back - version 2

About a month ago I lamented the fact that I changed to the New Blogger system.  Most of all I missed BlogJet the blogging tool I’ve been using for most of the last year.  Well there is a new version of BlogJet, version 2.  My last couple of blog entries have been with BlogJet and I’m glad to get it back, I’ll probably pay the license fee when the trial comes to an end.

I had a few problems getting BlogJet 2.0 to work with New Blogger.  I had to download a hot fix (version and re-create my account with BlogJet rather than import the old one but now it works fine.

BlogJet 2.0 also has some other improvements I’ll explore over time.  For the moment I’m glad there are more short cut keys.  Unfortunately it doesn’t do spell-as-you-type yet but you can spell check before you publish.

Two funny (odd) things….

Here is Blogger.  Blogging exists because it makes creating web pages an order of magnitude easier than writing HTML and uploading it.  In making it even easier blog tools change the content of the pages and the environment they exist in.

And then along comes a tool like BlogJet and makes blogging even easier!

Second, Blogging sites like Blogger are great examples of software as a service, and how easier tools delivered online are superior to installed software.  But then here is BlogJet, a classic install on Windows application that makes things even better!

So maybe, in a couple of years after we’ve all moved to SaaS and replaced Microsoft Word with Writely and Outlook with Google Calendar and GMail well rediscover software installed on a PC.

Book review: Siemens Best Practises

For once a review of a book I'm not recommending.  In fact I found Knowledge Management Case Book: Siemens Best Practises quite disappointing.  Not what I hoped for at all.

I bought this book for two reasons.  First knowledge management (KM for short) is so often a soft and vague kind of subject.  What exactly do you do when you are a knowledge manager?  Just saying "what do you do?" usually leads to the question "what is knowledge management?" and "how does knowledge benefit us?".  So I thought a book of case studies would shed some light on it.

Second Siemens were (are) one of the moving forces behind the European patterns movement
and several pattern people I know are Siemens employees.  So I looked forward to learning about how Siemens saw patterns.

Unfortunately I was disappointed on all counts and I'm giving up on the book after only 3 or chapters.
The chapter on best practise sharing starts well, for a start it acknowledges that defining what is best practise is hard.  How do you know you’ve found best practice? Who decides what is best? Does knowing the best stop you looking for better? 

From a good start the chapter goes down hill.  Although the chapter says it describes a "market place for best practise" I never found out what the market place was.  Did money change hands?  Did they all get together and shout?  In short the big idea was talked about but never revealed.  Instead there was a lot of small detail - a re-occurring theme.

What was interesting in this chapter was the route they used to document the "best practice."  There was a lot of overlap with the way patterns get written.  However no mention was made of patterns in this chapter or elsewhere in the book.

One chapter describes how Siemens created a knowledge management programme to train their managers in knowledge management. Again the chapter is heavy on detail but doesn't describe anything other than a online learning programme.  Yes managers came together for some classes, and they did some work on line but that's about it.  Nothing special about KM really, a staff development programme really.

Another chapter described how the company gave training to its managers.  Again I failed to see the KM connection.  It was an example of the application of knowledge but I didn't get any great new ideas from it.  Training managers is good, and yes it transfers knowledge but it felt this chapter was more about management development than KM.

I was disappointed that no mention at all was made of patterns or any of the people I know.  Perhaps Siemens is so over flowing with KM that patterns are a tiny part.  I just can't believe this.  I think the book missed something very important. 

Neither did the book escape the "what is knowledge management" and "why is knowledge good for business" questions.  Like so many knowledge management books this one spends too much time on these questions rather than just getting on with it.  With the added twist that everyone congratulations Siemens on being such a good KM company.

So I'm not recommending this book.  Sorry Siemens, I know you take KM seriously but this too much of a corporate book heavy on detail to recommend. 

Friday, February 02, 2007

Connecting blogs: Failure, Agile failure and bad companies

I ended my last blog entry - the one on Low Level Failure - promising to make a connection to the blog entry before that - the one suggesting we might be about to see the end of Agile software development as we know it.

Well there is a connection in my mind but it is somewhat hazy, I have to do some hand waving for it to make sense.  This is a clear sign that I don’t have a clear idea of what the connection is. So trying to explain it in a third blog entry is difficult. Consequently I feel I owe an explanation - a connection - but I’m finding it hard to do and its blocking thinking on other subjects.

If anything this will teach me not to give hostages to future blog entries. But here goes with the connection... please forgive me if I’m vague!

Really there are two levels. First about business as usual, both blog entries describe things that many working people - particularly those who read Dilbert - would consider perfectly normal and just routine business. Maybe we should call it the madness of companies. It is the madness that says: don’t rock the boat, don’t try and be different, don’t try to innovate.

This is a kind of despair: why do we have to settle with this poor situation? why can’t the world be better?

Maybe I’m just an idealist but it drives me to do better.

The second connection about the two blog entries and all the stories in them is that they are about dis-empowering people. They are all stories of how people were helpless inside some corporation - some mad corporation at that.

In all these stories the main characters are powerless. Whether it is the team leader who isn’t allowed to talk to customers, the developer left without a machine or a demand to follow some governance regulations, all these people are powerless. In these stories the main character can’t do anything.

Again maybe its me: why is it wrong to want to do better?

So in all these cases the stories are kind of depressing. They are stories of failure which could get us down, switch us off, drive us to drink. But they are also stories of opportunities. Stories of how things could be better. For those companies that kind overcome these problems a bight future exists simply because most can’t.

Now I write this down I understand my own reasoning so much better. I also see that I have connected not just the last two blog entries but one from November too!