Retrospectives need a learning culture

I attended a project retrospectives training session last week.  Although I’ve run quite a few retrospectives over the years I’ve never been trained in them.  My guide has been Norm Kerth’s Project Retrospectives book and a whole bunch of ideas I’ve picked up over the years.

This training session was organised and run by Rachel Davies and Diana Larson.  It drew on Diana’s new book Agile Retrospectives. 

Although the book and the training is aimed at Agile practitioners it could be used by other teams – its just that Agile teams are more likely to do this stuff.

The world outside of software development is also catching onto retrospectives.  In part this is thanks to the US Army and Marines who call them After Action Reviews (AAR).  Now they have a macho name and the backing of some generals they seem to be cropping up all over the place.

That said, I still think Retrospectives – or AAR to give them a more fashionable name – are still more talked about then practised.  The training session gave me an opportunity to reflect on this paradox (as well as learning some new techniques and improving my facilitation style.)

So why is it retrospectives get a lot of talk and little action?

There seem to be three basic failure modes.  First is retrospectives that don’t happen.  Usually the reason given is lack of time.  However I think this only an excuse to hide behind.  When people want to do something they do it, if something is important time is made for it.  Conversely when something is low priority or someone doesn’t really want to do it then there is not enough time.  We are all busy people, lack of time really means “is not a priority for me.”

I think one of the reasons people don’t want to do retrospectives is because they are scared of what might come out of it.  Individuals could be criticised, companies could be found wanting or someone may say something they are not supposed to say. 

Retrospectives expose the lie that your boss/manager/leader knows best.  By definition retrospectives are about asking teams what they can do better.  Traditionally we are brought up to expect that someone in authority, someone with a “big brain”, knows best.  So retrospectives can be threatening to managers who feel they are loosing some power.

Which brings us to the second common failure: ineffective results.  Simple formulaic retrospectives, perhaps they simply ask “What do we do right?  What do we do wrong?” and perhaps they limit the whole thing to an hour, and perhaps the team leader or project manager acts as facilitators.

Such retrospectives are unlikely to uncover deep thoughts because they are not asking the questions that will cause people to reflect deeply about the whole project.  Having a authority figure lead the retrospective will also cut off reflection, by leading this person is exercising their authority and sending a clear message “I am still in charge.”

Finally, the third failure mode of retrospectives is: Lack of action.  Things the team decide to do aren’t acted on.  This is actually closely related to the first two.  If it is hard to find the time for a retrospective do you have the time to make the change?  More likely you are risk averse.

And if the authority figure isn’t open themselves then they are not going to be fully behind the change.  This will be clear to anyone tasked with making the change.  That is, if anyone can be persuaded to take on the task.

Arie de Geus pointed out in his book The Living Company that when you have more people involved in making a decision it might take longer to actually make the decision but taking action once the decision is made will be quicker.  This is because more people were involved in making the decision.  If a retrospective is half hearted then any decisions will not have wide buy-in so taking action will be a slow process.

So what’s the answer to these problems?

Well I think I’ve realised were I was making my mistake.  Like many other people I’ve been seeing retrospectives in isolation.  As some kind of plug in solution, nicely self-contained.  But they are not, this was the mistake.

I’ve realised that retrospectives need to be embedded in a learning culture.  I used to think they could be one of the early tools to use in creating such a culture, now I realise they are an advanced tool to be brought in once you have the culture in place. 

An organization that does retrospectives is an advanced learning organization.  Most organisations are not learning organisations let alone advanced ones so it is hardly surprising that retrospectives are missing.

When you have such a culture not only do people value learning but it is a priority for them.  They make time for learning events.  They value the results and they take action.

In such a culture people feel less threatened by what might come out of a retrospective because they are accustomed to deep thinking and reflection.  Everyone knows the rules and people are open to insights and change.

In a learning culture managers have already accepted that the team knows best.  They have already changed the view of their role from command-and-control and “boss knows best”.  They define their role more in terms of helping individuals and teams learn and advance.  Neither are they threatened by loss of authority.

Finally, when you have such a culture, when people are not scared or threatened, when everyone appreciates the value of holding a retrospective then change does happen, people do follow through on the action identified in the retrospective.

So, how do we create such an advanced learning organization?  Well, that will have to wait for another blog entry and even my book.  In the meantime, if I can help with a retrospective or creating a learning culture please get in touch.

SPA London: Making BT Agile

The SPA conference is on this week and I’m not there.  SPA is a good conference but being so close to the ACCU conference I need a very good reason to go.  Since I’m not speaking there I’ve skipped it.

The BCS SPA group also holds monthly meetings which I sometimes go along to.  About three weeks ago I went along to a very interesting talk entitles “Large-Scale Agile Application Development”.  In fact the talk was actually about how BT had adopted Agile development techniques.  This was quite a challenge as those who know BT know that it is a very large company, a traditional telco and at one time a nationalised industry.  There could be few more challenging environments into which to introduce Agile.

BT haven’t completed their Agile transition but it seems to be going well.  Fortunately they had high level management backing on their side and all the kind of resources big companies can bring to bear on these problems: training, off-site boot-camps, consultants (mainly from Exoftware) and other resources.

I was particularly impressed by two of their ideas.

First was a plastic credit card like “pledge card.”  I’ve seen these done by political parties in the past but this one was for “BT Core Agile Delivery Practises.”  On one side was the Agile manifesto and on the other was BT’s core practises:

• Customer collaboration


• User stories


• Iterative development


• Automated testing


• Continuous integration

The card strikes me as good because a) it was a constant reminder of the practises, b) it was very corporate so endorsed the idea that the company wanted this to happen.

The second idea was a pre-printed task card with the main fields a developer would use.  I’ve seen index cards put through a laser printer before but this was more structured.  It contained fields to help describe the work task, the priority and the acceptance criteria.  Again good for re-enforcing the corporate message (“This is what we want”) and a reminder to teams as to what they needed to do.  My only criticism might be that for such a small card it had a lot of fields to complete.

BT seems to have invested a lot in this programme.  They had sent lots of people on training courses and held lots of off-site meetings and training.

They also spent a lot of time and money using coaching techniques.  In fact they used pair-coaching with an experienced coach working with a new coach to help train the coach.  To my mind this approach vindicates the coaching approach and shows how powerful it can be.

That say, it does seem the coaching model in use was very much drawn from the Agile coaching model rather than the model used by business coaches.  The difference is that the Agile model is much more directive about what needs doing while the business model is much more open.  Both have their place, I just wish Agile coaches were more aware of the other model and how it can help.

 

Application development is getting better

As regular readers may have noticed the state of the application development industry has driven me to despair recently (27 January).  But it appears I am wrong.

According to this report of a report in the Software Development Times things are getting.  The report in question – that is the one with the data, not the report of the report (don’t you just love journalists?) – is a new report from the Standish Group.  These are the people who produced the Chaos report in 1994.

You can read the 1994 report for free but the 2007 one is going to cost you, prices seem to start at $500.  I’d love to read the report but I can’t justify $500 right now.

Anyway, Software Development Times already offers the key findings:

• 35% of projects started in 2006 are considered successes – up from 16.2% 12 years ago.


• Project failures are down to 19% in 2006 compared to over 31% in 1994.

(For the moment we will accept the definitions of success and failure.)

And the reason for this change?  Well I’m sure the $500 report gives many, personally my money is on the move to Agile development methods.  The SD Times gives us three:

• Iterative development


• Better Project management


• Web infrastructure

Iterative development fits with my hypothesis.  Better project management, well we’d have to know whether they mean more project managers using Agile like techniques or whether PRINCE2 is finally about to save the world.  No prizes for guessing what I think.

And the web infrastructure?  Well, it might just be that technology can make a difference.

One more finding from the report.

In 1994 the value of software was 25c (US cents) for each $1 spent on the software, this has risen to 59c per dollar.  (Thats if I’m reading the statistic correctly, the sentence containing it is a bit of a mouthful, someone please correct me if I’m wrong.  But lets assume I’ve got it right…)

Think about that: Software is worth less than it cost to develop.

Most of the money spent developing application is still wasted, we would be better off keeping the money.  That is frightening.  But it is completely compatible with something I wrote about in December (17 December).

Companies that understand IT, and in this case application development reap the benefits.  Those that don’t understand don’t get the benefit.  The days when IT could be left to the IT department and everyone else could bury their head in the sand are gone.  If you don’t want to be bothered with IT then just don’t do it, save your money.  If you have to do it then do it properly.

Book Review: The Elegant Solution

I have just finished reading the The Elegant Solution by Matthew E. May.  I think I’m going to have to give up reading serious books to do with software development, business and change until my own book is finished.  Progress has been far quicker than I expected and the end is now in sight.  Problem is when I read a book that comes anywhere close to my subject things get confusing.

So to The Elegant Solution, subtitled “Toyota’s Formula for Mastering Innovation.”  The author mainly considers product innovation although the same things are true about process and practise innovation.  In effect he is talking about what Michael Kennedy calls  “Lean Product Development” or “Knowledge based product development” (Product Development for the Lean Enterprise).  Its about the application of Toyota’s lean thinking to the development of products.

Although May tries to be practical the book is still quite conceptual.  There are big bold ideas like “Learn to see” and “Master the tension.”  While these are backed up with stories (mainly from Toyota but a few other places too) most of the practicalities are left to the reader.  Its a good book for creating aspirations.

I also found that the book overlapped with my own thinking and the draft of my book.  I found myself thinking at one point “He’s said it!  I have nothing original to say!”  Actually I came to realise there is a difference – it is worth me continuing – and that his work just further validates my own thinking.  In particular May’s emphasis on learning is spot on.

I’ll recommend this book, its worth reading but I’m not completely happy with it.  I found it annoying in many places.  For a start is is very American – ironic when it is mainly about a Japanese company.  There are some points were I don’t actually understand what he is saying because he refers to figures from American history, or sporting metaphor's.  Neither do I have car-culture so some of that escaped me too.  As the book went on this annoyed me more and more, this is not a book for the international market.

While the book is short on references it is short in total.  Had the author included more references, expanded on his thinking and skipped the American turn of phrase it probably would have been a lot longer – and I would have complained about that too!

At the start of the book May sets out to define The Elegant Solution (TES).  He gives some examples but I’m not convinced, they can be a little subjective.  In fact I found myself thinking of Christopher Alexander and Quality Without the Name (QWAN – see The Timeless Way of Building) – more recently called Wholeness.

QWAN is the underpinning of Alexander’s Pattern theory.  The more I read May’s work the more similarities I saw between the two.   QWAN and TES are described in similar terms and are, to my mind at least, similarly subjective – although Alexander would claim QWAN is objective and I suspect May would claim the same for TES.

The similarities don’t stop there.  Alexander created Patterns to help create QWAN.  Patterns describe a problem, a set of forces and a solution.  May gives a set of practices describe a problem, a cause and a solution.

I think May is very close to Alexander and Patterns.  I don’t know if he has read any Alexander or has made the connection himself.  I suspect not but his references section is sadly lacking – he has one but its not a complete list of references or influential work which is a shame.

As a book The Elegant Solution also reminds me of Tom Peter's book Re-imagine! Both try to enthuse people to take a grip on the world themselves and step beyond the normal bounds.  However there is one big difference.  May’s book is all about incremental change and improvement.  But Peter’s rejects incremental change, he wants radical change. He sees small improvements (the type typically brought about by Lean) as preventing radical change.  Why change a little when you can change a lot?

This is a big divide.  Personally I come down on the incremental change side of the argument.  But – and this is a big but – incremental change has to be combined with a constant desire to do better and a systems view of the world so you are not sub-optimising.  This is an argument I’ve been making for a while, I probably picked it up from Peter Senge’s The Fifth Discipline, and its the same argument May makes.

Finally, I’m getting a little fed up of reading about Toyota.  I know, this is partly my own fault but I want to know were the other example's of Lean are?  I know a few, like Dell and Heathrow Terminal 5 but surely there are more still?  And when will thinking advance beyond Toyota and Lean?  What is next?

Perhaps what makes Toyota different is not so much Lean, or learning, or knowledge but simply a willingness to deal with problems and to grasp opportunities.  Maybe the secret is just that they do things.

Product Management in the UK

I went to a meeting of the UK Product Managers Forum last night.  This was the first time I have been to a meeting of this group and it is the first time that the group has held an event in London.  Naturally these two firsts are not unconnected!

I enjoyed the meeting, Ian Mapp of Respond spoke about the development of the product management function within the company.  This was very interesting itself, and as is so often the case the conversations with the other attendees was even more interesting.

These conversations support my belief that Product Management is not a well established role in the UK software industry.  When the product management doesn’t exist in a company there is a vacuum.  Without any guidance software developers will just develop the things they think are a good idea.  If there is a compelling need for your product this isn’t a problem in the short run: people need your product, good or bad they will buy it.

Nature abhors a vacuum and the same is true when product management is missing.  Somehow the vacuum will get filled.  In small companies it is often the founder(s), the CEO, CTO or similar who fulfils the role.  They often do it on gut feeling. If they are lucky they are right and the companies grow but then they don’t have the time to do it.

Sooner or later someone steps into the role.  It might be a project manager, a developer, an architect or even a tester.  These people do their best but they have other demands on their time – the job they should be doing.  There is competition not just for their time but their prioritisation decisions will be influenced by their duel roles.  Neither have they been trained for the role.

A lot of software development does wrong right here.  Without good product management to give customers a voice inside the organization companies will spend more time developing more features to sell fewer products.

(Product management can exist under the wrong name, I’ve met Architects who are really Product Managers and I’ve seen Project Manager roles advertised which are really Product Management jobs.  This just confuses things!)

The second thing I noticed at the meeting was how tight the profession is.  Most people knew of the Silicon Valley Product Group, 280 Group and Pragmatic Marketing.  Those of us who have ad product management training had had it through one of these groups in the US.  Again this is a sign of how underdeveloped the role is in the UK.  I believe this is about to change, Co-herence have licensed courses from the US and plan to deliver them in London.

The UK is not short of software companies.  It is short of product managers and product management skills.  Fixing the development process must involve product management.

If you are developing software application there are three things you must do:

1. Create a product management role (business analysis if you are in house)


2. Use source code control


3. Have a repeatable build process and run it at least once a night

If you aren’t doing these things then don’t bother leaving home, in fact, don’t even bother getting out of bed.  The scary thing is: most companies struggle to score two out of three on this list. 

(If I just scared you and you want help e-mail me, more on this next week.)

As an aside, Respond’s product is a customer feedback (i.e. complaints) management system.  Ian suggested that in some industries, notably airlines and mobile telecoms, the company got very little traction.  His conclusion was that for these companies customer service is not a high priority.  I can see his logic, if the companies in these sectors are not willing to spend money then it is reasonable to conclude that the companies do not see customer service as a priority.  One of the audience, form a major UK mobile phone operator took exception to this.  They claimed their company did take the issue of customer service and retention (churn) seriously. 

I’m in the process of reviewing my mobile telephone.  I’ve been with the same provider for 5 years now but I don’t feel like a valued customer.  The company has a memory like a goldfish – i.e. none – I might as well have been with them a year or a month.  I’d like to switch to another provider but a) I’m not convinced anyone else is better, b) it looks like my provider is actually the cheapest!

Recruitment and customer experience

Wednesday’s Financial Times carried the monthly technology review.  A couple of pieces caught my eye, both individually and when put together.

 

First this report In an e-world, IT controls the customer experience (subscription required) makes the point that if your business is conducted online then your customer experience is an electronic customer experience which in turn means the customer experience is decided by the IT department.  This is a point I’ve made myself in the past – although I’ve never came up with such a compact phase to describe it. 

 

In posts in January 2006 and December 2006 I sounded off against poor travel company web sites.  My customer experience is poor because the web sites are poor.  It doesn’t have to be this way but unless the IT department think about the customer experience, or the people responsible for the product involve themselves with IT this is what happens.  Unfortunately IT departments have spent years refining their image as uncooperative.

 

Given the increased use of e-commerce by all businesses - not just those like Amazon which only exist on the web but everyday companies like the travel companies I dislike – then any business that does any e-retailing has this issue.  The online brand can damage the off-line brand.

 

(Apologies if you follow back to those posts, it seems the line breaks have gone absent as I described a couple of weeks ago.)

 

The second piece that caught my attention was this one Can HR handle IT recruitment?  According to this article human resource departments do not have the skills and experience required to hire IT people.  The story rings true from my own experiences and from many anecdotes I’ve heard over the years. 

 

Although these stories all ring true I’ve also known several good HR people.  People who have their heads screwed on and want the best for the business.  Plus I’ve read a fair few HR text books and articles and I know HR people aren’t really as bad as IT people make out.  So what’s going on?

 

Well I have some possible explanations. 

 

Theory 1: Many companies have bad HR departments.  There are some good HR departments out there but possibly most of them are not worth the salaries we pay them.

 

Theory 2: Everyone has these problems with HR, it isn’t just IT.  HR is just badly understood by the rest of the business.

 

Notice that these explanations are not incompatible.  Neither are they incompatible with the FT article.  Which raises the questions: What are HR actually doing?  Why aren’t HR departments getting to grips with IT and trying to rectify the problem?  After all, strictly speaking, it is the HR departments job to hire people so it is they who have the problem.  They can’t blame IT for this problem.

 

Put all these theories together and you get: HR is important, HR doesn’t understand IT, many HR departments are poor, many HR departments fail to hire the right people.  As a result many IT companies do not hire the best staff.  And because good IT depends on good people few companies will have good IT.

 

Proof for this theory comes from Google and Microsoft.  Both of these companies have built hiring machine.  They both need a lot of IT staff and they have made sure HR can do the job properly.

 

So the lesson is: If you want good IT you start by fixing your HR department.

 

Finally, put those two articles together.

 

IT controls the customer experience but HR do not know how to hire the people IT needs.  In other words: HR departments do not know how to hire people who create the customer experience.  Am I the only one who sees this as a serious problem here?

 

Lean Question and Answer

A student at Fullerton University in California e-mailed a lot of people on the Lean Software Development mailing list with some questions for his masters thesis.  Since I had a little time on my hands I wrote him some lengthy answers.  Having done that I though I would share them here.  I’ve removed a few questions because I don’t think the question or answer are particularly interesting.

1.      How would you define lean development?

The strictest definition of Lean Development would be to say it is a methodology proposed by the Poppendiecks (Lean Software Development)based on lean manufacturing and product development.  This definition is useful for deciding who is doing lean and who is not but is probably too strict.

Another definition of Lean (Software) Development is that it is the application of the principles of Lean Manufacturing ( The Machine That Changed the World, Womack, Jones & Roo ) and "Knowledge Based Product Development" (Product Development for the Lean Enterprise, Kennedy).

Personally my definition is:

• Lean Software Development is one application of the principles of Organizational Learning to the software development environment. 


• Learning Organizations may come up with a different development way of thinking but I am not aware of any others.


• Agile Software Development is a further refinement from principles of Lean to practices.


• The various Agile development methodologies are further refinements of these with further practises.


• Underlying everything is the idea of a Learning Organization (Senge, Argyris, etc.)

So you can create a pyramid, more general at the bottom, more specific at the top: 

- XP -	->	Specific practices
-- Agile --	->	General practices
---- Lean ----	->	Ways of thinking
Learning Organization	->	Concepts

To put it another way:

• XP (or SCRUM, or DSDM, etc) are a specific set of practices for doing Agile software development


• Agile software development is set of principles and a specific set of lean practices


• Lean is a set of principles and a specific set of learning practices


• Organizational Learning is a set of principles to which some authors have added some practices

Note that you can, in theory, adopt an Agile methodology with very little learning. You could simply adopt, say, SCRUM practices and through focus and authority make people use these.  You probably would see some benefits however the team would not be learning and future change would not come naturally.

I examined Software Development as Organizational Learning for my own Masters dissertation

2.      What percentage of developers do you think are using Lean Development approach?

Very difficult to say, I think some ( less than 10%) will have picked a few practices, I think few will be working in a Poppendieck or Kennedy like way -  maybe less than 1% on this definition.

On the other hand, I think many are undertaking some organizational learning and using some agile practises but progress is very slow.

3.      How has the usage increased from previous years?

Slowly.

4.      Why do you think the usage has increased?

Various:

• The term is now commonplace thanks to Poppendiecks and others.


• Lean (and Agile) tell a very good story, developers want to work like this.  Managers see the parallels with lean manufacturing so also hear a good story.  It is an aspiration change.


• Natural selection: those who "get it" will out compete competitors.

5.      How do you expect the usage to be in the next five years?

Slowly.  I see an Agile backlash building, I hope lean will not be caught in it too but I think it will.

6.      Have you found resistance or pessimistic thinking towards Lean usage?

Yes - "Idealistic", "Will not work here"

7.      What kind of unintended results have you seen by using Lean?

Lean relies on exposing problems so they can be fixed.  (Removing "camouflage" to use Argyris term and rolling back what Senge would call "shifting the burden.")

As such it looks like things are getting worse to start with.  Problems that were hidden are now apparent, problems that could be ignored before need fixing now.  Therefore, Lean looks like it is making things worse before things get better.

When developers or junior managers get it but senior managers don't there is a mismatch.  Junior managers pay a heavy emotional toll and sometimes pay with their jobs.

8.      What suggestions do you have for a shop considering Lean?

• Emphasis the learning. 


• Take it from both ends: senior managers down and shop floor up.


• Get help, specifically coaching.

10.     Can Lean development be applied to non-Agile (i.e. waterfall) methodologies?

You could make a waterfall leaner but I doubt we would ever call it lean.
In suggesting a lean waterfall you are saying: "We want to improve X but we don't want to change Y".  You have put boundaries on the problem.  In doing so you are putting some things out of bounds.  This means you can't engage in Systems Thinking properly and are likely to come up with sub-optimal processes.

11.     How difficult is for a team become Agile and Lean after being non-Agile?

Very difficult.  You can start in a small way but you need to break with the past.

13.     What are the key indicators to tell if a team will succeed in becoming Lean?

The presence of a learning culture:

• Is the team holding retrospectives?


• Is the team changing their process?


• Is the team improving?

14.     Is it better to be Lean and then Agile or Agile and then Lean?

Given my definition above this question does not make sense.  There is no temporal relationship.

If you are Agile then you are (to some degree) Lean.  If you are Lean then you are Agile.

15.     What did I forget to ask you about lean?  Is there something about lean that you consider very important that we did not discuss?

The "change question".  We know how to be Lean and Agile but how so you change the organization?

Then, when you have become Lean how do you carry on changing?

Organizational change is more important than the practices.  The practices support learning and change but change is not a rational thing.  You don't do a value stream map and solve everything. 

ACCU London - 22 March

I'm the host for the next ACCU London meeting. This is being held at Waterstones book store Piccadilly on Thursday 22 March, 6.30pm for 7pm start.

ACCU London is a small initiative by several members of the ACCU - OK, mainly Paul Grenyer and myself - to establish a London chapter of the ACCU along the lines of the regular ACCU Silicon Valley meetings. So far we have been pretty successful. We have two formal meetings and several pub meetings behind us.

Next weeks meeting is something of an experiment because a) we are in a bookshop, b) we are tackling a non-technical subject.

Our speaker will be Stefano Cicu of The Wreay Partnership recruitment consultants. Stafano will be speaking about the role of the recruitment agent and how agencies work.

Since finding work is a regular topic on the ACCU General mailing list we think this should make for an interesting topic. By understanding how recruitment consultants operate we can all benefit.

If you are interested in attending please let me know, we need at least approximate numbers for attendees.

Venue: Waterstones, 203-206 Piccadilly, London, W1J 9LE

Does (system) documentation work?

OK, I promise this is the last entry on the Nurnberg funnel, after this I’ve worked it through my system…

In my entry on Nurnberg funnel and system development I wondered is system documentation actually works.  Well I missed one piece of evidence that it does work.  Go into your local bookstore, Waterstones, Borders, even Amazon online and look at the books on computing.  Specifically look at the technical books that describe things like programming languages, operating systems, APIs and the like.  If system documentation does not work how do we explain the existence of all these books?

After all, publishers are in the business of making money.  If these books did not work - i.e. did not help learning and transfer knowledge - then presumably nobody would buy them, there would be no profit in publishing them and consequently they wouldn't be taking up real-estate space in the book store.

So how do we explain these books if technical documentation doesn't work?

I don't have any figures on what is happening but here is my guess.

Firstly a whole load of this documentation does not work.  Many books get bought despite being bad.  Many more books are bad and simply don't get bought, publishers swallow the loss and pulp the books.  I would guess this category covers many many books.  Lets take a stab and say 40% of all technical books.

A second category of books do work and they work for exactly the reasons Carroll outlined in the Nurnberg funnel.  They are written along the lines of minimalism.  My guess it that a lot of those "For Dummies" and similar books fall into this category.  Still, this type of book represents a small percentage of all books.  Lets say 5% - probably an overstatement but it keeps the numbers easy.

OK, I still have 55% to account for - and I'm deliberately not saying if these are percentage of books sold or books published.

Next up I think there are some books which do not directly take on board the minimalist ideas but have a similar approach and work for similar reasons.  I'm specifically thinking here of patterns books like Design patterns and Pattern Languages of Program Design 5 (to which I was a contributor).

In part patterns are popular and accessible because they start with a problem and describe a solution.  This is a little like the task orientation that Carroll advocates.  I'm sure patterns books are not alone here but there aren't very many books in this category so lets just say another 5%.

This also starts to hint at another explanation and one which must account for many of the remaining 50% of books.  Simply minimalism is but one technique for creating documentation, it works particularly well in one field - user manuals - but it is not the only one.

For example I know of one other method, story telling.  This is usually advocated as a management or knowledge management activity to be conducted verbally.  (See books like The Springboard and Storytelling in Organizations). However there are technical books out there that take the same track. 

For example, I found the original version of Inside Windows NT quite story like.  Although perhaps this is a sign of how techno-geeky I was in the mid-late 1990's.  Somehow the second edition of the book never had the same plot or interest for me.

Less technical books about technical subjects are better examples of story telling.  Take The Soul of a New Machine for example.  This is a must read, I left it far too long before I read it but its right up there with The Mythical Man Month, Peopleware and The Psychology of Computer Programming if you want to know how software (and hardware) really get created.  But Soul of a New Machine is also a story, it can even be bedtime reading.

Books like these are more the exception than the rule so they are few and far between.  Lets say 1% are story books.

Another category is reference books like Charles Petzold’s Programming Windows.  I defy anyone to actually read this books cover to cover, you delve into when you have a specific problem or need to know some specific detail.  But reference books probably constitute a lot of the books written, say 20%.

That takes me to 71%.  I still can't account for 29% of books.  I give up.

The important thing is that people read books for different reasons.  Our motivation for reading one of these books will be different to the motivation for reading another.  That will change the way we approach buying and reading books. 

Which in a strange way brings us back to minimalism.  People use Petzold's books when they have a task to accomplish.  People pick them up under certain circumstances and like a Dummies book expect to get something specific from them.  But that doesn't mean all books should be written in the minimalist style.  Minimalism is just one of many styles.

So actually, we've come back to one of the re-occurring themes of this blog: know your customers, service their needs with a product that is designed to meet those needs. 

Products development doesn't end when the code is written, compiled and burnt onto a CD.  It has to include the documentation, service and support that is supplied with the products.  It also means ensuring that future product development can continue, whether through just-in-case documentation or just-in-time networks of people who know. 

Unfortunately obvious solutions – like big thick manuals - are not necessarily the best solutions.  We need to be on the look out for new solutions and new ideas.

And we need to understand the costs and benefits of all this.  This is difficult bit because so often we don't know the numbers and we can only speculate on what they are.  But somehow we need to put all of that together.

More on the Nurnberg funnel

Hans Wegener e-mailed me with some interesting footnotes on the Nurnberg funnel.  He pointed out that it is not enough to consider risk alone when deciding what documentation to write, we also need to consider cost. He is right of course, the question is: how do we establish the cost of writing documentation?

If we are going to talk about cost we should also talk about benefit.  As well as asking how much does documentation cost we need to ask: what are the benefits from the documentation?

In my experience written system documentation tends to go one of two ways.  Either the project considers it absolutely essential.  In this case documentation tends to get written even if it means sacrificing something else - like actual working code.

Alternatively projects start with lofty ambitions to product documentation but this tends to be pushed to the end of the project and sacrificed when deadlines approach.  In this aspect documentation is similar to testing: everyone agrees it is a good thing and we should do more off it but among the first sacrifices when problems set in.

In both cases the benefit of documentation is not understood.  What we need is a better way of judging the benefit of documentation.  If we could do that then we could determine how much to spend on it, whether to write lots of it or whether to sacrifice it.

My problem is that I have not data on this.  For all my comments and opinions it is all just speculation based on personal observation.  Trouble is this is the case for most other people too.  We all guess at how much system documentation to write, how much time to spend on it, and how many people to employ doing it.

Blip on the blog

Anyone reading this blog through RSS or an aggregrator may notice a blip as my entries on the Nurnberg funnel appear fresh when they are not.  There is another (last?) entry coming on this subject but not just yet.

Thing is… as I noted a few weeks ago I’ve moved back to using BlogJet.  There was a problem some of you may have noticed with formatting: extra line breaks. 

I think the cause was actually some interplay between the way BlogJet works and New Blogger.

Well I worked out how to fix this problem (blogger config option) but it meant that some other entries lost their line breaks.  Specifically the three entries on the Nurnberg funnel which were written outside of BlogJet. 

Anyway, it meant that these entries looked odd so I’ve had to fix them by hand – there is dedication for you!

Two points to note.

First: I’ve now payed my $19 to officially upgrade to BlogJet.  It still makes blogging much easier and I’m glad I have it.

Second: the Blogger upgrade has been nothing but pain for me.  I don’t see the benefits of upgrade and its has caused me a host of little problems – mainly because of the BlogJet interface. 

Given that Blogger is a software service owned by Google I wonder what this says about Google’s (or just Blogger’s?) approach to compatibility and upgrades.  The process has not been smooth, hopefully Google will learn and things will be better next time. Alternatively they may just stamp their foot and say it was all BlogJet’s fault – in which case it says something about their attitude to their ecosystem.

Hopefully normal service will now be resumed.

Bits and pieces

Back from a week skiing – French Alps if your interested.  Seemed to have half a dozen different types of weather in one week: snow, rain, ice, sun, slush. 

Skiing is the perfect holiday for English people because of the weather.  There is no perfect weather for skiing: no snow is bad, too much snow is bad, wet snow is bad, sun is good, until it melts the snow when it creates slush which is bad, so too warm is bad but so is too cold because not only do you get physically cold but it also creates ice. 

Conclusion: an Englishman on a skiing holiday always has something to complain about and to talk about.

The other thing to talk about is flying.  As much as I like flying and airplanes (the kid in me) I know something is seriously wrong with the system.  Not only does it do serious damage to the environment but our airports are a mess (queue here, queue there, queue again.)  There is something seriously wrong with the world when we have to spend so much time queueing to get ourselves cleared through security.

Nowhere is this absurdity clearer than with liquids and water.  Security at Gatwick airport took my bottle of water off my so I had to spend £1 to buy another.  On the return Grenoble took my water off me forcing me to buy a another for €2.  Airlines never give you enough water once you are on-board but tell you to drink more water.

At the risk at getting myself banned from airplanes I’ll point out what I see as the big whole in airport security: The Duty Free shop.  What is to  stop anyone buying a nice bottle of wine, whisky or perfume, boarding the plane with the bottle.  Once airborne smash the bottle, you now have a handy weapon to threaten people with.

When I’m going through these check points I always think of the Bruce Schneier piece on the danger of liquids on planes.  (That isn’t sarcasm thats just me self-censoring so I don’t get caught on filters, paranoid?  Me?)  – And a note to myself: read more Schneier.

Anyway hardly had I left home than more thoughts on the Nurnberg funnel came to me.  Plus I got home to some mail on the subject to that will be the subject of the next blog entry.

Finally, during the holiday we thought of lots of people we haven’t seen for a while and should catch up with.  I know many of readers of this blog so if I don’t call you for a please call me.  Though don’t be put out if I don’t call, I have a lot of people to catchup with and some of them will get to me first!