Tuft, Conway and PowerPoint

People I know who know about graphics and data visualisation tell me Edward Tufte is The Authority. I also hear he is a great presenter and his tutorials worth attending - if you into graphics and visualisation that is. So when I see his name on something I pay attention.

As regular readers know I have a thing about Conway’s Law. So when Tufte’s name came up in connection with Conway’s Law on a recent search I paid attention. Tufte’s point is actually about the inadequacies of PowerPoint.

I agree with him. I really dislike PowerPoint and wish I could give it - and similar tools - up. I’ve blogged about this before. I should report I have failed to give it up. I have two up coming presentations - BCS Project Management and Oredev - and I’ve prepared PowerPoint decks for both.

Why?
• Because they are a crutch to lean on, parts of me is scared without it
• Because the event organizers expect it
• Because I want the attendees to have something to take home
• Because it helps to plug my book (nice big picture!)

The comments and links on Tuft’s piece are even more interesting. Seems NASA are concerned about engineering by PowerPoint and the US military also have PowerPoint trouble.

All the more reason to redouble my efforts and kick the habit.

New writing, new patterns online

Two new pieces on my website for anyone who’s interested.

I’ve kicked off a new series of articles in the pages of ACCU Overload this month entitled On Management. The first of these pieces, Triangle of Constraints is now on my website. Alternatively it is available in the Overload August download.

After post conference editing my patterns from EuroPLoP 2008 are now online. These four patterns continue my series of business strategy design patterns for software companies. They are: Single Product Company, Whole Product, Product Portfolio and Product Roadmap.

I deliberately entitle these patterns “for software companies” although many people are quick to point out these patterns are applicable to many technology companies and to many companies in general. While I recognise this I stick to the “for software companies” formula for two reasons. First I write from experience, I won’t want to step too far outside my experience zone. Second to do proper justice to the wider context I would need to write longer patterns. I wont to keep these patterns short and readable - I aim for two pages and most of them end up at 4 pages!

Actually there is a third reason: a challenge to the reader. I leave some blanks for the reader to fill in, so the reader can make the pattern their own. I drop hints - some of the pictures and examples deliberately come from other industries - but leave it to the reader to bridge the gap.

The Open Source Software Myth

Once in a while I get asked my opinion on Open Source Software projects. Personally I think Open Source software (OSS) is great: Apache, GNU C++, Linux to name a few. But as a project management or product development technique I can’t say I recommend it. There are two reasons I give for this.

First there is no single OSS development model: Apache has a stack of IBM cash, GNU C++ is largely volunteers with chip makers (and others) contributing expertise and resources to help their own products, Mozilla was seeded by Netscape and has done a good job of marketing itself, etc. etc. Some OSS projects have paid development teams, some have single individuals and so on. It is not enough to say “We will do it open source” you need to be more specific.

Second: most OSS projects fail. “Go look at the number of web browsers on Source Forge” I say - a quick search this morning shows 19,362. There are lots and lots of OSS project that go no where. Commercial projects (famously) fail too but the important point is: OSS is no silver bullet.

So with this in mind I was interested to find and read a paper by Kieran Healy and Alex Schussman from the University of Arizona entitled: The Ecology of Open-Source Software Development. Although from 2003 I think the analysis and arguments probably still stand. (This paper appears in several places on the web so I guess it is already well know, maybe I’m late to the party.)

The authors did some analysis on projects in Source Forge and came up with some interesting results. Basically it turns out that when most people talk about the Open Source model, and successful examples, they are talking about the top 1% or less of all Open Source projects. The true OSS model has a long tail.

Taken as a whole OSS projects follow the Power Law - its that law again! The authors say:

“The median number of developers is one. The 95th percentile is 5. Relative to the whole field, only a tiny number of projects have more than a handful of developers. The median number of cvs commits is zero. At the 75th percentile it is 1 and at the 90th percentile it is less than 100. This indicates that there is little or no programming activity taking place on more than half of the projects. Examining the number of message authors across all forums shows that only projects at the 90th percentile and above have more than two contributing message writers.”

And later:

“In summary, we found power-law type distributions for all activity measures in our dataset. In each case, a tiny number of projects dominate an activity-type when measured by volume.”

The projects which are actively developed (in this study these include Crystal Space 3D, Open CASCASE, gkernal) are not the most downloaded (in this study CDex, VirtuakDubm ZSNES). In fact the authors say:

“Measures of user interest in a project — such as site views and downloads — are not closely related to measures of developer activity on a project.”

So what are we to make of these findings?

Well I think it bears out my initial comments. OSS is no silver bullet.

The papers authors propose three hypothesis for further consideration - and given that the paper was five years ago they may well have developed these further.

First they suggest that the involvement of professional developers is key. Despite the gifted ametuer image of some projects it helps to have professionals along. Second the role of project leader is critical - nothing new here perhaps but more evidence that OSS alone is not enough. (Actually, I was reminded of Fred Brook’s Chief Programmer model while I was reading this.)

And third, most interesting to me: the authors suggest that successful OSS projects with have hierarchical structure between leader and development team. Many OSS proponents (e.g. Eric Raymond) suggest a more chaotic structure in OSS projects which may not be the case (remember Healy and Schussman are suggesting a theory here without proof).

Personally all three suggestions ring true for me. There are wider lessons here too for the Agile and business communitiies.

There are those in the Agile community who see it as a route to emulate the success of OSS. However this study shows that the OSS success if not good enough for commercial projects.

Second some elements of leadership and hierarchy are necessary to have a successful project. In other words, relying exclusively on self-organising teams and emergent behaviour may not be enough. Some element of control is justified.

In the business community there has been a lot of talk in the last few years of applying Open Innovation - that is applying Open Source ideas to business problems and products. Proctor and Gamble have one example, Walkers Crisps in the UK is trying an Open Source type idea at the moment. If the OSS experience is anything to go by open innovation may come up with some good ideas but it is not enough alone.

Talk to the BCS Project Management group

I’ve been invited to speak to the BCS Project Management Group - PROMS-G - subject is “Why and How to become Agile”. Its rather short notice this one, 9 September in Central London. I believe all are welcome but advance booking is necessary.

Value Stream Accounting for Lean

Just to finish off my Financial problems with Lean blog post about the financial problems with Lean.

I read the second half of How to Manage Through Worse Before Better the in the MIT Sloan Management Review and while it was interesting it didn’t give my any more great insights. Basically the second half answers the problem described in the first half.

And the answer is.... Value Stream Accounting. Yes it makes sense, yes it is good, and yes its accounting. Thus I found it a little bit boring and I’m not going to describe it here. If you want to know the details read it yourself. Sorry to be mean, I’m busy at the moment so things have to get my interest to get prioritised.

Financial problems with Lean

From time to time the dark side of Lean surfaces. On the whole a lot of this dark side is bunk (rubbish), Lean gets the blame for something that isn’t really anything to do with Lean. Though Lean is not without its problems most of the well publicised cases (like karoshi, death by overwork) seems to have more to do with (Japanese) society than Lean itself.

But it hadn’t occurred to me until today that there were also financial problems with Lean, or to be specific, the transition to Lean. The summer issue of the MIT Sloan Management Review has a piece about financial problems (How to Manage Through Worse Before Better) which can afflict companies transitioning to Lean.

Although this piece is about manufacturing companies not software companies I think its worth discussing the reasons and how they might also effect software companies:

• Lean companies hold little or no inventory, so a company transitioning to Lean will reduce the amount of completed work held in stock (not to forget raw materials and work in progress). But this stock is also shown as an asset on the balance sheet. So reducing inventory reduces the value of the company.

• It is not only the transition company which hold stocks, so too do customers. As the company becomes more Lean the lead time for orders will reduce so customer no longer need to order so far in advance and hold buffer stocks. They will adjust their orders, rather than ordering 6 weeks in advance they may order 1 week in advance. Therefore sales will appear to reduce as customer change their schedule.

• Although the Lean will make the company more productive in the short run it is difficult for the company to realise this benefit. It is not possible to reduce the work force during the transition because managers and workers need to co-operate so managers can’t fire workers (well you could, but imagine what it would do to the rest of your change programme).

• Neither can the extra productive capacity be used for new manufacturing because the company is still in transition and it takes time to introduce new products to manufacturing and optimise the system along Lean lines.

So what about software companies? Actually I think they do, perhaps to a lesser degree:

• Software companies don’t hold stock, but work in progress could appear on the balance sheet, it all depends how the company accounts for R&D so this could be an issue.

• Customers might delay orders once they know they can, this depends on your sales model.

• Software companies face the same problem in laying off surplus workers, if anything the problem is worse. Even though most software companies I’ve ever seen don’t have enough people to do the work they are asked to it these benefits won’t show on the balance sheet or cash flow statement.

• Finding work for newly available resources is also going to be a problem. I’ve blogged before about the importance of Product Managers and the idea that The Bottleneck has moved - if you have developers who are more productive you need more Product Managers - so costs will go up!

Actually things might be worse in software development because of the quality issue. A Swiss friend of mine claims that his software company was forced out of business when they adopted TDD and improved quality. It seems many of their customers were buying new versions of the software, and paying support fees, to get bug fixes. Improve the quality, reduce the bugs and why do they need to spend more money?

My reply is always: its not much of a business model if you rely on your customers paying for fixes. But the point here is that during the transition phase you loose the revenue before the benefits kick in.

I’ve not read the second half of the Sloan piece yet so I don’t know how to resolve these issues but I think this analysis is helpful in understanding why companies find it hard to transition.

Agile control and management

I was expecting slightly more response to my “Why managers don’t get Agile” post last week because this is an issue that comes up again and again and again. Hans Wegener (of metadata fame) did respond and for some unknown reason his comment ended up on the wrong blog post (May’s post on making strategy) - his comment is here.

I certainly didn’t intend to come across as anti-control. True, I do believe that less control and more self-organization can improve results but I know that some control is necessary. The point i was trying to make is: Agile methods do not discuss the role of the manager much, and they appear to loosen control on the development effort. Therefore it is hardly surprising that managers don’t “get” Agile.

Actually I believe that Agile development improves control in an organization for several reasons.

First the illusion of control is exposed. Estimates, GANTT charts and “management by exception” never really controlled development work, they just presented an illusion of control. So much upfront planning was just planning theatre, it never really changed much, development work carried on much the same, work took as long as it took whether the estimate was 1 day or 1 month. Agile strips away many of these illusions - that’s why it is painful at first.

Secondly Agile improves control because work can be stopped at any time and still produce benefits. Stop a traditional project half way through and you have a lot of code which implements some features but is so full of bugs that it is usually unusable. Stop an Agile project half way through and you may have fewer features but the features you have are usuable, there is no need for a test-fix-test cycle.

The removal of the test-fix-test cycle also improves management control. Traditionally this cycle takes up an unpredictable amount of time at the end of the project. Because it is unpredictable control is lost.

Third - and final for the moment - because Agile teams are focused on delivering what the business wants they don’t develop anything else and do deliver what was asked for. Again more effective control.

However this third point does put a responsibility on the customer/business side of things to be able to articulate what it actually wants and to take a part in creation and verification. Again this might look like a loss of control but it isn’t.

For example, a traditionally request might be “Add a widget to the website” and at some unknown date in the future the widget would be added - the colour, size, shape and functionality of the widget might not be what was expected but was there. Such a request given to an Agile team is likely to result in a whole bunch of questions: “What shape do you want it? Colour? When by?” etc. etc. This might look like a loss of control but in fact the team is giving more control to the requester.

One of the things I always tell clients is that Agile exposes problems, and in this case a problem with the management of IT work has been exposed.

Hans ends his post with the comment: “I can't prove this, but I have a theory: developers don't "get" management” and I have to agree with him. Superficially management is like coding, its about organizing stuff, connecting things and putting things in order. However it is very different: the fundamental building block of management is not lines of code but people. Mangers work in ambiguous environments with a lack of information and few ways of testing a hypothesis.

I’m also convinced that many (even most) of the problems software development faces are the result of poor management not technology. Developing software may look easy but it is hard, not only do you need good coders and testers but you need good managers. Being a good coder does not make you a good manager and being a good manager does not make you a good development manager.

Using patterns to capture and communicate knowledge

I have a short piece entitled “Using patterns to capture and communicate knowledge” in the August issue of Knowledge Board. One small contribution to getting patterns used in a wider range of fields.