People or the system?

“the two view-points are always tenable. The one, how can you improve human nature until you have changed the system? The other, what is the use of changing the system before you have improved human nature?”

George Orwell, “Charles Dickens” essay in Shooting and Elephant and Other Essays, Penguin Books

I am sure I am not alone in exhibiting another of Orwellian trait: Double think.

For several years I have been guilty of agreeing with two contradictory points of view:

• The Deming idea that the system governs how people will perform and act
• That no matter what system (e.g. Scrum, XP, Prince2, DSDM, etc.) you use to develop software the motivation and quality of the people is the overriding factor

It wasn’t until John Harvey tweeted about the latter recently that I realised In was double thinking, my positions were not compatible with one another.

I should have noticed the cognitive dissonance sooner. I’ve been known to say, and still believe: “The true test of any software development method (e.g. Kanban, Scrum, etc. etc.) is whether it can help mediocre people perform, i.e. deliver, be more productive”

In that one statement I epitomise the contradiction. I’m hoping that writing this blog will allow me to resolve this!

This isn’t an academic problem: depending on which side of the argument you come down on will determine how you go about improving things. It also explains why some people in the Agile moment emphasis: trust, communication and other inter-personal issues while others emphasis a method, technical practices, etc.

I’m thinking of my experiences:

• I have worked in environments where the system defeated the people: Railtrack privitisation, Sema had an ISO-9000 waterfall process and 120 people. Of course in 120 people some were good and some not so good.
• I have worked in places where (good) people have overcome the system: two years after leaving Railtrack I returned, a much smaller team were in maintenance mode. The same process was in place but we circumvented it, we faked the documentation and so on.
• And I have worked places where everyone was treading water and complying enough with the system

I have seen different things happen within the same organisation, even within what the company regarded as “one team.” I saw Reuters destroy its own processes in an effort to impose a CMMI level 2 (going to 3) process. (Managers were a bit like the American General in Vietnam who supposedly said “we have to destroy this village to save it”. Like the American’s Reuters ultimately lost.)

I have worked places were the concept of “system” would have been laughed it, things might be more accurately characterised as “Chaos.” But even in this chaos there was a system of sorts.

At Dodge Group it was hard to define any system or process, but there was somewhere a process. The company was trying to sell, sell, sell so they could IPO and the software team just had to keep up as best they could.

The good people at Dodge won out for a while. I instigated a simple process we could now recognise as a form of Agile but ultimately the company, the system, killed the success. Partly by hiring bad people, partly by not recognising the good people and partly by letting the system run its course.

In my “Agile will never work in investment banks” posting a while back I made a similar point: Good people in banks can install a good process which can deliver. But in time the bank - the systems, culture and contradictions - will kill the system and dispose of the good people.

Then there is the question of whether “good people” are themselves the result of the system. Can you pick up a “good person” from company A, put them into company B and let them do their good stuff?

An article in the MIT Sloan Management Review a few years back suggested that when “star players” move to a new team they don’t necessarily, or even normally, keep their “star player” performance. (When ‘Stars’ Migrate, Do They Still Perform Like Stars?, Fall 2008 issue.)

So where does all this lead us?

I think there is truth in both the System-first and People-first point of views. Things are not so black-and-white. Sometimes the System-first view is right and sometimes the People-first is right. There are forces at work in different contexts.

Some of these forces are:

1) The strength of the system: the stronger the system the more likely it is to win out.

The strength of the system derives form multiple sources including: the effectiveness of the system, the degree to which it is enforced and the degree to which people see their interests being served by the system.

Reuters CMMI initiative also put a lot of effort into policing. At Dodge the system was weak day-to-day, developers got on with it and they made good stuff.

At Sema Railtrack (first time) a lot of effort was put into policing the system, the system delivered ISO-9000 which helped the company win contracts. As a programmer woking in the system it didn’t seem particularly effective but I guess it was effective at generating money.

2) The System affects the Good People

At Reuters the system encouraged people to leave, I was one. At Dodge the system eventually recruited the wrong people, forced others out and wore down the good people who submitted.

3) Good people can make an poor system work

Primarily through ignoring it, compensating or changing it. Which leads to…

4) Good people can change the system

I’ve seen it happen. In fact you need at least one good person to cause they system to change or it will not.

Again a strong system is harder to change.

5) How “good” are the people?

Yes “good people” can do things change thing but how “good” do you need to be? Is it real a question of having super-stars? Most people aren’t super-stars. Surely we should be more concerned with the mediocre.

And before anyone rushes to say “Good developers are 10 times (or 28 times) better than average” please read Laurent Bossavit book “Leprechauns of Software Development” where he shows the evidence for this claim doesn’t hold up.

I guess these conclusions are going in the Deming direction.

Yet good people can make a bad system work; but if they do not succeed in actually changing the system it will eventually impose itself.

Can I suggest it comes down to a question of time scales: In the short-run good people can overcome a poor system, but in the long-run, unless they change the system for something better the system will overwhelm them.

And the mediocre people? - How about we say this:

Over the short-run, the better your people are the more likely they will overcome a poorer system, but in the long-run the hard it will be for people to sustain the effort if they do not change the system.

A good system will help everyone work the best. It will allow star-performers to shine and mediocre people to contribute and feel good, I’d even like to think a good system will improve the mediocre people.

Conway's Law v. Software Architecture

I've written about Conway's Law before (Return to Conway’s Law (2006) and a Focus Group I ran at EuroPLoP “What do we think of Conway’s Law Now?”)

but I make no apologies for revisiting it again because I still think it hold true.

There are times when I wonder if there is any point to the discipline called "Software Architecture." Sure design can make a difference in the small but when it comes to the big then, for my money, Conway's Law is a far more powerful force than the plans, designs and mandates of an enterprise architect.

Conway's Law predates Agile and Waterfall but it applies to both:

"Any organization that designs a system (defined more broadly here than just information systems) will inevitably produce a design whose structure is a copy of the organization's communication structure." Conway, 1968

The original paper “How do committees invent?” is available from Conway’s own website. The original publication was in Datamation, April 1968.

While Conway's Law can be seen at the macro level, the company level, it is also observed in in the small, at the micro level. The law if often restated in the small as:

"If you have four developers writing a compiler you will get a four pass compiler"

Or

"If you have three developers writing a UI you will get three ways of doing everything (mouse click, menu item, short-cut key)"

For example, if a development team is set up with a SQL database specialist, a JavaScript/CSS developer and a C# developer you they will produce a system with three tiers: a database with stored procedures, a business middle tier and a UI tier. This design will be applied whether it is the best or not. Indeed, the system might not even need a relational database or a graphical interface - a flat file might be quite adequate for data storage and a command line interface perfectly acceptable.

It is a little like a children's cartoon where the superheroes need to rescue somebody: Superman says "I can use my super strength to lift the ice boulders out of the way", while Fireman says "I can use my fire breath to melt the ice" and Drillerman says "I use my drill hands to make a rescue tunnel". Once added to the picture each specialist finds a way to utilise his or her special powers. For a software team this can lead to poor architecture, over complicated designs and competing modules.

In setting up a team architectural decisions are being made even if no architecture diagrams are being drawn. Just deciding to staff the team with four developers rather than three will influence the architecture because work now needs to be found for the fourth team member.

These decisions are made at the time when the least information is known - right at the beginning. They may made by planners - project managers - who have little knowledge of the technologies or by people who will not actually be involved in the development effort, e.g. enterprise architects.

To avoid this my advice is to start a team as small as possible. Try to avoid making architectural decisions in staffing the team by understaffing it. Keeping the team hungry will reduce the possibility of building more than is needed or over architecting it.

There are those who would quickly point out the risk of under architecting a system, not looking to the future and not building a system that can change and grow. But the risks of over architecting are if anything worse. Too much architecture can equally prevent a system changing and growing, and too much architecture leads to more time consuming and expensive code to cut. Then there is the risk of not shipping at all, too long spent producing the "right" design may result in a system too late to be viable.

Second staff the team with people who are more generalist than specialist, people who have more than one skill. Yes have a Java developer on the team but have one who knows a bit of SQL and isn't scared of a little UI work. In time you might need to add database and UI specialists but delay this until it is clear they are needed.

These suggestions echo Conway's own conclusion at the end of his paper:

"Ways must be found to reward design managers for keeping their organizations lean and flexible. There is need for a philosophy of system design management which is not based on the assumption that adding manpower simply adds to productivity."

It is worth remembering that Conway was writing over 20 years before Womack and Jones coined the term Lean to describe Toyota.

Reverse Conway's Law & Homomorphic force

Since Conway wrote this a lot more systems have been developed. Many of these systems are now referred to as "Legacy Systems." In some cases these systems force certain structure on the team who maintain the systems and even on the wider company.

For example, take the 3-tier database, business, GUI system from above. Years go by, the original developers leave the company and new staff are brought in. Perhaps some of these have been and gone in the years. The net effect is a system so complex in all three tiers that each requires a specialist. The database is so rich in stored procedures, triggers and constraints that only a SQL expert can understand it. The GUI is crammed full of JavaScript, CSS and not HTML 5 that only someone dedicated to interfaces can keep it all working. And the middle tier isn't any different either.

Given this situation the company has no option but to staff the team with three experts. Conway's Law is now working in reverse: the system is imposing structure on the organization.

Again this happens not just at the micro-level but at the macro-level. Entire companies are constrained by the systems they have. Economists might call this path-dependency: you are where you are because of how you got here not because of any current, rational, forces.

Underlying both Conway's Law and Reverse Conway's Law is the Homomorphism, or as I prefer to think of it: The Homorphic Force.

This force acts to preserve the structure of system even as the system itself moves from one technology to another, from one platform to another.

Both forms of Conway's Law and the Homomorphic Force pose a dilemma for any organization. Should they work with the force or try to break it?

I can't answer this question generically, there is too much context to consider. However I tend towards saying: Work with Conway's Law, not against it - like woodworking, work with the grain not across it. Be aware of Conway's Law and Learn to play it to your advantage.

Conway's Law does contain a get out clause: the system that will be created will be a copy of an existing system, if you can create a new system, a system not pre-loaded with assumptions and well-trodden communication paths then maybe you can create something new and of a better design.

Thus I sometimes view myself as an architect. Now I don't think I'd accept membership of the Architects-Club even if it was offered, and probably my software design skills are not up to the job but it doesn't stop me dreaming.

But I architect by (trying) to bring about a good organizational architecture which will in turn bring about a better system architecture via Conway's Law.

Links! - 2 conferences, 1 week

I’ve been to two conferences this week!

The first was Agile Dev Practices in Potsdam, outside of Berlin. At I presented my Retrospective Dialogue Sheets (www.dialoguesheets.com), well I say presented, it was 10 minutes of introduction, 60 minutes of attendees doing Dialogue Sheets and 20 minutes wrap up.

Anyone who has attended one of my Dialogue Sheet sessions will recognise the format. Which also means there aren’t a lot of slides for download. There are as few slides, essentially the same as Agile Cambridge last September so if you grab those slides you’ll be fine - Dialogue Sheets presentation as PDF download or Dialogue Sheets presentation on SlideShare, take your pick.

The second conference was DevWeek 2013 here in London. This was a completely new presentation: 10 Tips to make your Agile Adoption more successful - so far I only have this up on SlideShare. This presentation was based on an article I wrote for InfoQ last year My 10 things for making your Agile adoption successful.

If you check out the Allan Kelly InfoQ page you will find more on Dialogue Sheets too - I say this because I’m expecting InfoQ to publish another piece on Dialogue Sheets before long.