Writing that last blog entry on my introduction of Kanban also led me to reflect a bit on my Blue-White-Red Agile method.
I’ve long felt Blue-White-Red was largely derived from XP, thats what the developers were excited about. I called it Blue-White-Red because it was a cut down XP, I was introducing the ideas incrementally. Also, the word “extreme” frightened the managers and “pair programming” frightened most developers.
In the last few years I seen more and more Scrum in BWR. At the time I “invented” it I hadn’t read much about Scrum books so maybe it is surprising how much like Scrum it looks. But, big but, I think this also goes to show how little difference there is between XP and Scrum. I’d have to go back and read the original XP book to be sure but it seems to me the main difference is XP’s emphasis on engineering practices (TDD, refactoring, etc.) while Scrum emphasis the project management bit (just without the project manager).
The other influence was Lean. I’d started reading about lean sometime around 2000. In fact I wrote about “Thinking Beyond Lean” in the April 2002 issue of Overload. I’d also been exposed to a lot of lean on my MBA course.
I distinctly remember telling people the white board was a “Kanban board” and when people asked “What if I have too many cards to put see the board?” | telling them “That is a sign you have too much work on the go” - I was attempting a crude form of work in progress limits.
So what would I recommend to teams in future?
Well as always it depends. But I’m going to favour Kanban. I think Scrum (and BWR) still have their place, they are more suited to environments were large project specifications exist up front. In addition, Scrum has the advantage when the organization needs to see external validation and BWR has the advantage because there is less baggage. There is less written about it.
That last bit might seem a little off but I mean it. The fact that it is less well knows means it brings fewer assumptions, fewer stories about how it failed and perhaps a cleaner sheet. Kanban has many of those advantages today but I don’t expect it to last, success will remove them.
Thursday, January 29, 2009
Notes on a Kanban software development experience
I’ve mentioned the Kanban software development method in this blog before. For those who don’t know its “the new kid on the block” in Agile circles - although the originator (David Anderson) would be quick to point out it is designed to be a Lean development method.
Last year I did some consulting with a large online travel agency. I was involved with helping five teams “get Agile.” This presented an opportunity to try out Kanban in the wild. What I found was: it works, and I feel it is a better models of my own approach to software development than other methods.
What made this engagement more interesting was that I was able to start one team using the Blue-White-Red Agile (BWR) development method I’ve described before and two teams using Kanban so making for an interesting comparison.
The BWR team had been tasked with a project for which a large functional specification had been written. The spec wasn’t perfect but the organization expected them to complete it. BWR allowed the team to break the spec down and attempt it piecemeal. Some upfront estimates were made and a burn-down chart created.
(It almost looked Scrum like, that is, if you ignored the Project Manager, Development Manager, Team Leader, Architect and second Project Manager who joined the project later - and lets not forget me as Agile Coach. Its hard for a team of five developers (and a Product Mmanager) to be self-organizing when there is an equal number of self-organizers.)
This worked well. In the first few weeks I took a lot of heat because I refused to let a completion date for the work be given. Simply we did not have enough date to go on. Once the data started to come through I could do that.
The two Kanban teams were in a different position. Although they had “projects” to do most of their work was sustaining work. There were systems in place and issues needed fixing. Many of the things the company called “projects” were really to small to be worthy of the name - but that’s another story.
As a result the teams had more diversity in their work and more variability. Work would just appear - and disappear. They would have been hard pressed to keep to an unchanged task list (backlog) for a one week iteration, let alone a four week sprint.
This is the first good point about Kanban: its easier to handle changing work loads.
The teams still did “iterations” but the planning meetings quickly changed. Some work would be planned out but this could be usurped at any time. The iteration became the basic unit of comparison, allowing two periods to be compared. This meant that the start/end of iterations meetings were rather more of a review and retrospective session than planning session.
In future I think I would nominally keep iterations for this reason. They are useful time periods to review work and provide a regular review/retrospective point.
Second good point about Kanban: its easier to pick up.
On the whole I found the two Kanban teams needed less training, instruction and coaching than the BWR team (some of whom had been on Scrum training courses.) However, there was some work needed to map out the correct process flow for the Kanban teams to have on their boards.
For anyone creating a Kanban board some advice: big boards a better, involve the team and use coloured tape - insulation tape works well. In the past I’ve just drawn lines on the white board, with Kanban there is more movement and the lines are lost so I used insulation tape to create hard barriers.
A Kanban board
I expected the work in progress (WIP) limits in Kanban to pose a problem. I was expecting managers and others to want to break the WIP limit. This didn’t happen but I found developers breaking the WIP limits.
Developers broke the limits not because they disagreed but because they were accustomed to starting new work when they hit a problem. For example, a developer is working on item X, they come to a block on X so start on Y. In the extreme they block on Y and start on Z. You now have three partially done pieces of work. Trying to instil a “resolve the block” approach was difficult - largely because this organization was overloaded with managers and developers had been “taught” not to resolve these type of issues themselves.
In this organization lots of things blocked. Blocked on other teams. Blocked on IT support. Blocked on administration, etc. etc. I didn’t like it but I added a “blocked” column to the Kanban board. I also added a tally sheet to identify the biggest reoccurring blocks. I then tried to have the managers feel responsible for resolving the blocks. This was only partially successful because managers were themselves blocked because there were so many other “managers” who wanted their time. (The organization used talk as a substitute for action.)
All the teams held daily stand-up (Scrum) meetings and I encouraged the team manager, product manager and BAs to have their own pre-meeting to ensure the prioritisation queue was full and correct. This worked well.
I had the teams tracking cards by making a note on the back of the date as it progressed across the board. The date when it entered the priority queue, the date when work started and stopped, the date when it went for sign-off or test and the date when it was “done.” This data (when complete) was fascinating and started to reveal issues and opportunities, and give an idea of flow. In fact there was so much date I was overwhelmed.
In conclusion I find myself remembering Karl Scotland’s comment from a few months back. Karl had a quote which was something like: “The emphasis in Scrum is on being Agile and improvement follows; the emphasis in Kanban is on improving and being Agile follows.”
In the past I’ve shied away from labelling a team as doing “Scrum” or “XP” because I was aware they were not going it completely and we weren’t seeing all the benefits. I don’t feel that about Kanban, I’m happy to call these teams Kanban. Yes they had problems but we were resolving them. The teams were on a journey.
As anyone who has read Changing Software Development will know, I’m an advocate for incremental improvement. While Scrum (and XP, etc.) hold out the promise of overnight Agile - one big bang and your there - I see Agile as a journey not a destination.
Last year I did some consulting with a large online travel agency. I was involved with helping five teams “get Agile.” This presented an opportunity to try out Kanban in the wild. What I found was: it works, and I feel it is a better models of my own approach to software development than other methods.
What made this engagement more interesting was that I was able to start one team using the Blue-White-Red Agile (BWR) development method I’ve described before and two teams using Kanban so making for an interesting comparison.
The BWR team had been tasked with a project for which a large functional specification had been written. The spec wasn’t perfect but the organization expected them to complete it. BWR allowed the team to break the spec down and attempt it piecemeal. Some upfront estimates were made and a burn-down chart created.
(It almost looked Scrum like, that is, if you ignored the Project Manager, Development Manager, Team Leader, Architect and second Project Manager who joined the project later - and lets not forget me as Agile Coach. Its hard for a team of five developers (and a Product Mmanager) to be self-organizing when there is an equal number of self-organizers.)
This worked well. In the first few weeks I took a lot of heat because I refused to let a completion date for the work be given. Simply we did not have enough date to go on. Once the data started to come through I could do that.
The two Kanban teams were in a different position. Although they had “projects” to do most of their work was sustaining work. There were systems in place and issues needed fixing. Many of the things the company called “projects” were really to small to be worthy of the name - but that’s another story.
As a result the teams had more diversity in their work and more variability. Work would just appear - and disappear. They would have been hard pressed to keep to an unchanged task list (backlog) for a one week iteration, let alone a four week sprint.
This is the first good point about Kanban: its easier to handle changing work loads.
The teams still did “iterations” but the planning meetings quickly changed. Some work would be planned out but this could be usurped at any time. The iteration became the basic unit of comparison, allowing two periods to be compared. This meant that the start/end of iterations meetings were rather more of a review and retrospective session than planning session.
In future I think I would nominally keep iterations for this reason. They are useful time periods to review work and provide a regular review/retrospective point.
Second good point about Kanban: its easier to pick up.
On the whole I found the two Kanban teams needed less training, instruction and coaching than the BWR team (some of whom had been on Scrum training courses.) However, there was some work needed to map out the correct process flow for the Kanban teams to have on their boards.
For anyone creating a Kanban board some advice: big boards a better, involve the team and use coloured tape - insulation tape works well. In the past I’ve just drawn lines on the white board, with Kanban there is more movement and the lines are lost so I used insulation tape to create hard barriers.
A Kanban boardI expected the work in progress (WIP) limits in Kanban to pose a problem. I was expecting managers and others to want to break the WIP limit. This didn’t happen but I found developers breaking the WIP limits.
Developers broke the limits not because they disagreed but because they were accustomed to starting new work when they hit a problem. For example, a developer is working on item X, they come to a block on X so start on Y. In the extreme they block on Y and start on Z. You now have three partially done pieces of work. Trying to instil a “resolve the block” approach was difficult - largely because this organization was overloaded with managers and developers had been “taught” not to resolve these type of issues themselves.
In this organization lots of things blocked. Blocked on other teams. Blocked on IT support. Blocked on administration, etc. etc. I didn’t like it but I added a “blocked” column to the Kanban board. I also added a tally sheet to identify the biggest reoccurring blocks. I then tried to have the managers feel responsible for resolving the blocks. This was only partially successful because managers were themselves blocked because there were so many other “managers” who wanted their time. (The organization used talk as a substitute for action.)
All the teams held daily stand-up (Scrum) meetings and I encouraged the team manager, product manager and BAs to have their own pre-meeting to ensure the prioritisation queue was full and correct. This worked well.
I had the teams tracking cards by making a note on the back of the date as it progressed across the board. The date when it entered the priority queue, the date when work started and stopped, the date when it went for sign-off or test and the date when it was “done.” This data (when complete) was fascinating and started to reveal issues and opportunities, and give an idea of flow. In fact there was so much date I was overwhelmed.
In conclusion I find myself remembering Karl Scotland’s comment from a few months back. Karl had a quote which was something like: “The emphasis in Scrum is on being Agile and improvement follows; the emphasis in Kanban is on improving and being Agile follows.”
In the past I’ve shied away from labelling a team as doing “Scrum” or “XP” because I was aware they were not going it completely and we weren’t seeing all the benefits. I don’t feel that about Kanban, I’m happy to call these teams Kanban. Yes they had problems but we were resolving them. The teams were on a journey.
As anyone who has read Changing Software Development will know, I’m an advocate for incremental improvement. While Scrum (and XP, etc.) hold out the promise of overnight Agile - one big bang and your there - I see Agile as a journey not a destination.
Wednesday, January 21, 2009
ACCU 2009 conference and my keynote
I know many readers of this blog already go to the ACCU conference each year because people at the conference tell me they read this blog. Still there are those who don’t.
The 2009 schedule for the conference and the registration system is now up and running and it is a stronger programme than every - and that’s saying something! I used to be on the programme committee for the conference and every year we used to reject a few submissions. This year I believe there were as many proposals rejected as accepted.
And I am one on the keynote speakers. To be honest, its a little bit daunting to give a keynote at such a strong conference. (Especially when one of the other keynote speakers is a Baroness.)
I’m still putting my ideas together for the talk but I know it will be something along the lines of “Considerations for successful management of software development.” In addition I have a regular session of my own and session with Kevlin Henney in which we intend to involve several other speakers.
The one problem with the ACCU conference is that many people think the ACCU is the Association of C and C++ Users and its NOT. It was once upon a time but hasn’t been for some years. ACCU has followed the examples of BP (once British Petroleum), BT (once British Telecom) and others and kept the letters and dropped the meaning.
Yes there is C and C++ on the programme but there is also Objective-C, D, Java, C#, AJAX, Python, SOA, Erlang, Agile, Lean and lots more. In fact last year there was less C++ on the schedule than functional languages (F#, Erlang, Haskell and Lisp) - I haven’t done the analysis for this year yet.
Tuesday, January 20, 2009
Project plans (part 3 of 3): Planning cycles
This is the third in a short series of blog posts which started with me explaining why I don’t like Gantt charts as a “project” planning tool and then suggesting an alternative approach.
Short term project planning (1 to 6 months)
A regular release cycle is the first step to delivering. Decide what your cycle is, 1 month, 2 months - probably no more than 3 months and do work in those chunks. Size the work to fit in the cycle. Within the release cycle have several work cycles (iterations) of several weeks length. The more variable or unknown your work the shorter your cycles.
Its OK to sketch out what will be in upcoming iterations and cycles but don’t get too attached to what is allocated where. It will change. And for that reason don’t go into too much detail - certainly no estimates.
All work that goes into the iteration or cycle is rigourously prioritised - 1, 2, 3, ... to N. (As mentioned in Run Scope Creep backwards recently.) At the start of each cycle, then each iteration, the team - and business owner - can have detailed conversations about what is to be done and how it is to be done. It doesn’t make sense to have these conversations earlier - because less is know about the need and thing might change. And it doesn’t make sense to have these conversations later because that is too late.
These items aren’t done until they are completely done. All or nothing, no bugs left to fix, no UI to complete, no db schema to revisit, etc. Done as in ready to go to customer.
Ideally each release should have its own small goal but sometimes it ends up being a hotch-potch of things. And ideally the team should commit to doing the work. Many Agile books and methods talk about the importance of commitment, I agree but... its hard to get one day-1, it comes with time.
Roadmapping: Medium term (3 months to 1 year)
For the 3 months to 1 years period create a near term product roadmap. The roadmap is a living document which should be changing. It should be marked with the time buckets of your cycles and no finer. Things can and should move between buckets but everything except the bucket dates is flexible.
A roadmap is not a direct substitute for a project plan. Roadmaps are different. They are need driven not date driven. They are an artefact of product management not project management. They are a learning tool not a commitment. They create discussion not enforcement.
And yes, sanitised version might be shown outside the company to customer but only with the caution that it might change.
Roadmapping: 1 year to 5 years
All roadmaps are rolling but beyond the next year it doesn’t make sense to have so much detail so have another roadmap for the mid-term. You can get more general, more speculative simply because more things can change. So keep your near term and long term roadmaps separate.
This roadmap should show major events inside and outside your company: planned IPO, proposed legislation changes, anticipated competitor products and major event with your major customers.
So why are the roadmaps I describe better than project plan Gantts? Well...
• They are designed to keep the debate open and keep discussion on what is needed
• They do not forecast into the future they are one scenario
• Items in on the roadmap turn into goals supporting the larger visions for the product and company
• Dates are kept vague and planning is separate from commitment
See my Product Roadmap pattern for more about roadmaps, this is part of my business pattern series.
Beyond 5 years: Strategy and scenario planning
These technique tends to work best in the short term. In the long long term where you are going, what your product looks like are a matter of business strategy. Top management should pass down some direction here. Planning for the longer term needs different tools. For the long term - 5 years or more I suggest you start with some scenario planning.
There you go, a rough guide to project planning. Some people would call it Agile project planning, personally I don’t care what you call it. If you want more details it depends on your situation so get in touch.
• Project plans (part 1 of 3): Why I don't like Gantt charts
• Project plans (part 2 of 3): Turn the question around
• Project plans (part 3 of 3) - Planning cycles
• Run Scope Creep backwards
Short term project planning (1 to 6 months)
A regular release cycle is the first step to delivering. Decide what your cycle is, 1 month, 2 months - probably no more than 3 months and do work in those chunks. Size the work to fit in the cycle. Within the release cycle have several work cycles (iterations) of several weeks length. The more variable or unknown your work the shorter your cycles.
Its OK to sketch out what will be in upcoming iterations and cycles but don’t get too attached to what is allocated where. It will change. And for that reason don’t go into too much detail - certainly no estimates.
All work that goes into the iteration or cycle is rigourously prioritised - 1, 2, 3, ... to N. (As mentioned in Run Scope Creep backwards recently.) At the start of each cycle, then each iteration, the team - and business owner - can have detailed conversations about what is to be done and how it is to be done. It doesn’t make sense to have these conversations earlier - because less is know about the need and thing might change. And it doesn’t make sense to have these conversations later because that is too late.
These items aren’t done until they are completely done. All or nothing, no bugs left to fix, no UI to complete, no db schema to revisit, etc. Done as in ready to go to customer.
Ideally each release should have its own small goal but sometimes it ends up being a hotch-potch of things. And ideally the team should commit to doing the work. Many Agile books and methods talk about the importance of commitment, I agree but... its hard to get one day-1, it comes with time.
Roadmapping: Medium term (3 months to 1 year)
For the 3 months to 1 years period create a near term product roadmap. The roadmap is a living document which should be changing. It should be marked with the time buckets of your cycles and no finer. Things can and should move between buckets but everything except the bucket dates is flexible.
A roadmap is not a direct substitute for a project plan. Roadmaps are different. They are need driven not date driven. They are an artefact of product management not project management. They are a learning tool not a commitment. They create discussion not enforcement.
And yes, sanitised version might be shown outside the company to customer but only with the caution that it might change.
Roadmapping: 1 year to 5 years
All roadmaps are rolling but beyond the next year it doesn’t make sense to have so much detail so have another roadmap for the mid-term. You can get more general, more speculative simply because more things can change. So keep your near term and long term roadmaps separate.
This roadmap should show major events inside and outside your company: planned IPO, proposed legislation changes, anticipated competitor products and major event with your major customers.
So why are the roadmaps I describe better than project plan Gantts? Well...
• They are designed to keep the debate open and keep discussion on what is needed
• They do not forecast into the future they are one scenario
• Items in on the roadmap turn into goals supporting the larger visions for the product and company
• Dates are kept vague and planning is separate from commitment
See my Product Roadmap pattern for more about roadmaps, this is part of my business pattern series.
Beyond 5 years: Strategy and scenario planning
These technique tends to work best in the short term. In the long long term where you are going, what your product looks like are a matter of business strategy. Top management should pass down some direction here. Planning for the longer term needs different tools. For the long term - 5 years or more I suggest you start with some scenario planning.
There you go, a rough guide to project planning. Some people would call it Agile project planning, personally I don’t care what you call it. If you want more details it depends on your situation so get in touch.
• Project plans (part 1 of 3): Why I don't like Gantt charts
• Project plans (part 2 of 3): Turn the question around
• Project plans (part 3 of 3) - Planning cycles
• Run Scope Creep backwards
Project plans (part 2 of 3): Turn the question around
As my last post was a moan I promised to give some advice on how to go about project planning. There is a lot to this and I can’t possibly give all the details in a short blog so I’ve split this into three blog posts. Still there is a lot more I could say.
Turn the question around
Instead of a confrontational question and defensive answer have a rich discussion.
When someone asks “when will it be ready?” turn the question around - (1) When is “it” needed by?
When does your business need a product to use? What are the important dates for the company - end of quarter? end of year? trade show? competitors? contract signing?
I always prefer running a project were there are hard business driven milestones to make. Dates which mean something.
Next (2) What is needed on these dates?
Unfortunately too many conversations in the software business seem to involve someone saying “I need everything and I need it yesterday” - not a very useful position, and if its true you should start looking for a new job because there is no way anyone can make that happen.
Look at each of those date and define what is needed. What are the qualities of the thing that is needed? What is the minimum marketable product or feature for that date?
At this point you know: what is needed, how long you have, and who you have to do it.
Yes, I slipped that last one in because its true: you have the people you have and in the short run your resources are constrained because of Brook’s Law and because hiring people takes time.
So within these parameters the engineering team can now come up with options. How can this need be met within this time with these people? This activity separates the real engineers from the rest. Engineers create solutions within these parameters.
Wikipedia (today) says: “Engineers are concerned with developing economical and safe solutions to practical problems, by applying mathematics and scientific knowledge while considering technical constraints. As such, the work of engineers is the link between perceived needs of society and commercial applications.”
And if you cant? Then go back to the “customer” and discuss it. Find a smaller feature, extend the time (shoot for a different trade show), find an alternative way of delivering.
(Tom Gilb has some good examples of engineering solutions into tight constraints and his impact estimation tables can come in useful here.)
Vision
All work needs to be guided by a bigger vision. You can call it a vision or a goal if you wish I don’t mind. Personally I like the term BHAG - Big Hairy Audacious Goal. You’ve got to know where you are going to give context to everything else.
There are two ways to run a project. The first - exemplified by Gantt charts - is to get all the pieces of string you think need to do the project, lie them end-to-end and see where you get to. If you read my previous post you’ll see why I don’t like that approach.
The second way, I’ll call it the BHAG way, is what I discuss above. Decide where you want to be, when you want to be there, what is will look like and then find a way of getting there. How you get there is called engineering. It is what engineers do.
My favourite example of the BHAG way? The Digital OpenVMS team committed to a delivery schedule and said “We don’t know how to achieve this, but we commit to finding a way.”
• Project plans (part 1 of 3): Why I don't like Gantt charts
• Project plans (part 2 of 3): Turn the question around
• Project plans (part 3 of 3) - Planning cycles
• Run Scope Creep backwards
Turn the question around
Instead of a confrontational question and defensive answer have a rich discussion.
When someone asks “when will it be ready?” turn the question around - (1) When is “it” needed by?
When does your business need a product to use? What are the important dates for the company - end of quarter? end of year? trade show? competitors? contract signing?
I always prefer running a project were there are hard business driven milestones to make. Dates which mean something.
Next (2) What is needed on these dates?
Unfortunately too many conversations in the software business seem to involve someone saying “I need everything and I need it yesterday” - not a very useful position, and if its true you should start looking for a new job because there is no way anyone can make that happen.
Look at each of those date and define what is needed. What are the qualities of the thing that is needed? What is the minimum marketable product or feature for that date?
At this point you know: what is needed, how long you have, and who you have to do it.
Yes, I slipped that last one in because its true: you have the people you have and in the short run your resources are constrained because of Brook’s Law and because hiring people takes time.
So within these parameters the engineering team can now come up with options. How can this need be met within this time with these people? This activity separates the real engineers from the rest. Engineers create solutions within these parameters.
Wikipedia (today) says: “Engineers are concerned with developing economical and safe solutions to practical problems, by applying mathematics and scientific knowledge while considering technical constraints. As such, the work of engineers is the link between perceived needs of society and commercial applications.”
And if you cant? Then go back to the “customer” and discuss it. Find a smaller feature, extend the time (shoot for a different trade show), find an alternative way of delivering.
(Tom Gilb has some good examples of engineering solutions into tight constraints and his impact estimation tables can come in useful here.)
Vision
All work needs to be guided by a bigger vision. You can call it a vision or a goal if you wish I don’t mind. Personally I like the term BHAG - Big Hairy Audacious Goal. You’ve got to know where you are going to give context to everything else.
There are two ways to run a project. The first - exemplified by Gantt charts - is to get all the pieces of string you think need to do the project, lie them end-to-end and see where you get to. If you read my previous post you’ll see why I don’t like that approach.
The second way, I’ll call it the BHAG way, is what I discuss above. Decide where you want to be, when you want to be there, what is will look like and then find a way of getting there. How you get there is called engineering. It is what engineers do.
My favourite example of the BHAG way? The Digital OpenVMS team committed to a delivery schedule and said “We don’t know how to achieve this, but we commit to finding a way.”
• Project plans (part 1 of 3): Why I don't like Gantt charts
• Project plans (part 2 of 3): Turn the question around
• Project plans (part 3 of 3) - Planning cycles
• Run Scope Creep backwards
Project plans (part 1 of 3): Why I don't like Gantt charts
The first of three posts: this one sets the problem and the next two attempt a solution...
A friend e-mails to say that his team have almost finished their project plan. There is something about the words “project plan” which are like a red flag to this bull... while I’m prepared to admit I may over react sometimes I don’t think I’m completely crazy and I think I can put some logic behind my dislike of “project plans”.
Lets start by defining what I mean. Its not so much a “project plan” - after all I think planning is a great way to explore your problem. As a mental exercise a team planning what they are going to do is good. And I’m all for a bit of critical path analysis to work out what the dependencies are. And I think scenario planning is great. From what I’ve heard war-gaming is another good for of planning.
I should admit to an assumption. When this friend says “project plan” I’m immediately imagining Gantt charts and PERT diagrams. There are many other forms of plan which make a lot of sense. I think product road-mapping is an essential activity. Its specifically Gantt and PERT charts that make me see red - and as layman in the field I don’t see much difference between Gantt and PERT to be honest.
So what is it about Gantt charts? Here’s some:
• The charts are built on estimates and estimates are just that, estimates. They are not commitments, they are not fact, they are not enforceable. Therefore Gantt charts are not an accurate way to plan a project.
• The charts create illusion of control; here is a piece of paper which says the project will be done on 1 May 2010 - or whenever. It won’t. That date is the one date the project will not be ready. It might be ready before, more likely it will be ready after but that one date is fantasy.
• Many - too many - discussions now centre on “why are you deviating from the plan” - “why are the estimates wrong?” - “how can we get back on plan?” - arguments centre over the plan not the thing being produced.
• There is now work to do “correcting” the plan; updating it for what has happened and attempting to re-forecast. This makes work but does not add value.
• The charts are never accurate but they are treated as such.
• The “plan” is taken as a form of commitment.
• The further out the plan the more inaccurate it is: planning for next month is one thing but 5 years out? There is simply not enough information to do it accurately. Plus, things change too much, the further out you look the more things can (and will) change.
Basically, what I’m driving at is: once the plan is in place disucssion moves from what is important (what are we building? when do we need it by?) to the plan. Its goal deferment.
There is a lot more I could say here but I’d be repeating myself. If you are interested read An alternative view of design (and planning) and the section in Changing Software Development.
Fortunately I showed this to my friend and he says they’ve not drawn a Gantt chart, phew! But before he tries I need to give some advice, the next two post will try and do that.
• Project plans (part 1 of 3): Why I don't like Gantt charts
• Project plans (part 2 of 3): Turn the question around
• Project plans (part 3 of 3) - Planning cycles
• Run Scope Creep backwards
A friend e-mails to say that his team have almost finished their project plan. There is something about the words “project plan” which are like a red flag to this bull... while I’m prepared to admit I may over react sometimes I don’t think I’m completely crazy and I think I can put some logic behind my dislike of “project plans”.
Lets start by defining what I mean. Its not so much a “project plan” - after all I think planning is a great way to explore your problem. As a mental exercise a team planning what they are going to do is good. And I’m all for a bit of critical path analysis to work out what the dependencies are. And I think scenario planning is great. From what I’ve heard war-gaming is another good for of planning.
I should admit to an assumption. When this friend says “project plan” I’m immediately imagining Gantt charts and PERT diagrams. There are many other forms of plan which make a lot of sense. I think product road-mapping is an essential activity. Its specifically Gantt and PERT charts that make me see red - and as layman in the field I don’t see much difference between Gantt and PERT to be honest.
So what is it about Gantt charts? Here’s some:
• The charts are built on estimates and estimates are just that, estimates. They are not commitments, they are not fact, they are not enforceable. Therefore Gantt charts are not an accurate way to plan a project.
• The charts create illusion of control; here is a piece of paper which says the project will be done on 1 May 2010 - or whenever. It won’t. That date is the one date the project will not be ready. It might be ready before, more likely it will be ready after but that one date is fantasy.
• Many - too many - discussions now centre on “why are you deviating from the plan” - “why are the estimates wrong?” - “how can we get back on plan?” - arguments centre over the plan not the thing being produced.
• There is now work to do “correcting” the plan; updating it for what has happened and attempting to re-forecast. This makes work but does not add value.
• The charts are never accurate but they are treated as such.
• The “plan” is taken as a form of commitment.
• The further out the plan the more inaccurate it is: planning for next month is one thing but 5 years out? There is simply not enough information to do it accurately. Plus, things change too much, the further out you look the more things can (and will) change.
Basically, what I’m driving at is: once the plan is in place disucssion moves from what is important (what are we building? when do we need it by?) to the plan. Its goal deferment.
There is a lot more I could say here but I’d be repeating myself. If you are interested read An alternative view of design (and planning) and the section in Changing Software Development.
Fortunately I showed this to my friend and he says they’ve not drawn a Gantt chart, phew! But before he tries I need to give some advice, the next two post will try and do that.
• Project plans (part 1 of 3): Why I don't like Gantt charts
• Project plans (part 2 of 3): Turn the question around
• Project plans (part 3 of 3) - Planning cycles
• Run Scope Creep backwards
Sunday, January 18, 2009
Running Scope creep backwards
One of the criticisms levied at Agile software development is that of Scope Creep. Project managers seem particularly sensitive to this issue, probably because its been fatal to many projects before and thus Project Manager are trained to watch for it and come down hard on it.
However, I’ve come to the conclusion that in a lot of Agile work the secret is to run Scope Creep backwards. Rather than have a project get bigger as more and more requests are added to it what we need to do is have it get smaller over time. Let me explain.
Typically a product/project gets specified and then work proceeds, as it does more and more "change requests" appear, things people hadn't thought of, ideas for what else it could do - scope creep. (See my piece, Why do requirements change from a few years ago if you want to know more about the why.)
As projects grow in size the chances of them meeting any deadline diminish. This is why traditional Project Managers look at Agile and throw their hands up in the air seeing rampant Scope Creep.
But if you run it backwards, write the spec, write it as big as you like, who cares how big you write it or how many features you put in. The bigger it is the less people will remember. (See my piece on the Documentation myth for why big documents don’t work.)
Allowing everything into the spec means you don’t have to have those hard conversations about what’s in and what’s out. Everyone gets to feel there thing is in.
Now prioritise. If its really big make a first pass with MoSCoW rules. Anything which isn't MUST have can be ignored for now - you've probably got more must's than you have time for anyway. Generally I’m not a fan of Moscow rules but they are OK as a starting point when you have a lot of requests.
(Note: that’s MoSCoW rules, not Moscow rules - personally my only Moscow rule is avoid the city, my last trip there was awful so future visits are restricted to changing planes.)
Now within the MUSTs prioritise 1, 2, 3.... N. No duplicate priotities. This way when something increases in priority ther things fall automatically, people immediately see the consequences of their decisions.
What criteria you use for prioritisation is up to you. Doing it by value makes a lot of sense, so does doing it by risk. It also makes sense to get some early wins under your belt.
Next work through the items in priority order.
Anything that comes in later (Scope Creep) has to be slotted into these priorities. Add a new item with priority 3 and everything 4..N moves down one.
Notice I’m not interested in change control. I don’t make any effort to restrict the work that is asked for. All I do is demand it is prioritised along with everything else. Valuable ideas can arrive at any point in the work so why draw an arbitary line to differenciate what was “in scope” and what was “out of scope.” The objective is deliver value.
Now as you work through the list you will find that with items 1..N' you have a a valuable product. Something you can at least show to customers, something people might even pay money for today.
Once you start getting real feedback - especially the sort represented by money - everything changes. Once you know what real customers want you are, literally, in business. You’ll want to add new items and look again at your priorities.
At this point you've saved yourself the work N'..N plus all the Should Have's, Could Haves, etc. This is running Scope Creep backwards.
What all of this highlights is that building a software product isn’t a binary decision. It isn’t “all of this stuff” or nothing. Its an analogue thing not a digital thing. Traditional projects spend a lot of time trying to prevent scope creep through tough change control procedures but really it doesn’t matter when a request comes in. What is important is what is the value it will bring.
Value, not arbitary dates, should be the criteria for deciding what to build.
However, I’ve come to the conclusion that in a lot of Agile work the secret is to run Scope Creep backwards. Rather than have a project get bigger as more and more requests are added to it what we need to do is have it get smaller over time. Let me explain.
Typically a product/project gets specified and then work proceeds, as it does more and more "change requests" appear, things people hadn't thought of, ideas for what else it could do - scope creep. (See my piece, Why do requirements change from a few years ago if you want to know more about the why.)
As projects grow in size the chances of them meeting any deadline diminish. This is why traditional Project Managers look at Agile and throw their hands up in the air seeing rampant Scope Creep.
But if you run it backwards, write the spec, write it as big as you like, who cares how big you write it or how many features you put in. The bigger it is the less people will remember. (See my piece on the Documentation myth for why big documents don’t work.)
Allowing everything into the spec means you don’t have to have those hard conversations about what’s in and what’s out. Everyone gets to feel there thing is in.
Now prioritise. If its really big make a first pass with MoSCoW rules. Anything which isn't MUST have can be ignored for now - you've probably got more must's than you have time for anyway. Generally I’m not a fan of Moscow rules but they are OK as a starting point when you have a lot of requests.
(Note: that’s MoSCoW rules, not Moscow rules - personally my only Moscow rule is avoid the city, my last trip there was awful so future visits are restricted to changing planes.)
Now within the MUSTs prioritise 1, 2, 3.... N. No duplicate priotities. This way when something increases in priority ther things fall automatically, people immediately see the consequences of their decisions.
What criteria you use for prioritisation is up to you. Doing it by value makes a lot of sense, so does doing it by risk. It also makes sense to get some early wins under your belt.
Next work through the items in priority order.
Anything that comes in later (Scope Creep) has to be slotted into these priorities. Add a new item with priority 3 and everything 4..N moves down one.
Notice I’m not interested in change control. I don’t make any effort to restrict the work that is asked for. All I do is demand it is prioritised along with everything else. Valuable ideas can arrive at any point in the work so why draw an arbitary line to differenciate what was “in scope” and what was “out of scope.” The objective is deliver value.
Now as you work through the list you will find that with items 1..N' you have a a valuable product. Something you can at least show to customers, something people might even pay money for today.
Once you start getting real feedback - especially the sort represented by money - everything changes. Once you know what real customers want you are, literally, in business. You’ll want to add new items and look again at your priorities.
At this point you've saved yourself the work N'..N plus all the Should Have's, Could Haves, etc. This is running Scope Creep backwards.
What all of this highlights is that building a software product isn’t a binary decision. It isn’t “all of this stuff” or nothing. Its an analogue thing not a digital thing. Traditional projects spend a lot of time trying to prevent scope creep through tough change control procedures but really it doesn’t matter when a request comes in. What is important is what is the value it will bring.
Value, not arbitary dates, should be the criteria for deciding what to build.
Wednesday, January 14, 2009
Software East next Thursday (22 January)
A shameless act of publicity - for myself and Software East in Cambridge.
I’ll be part of a panel at Software East next Thursday. The subject is Building a Software Business, pre-registration is required.
I’ll be part of a panel at Software East next Thursday. The subject is Building a Software Business, pre-registration is required.
Monday, January 12, 2009
Timing and three types of Retrospectives
I was told the following story this week by one of my clients. He works for a large organization with many software development groups. The company believes retrospectives are a good thing and some people in the company, including my client, are trained in facilitation.
A few weeks before Christmas my client got a call from another group lead in the company who asked if he could facilitate a retrospective. “Sure” said my client who went to discuss it with the group lead.
Now it turns out that this group has not been holding regular (iteration) retrospectives during the project so this was to be the end of project retrospective. But the project had slipped a couple of months so wouldn’t ship until February. “Arh” says my client, “wouldn’t it make sense to hold this retrospective then?”
“Yes” says the group lead, “but, one of my objectives for the year is to hold a project retrospective so we need to do it this side of Christmas”. This company operates management by objective and the group lead had this objectives and his manager didn’t want to vary the objectives so an “almost done” retrospective it would be.
There are two schools of thought on when to hold a retrospective and this case was neither of them. The first school, exemplified by Norm Kerth in Retrospectives is the Big End of Project Retrospective (BEPR). The problem with this type of retrospective is that it occurs too late to make a difference to a project. Only if the same people will be working on something similar sometime soon is there a lot of benefit to the participants.
Still the BEPR is a good way of getting started with retrospectives. If you’ve never held one before there is a lot of ground to cover. I find them useful when I’m getting started with a team as a first step towards regular retrospectives and to help loosen the team up. It doesn’t necessarily have to be at the end of a project but to have value it needs to be held in time to make a difference. Do it in “stoppage time” probably makes little difference.
The second school of thought is exemplified in Diana Larsen and Esther Derby’s book Agile Retrospectives is the Regular Iteration Retrospective (RIR). I recently came across Joshua Kerievsky’s “How To Run An Iteration Retrospective” which is a good one page guide to this type of retrospective.
These are smaller, shorter, retrospectives that occur on a regular basis. Typically these are held at the end of an iteration so anything between 1 week and 4 weeks. By occurring regularly such retrospectives can make a difference while work is in progress. There are two problems here, the first is confusion and the second freshness, let me explain.
Simply, people get confused about what is a retrospective at the end of an iteration and what is not. This isn’t helped when you occasionally skip a retrospective.
I’ve been known to skip retrospectives, typically when the team is still settling and I can see changes occurring organically. In these cases I prefer to leave things to improve by themselves.
I’ve also skipped them when I feel that they will sap morale. I recently worked with a large client where I was coaching several teams. It quickly become apparent that the problems the teams were facing were systematic and beyond the teams ability to fix. What was needed was structural change across the organization. This wasn’t going to happen any time soon but until it did asking the teams to reflect on the problems they faced would only revisit the same old ground.
At the end of an iteration there are three types end of reflection to my mind:
• Review: this happens at the end of every iteration. Look at the work done, count the points (if using), talk about what was done and problems encountered. This needs to happen to close the iteration. Not really a retrospective but sometimes its seen as one.
• Informal retrospective: as part of the review or immediately afterwards. A discussion of what has happened, what could be useful and how the team might change. Informal retrospectives are useful when there is not the time for a formal one (they flow naturally after a review), when the team are still settling in or when the team are recovering from a lot of micro-management and learned dependency.
• Formal retrospectives: a formal period of time for reflection with some exercises to facilitate thinking. This format also allows you to keep track of what action was decided on and follow up changes. Ultimately all teams should be aiming for this type of retrospective.
The second problem with RIR’s is freshness and mainly the formal retrospectives. If you are holding these every second week and doing the same exercises they quickly become boring. The trick is to vary the exercises and try new things. Again this is one reason why I’m happy to use informal retrospectives, more variety.
I’ve recently had success using dialogue sheets as a facilitation tool in formal retrospectives. I’ll post my template online soon and blog about it in more detail.
One of the big difference between BEPR and RIR is the timeline. In a BEPR the creation of a timeline is normally a (even the) major event. The team map out what took place and when it happened, and then reflect on it. If you are holding RIR then creating a timeline for, say, 2 weeks then it doesn’t get you very far. Hence the need for more activities to trigger reflection.
As you might have gathered from above, RIR are in many places displacing BEPR as teams move to more Agile development. Generally this is a good thing but I suspect the BEPR is due a comeback and I think the force for this will be the Kanban development process.
Two factors make RIR less suitable to Kanban. Firstly the most advanced Kanban teams don’t bother with iterations. The pull system renders them less necessary, therefore the natural place to hold an RIR disappears.
Second, drawing from Lean, Kanban teams may well adopt a “Stop the line” mentality. That is, if the team see a problem the immediately address it rather than waiting for the retrospective to come along.
Yet I still think there is are occasions when it is worth the team stopping and actively taking time to reflect. In Lean we might think of this as a Kaizen or Kaikaku event. At such time a BEPR retrospective might be exactly right. Still, schedule them every quarter. Make them frequent enough to make a difference and avoid the problems my client had.
A few weeks before Christmas my client got a call from another group lead in the company who asked if he could facilitate a retrospective. “Sure” said my client who went to discuss it with the group lead.
Now it turns out that this group has not been holding regular (iteration) retrospectives during the project so this was to be the end of project retrospective. But the project had slipped a couple of months so wouldn’t ship until February. “Arh” says my client, “wouldn’t it make sense to hold this retrospective then?”
“Yes” says the group lead, “but, one of my objectives for the year is to hold a project retrospective so we need to do it this side of Christmas”. This company operates management by objective and the group lead had this objectives and his manager didn’t want to vary the objectives so an “almost done” retrospective it would be.
There are two schools of thought on when to hold a retrospective and this case was neither of them. The first school, exemplified by Norm Kerth in Retrospectives is the Big End of Project Retrospective (BEPR). The problem with this type of retrospective is that it occurs too late to make a difference to a project. Only if the same people will be working on something similar sometime soon is there a lot of benefit to the participants.
Still the BEPR is a good way of getting started with retrospectives. If you’ve never held one before there is a lot of ground to cover. I find them useful when I’m getting started with a team as a first step towards regular retrospectives and to help loosen the team up. It doesn’t necessarily have to be at the end of a project but to have value it needs to be held in time to make a difference. Do it in “stoppage time” probably makes little difference.
The second school of thought is exemplified in Diana Larsen and Esther Derby’s book Agile Retrospectives is the Regular Iteration Retrospective (RIR). I recently came across Joshua Kerievsky’s “How To Run An Iteration Retrospective” which is a good one page guide to this type of retrospective.
These are smaller, shorter, retrospectives that occur on a regular basis. Typically these are held at the end of an iteration so anything between 1 week and 4 weeks. By occurring regularly such retrospectives can make a difference while work is in progress. There are two problems here, the first is confusion and the second freshness, let me explain.
Simply, people get confused about what is a retrospective at the end of an iteration and what is not. This isn’t helped when you occasionally skip a retrospective.
I’ve been known to skip retrospectives, typically when the team is still settling and I can see changes occurring organically. In these cases I prefer to leave things to improve by themselves.
I’ve also skipped them when I feel that they will sap morale. I recently worked with a large client where I was coaching several teams. It quickly become apparent that the problems the teams were facing were systematic and beyond the teams ability to fix. What was needed was structural change across the organization. This wasn’t going to happen any time soon but until it did asking the teams to reflect on the problems they faced would only revisit the same old ground.
At the end of an iteration there are three types end of reflection to my mind:
• Review: this happens at the end of every iteration. Look at the work done, count the points (if using), talk about what was done and problems encountered. This needs to happen to close the iteration. Not really a retrospective but sometimes its seen as one.
• Informal retrospective: as part of the review or immediately afterwards. A discussion of what has happened, what could be useful and how the team might change. Informal retrospectives are useful when there is not the time for a formal one (they flow naturally after a review), when the team are still settling in or when the team are recovering from a lot of micro-management and learned dependency.
• Formal retrospectives: a formal period of time for reflection with some exercises to facilitate thinking. This format also allows you to keep track of what action was decided on and follow up changes. Ultimately all teams should be aiming for this type of retrospective.
The second problem with RIR’s is freshness and mainly the formal retrospectives. If you are holding these every second week and doing the same exercises they quickly become boring. The trick is to vary the exercises and try new things. Again this is one reason why I’m happy to use informal retrospectives, more variety.
I’ve recently had success using dialogue sheets as a facilitation tool in formal retrospectives. I’ll post my template online soon and blog about it in more detail.
One of the big difference between BEPR and RIR is the timeline. In a BEPR the creation of a timeline is normally a (even the) major event. The team map out what took place and when it happened, and then reflect on it. If you are holding RIR then creating a timeline for, say, 2 weeks then it doesn’t get you very far. Hence the need for more activities to trigger reflection.
As you might have gathered from above, RIR are in many places displacing BEPR as teams move to more Agile development. Generally this is a good thing but I suspect the BEPR is due a comeback and I think the force for this will be the Kanban development process.
Two factors make RIR less suitable to Kanban. Firstly the most advanced Kanban teams don’t bother with iterations. The pull system renders them less necessary, therefore the natural place to hold an RIR disappears.
Second, drawing from Lean, Kanban teams may well adopt a “Stop the line” mentality. That is, if the team see a problem the immediately address it rather than waiting for the retrospective to come along.
Yet I still think there is are occasions when it is worth the team stopping and actively taking time to reflect. In Lean we might think of this as a Kaizen or Kaikaku event. At such time a BEPR retrospective might be exactly right. Still, schedule them every quarter. Make them frequent enough to make a difference and avoid the problems my client had.
Sunday, January 04, 2009
Empirical research supports Conway's Law
Regular readers of this blog will know I have a thing about Conway’s Law. In my mind it makes sense and it links what I did before (software coding and architecture) with what I do now (organization and process improvement). My logic is: If Conway’s Law holds then architecting software starts with the organization.
So it was with interest I ran across a working paper from Harvard Business School entitled “Exploring the Duality of Product and Organizational Architectures: A Test of the Mirroring Hypothesis” (Alan D. MacCormack, John Rusnak, and Carliss Y. Baldwin). The author’s note Conway’s Law as one example of the mirroring hypotheses and cite several others too - which adds to the evidnece that there is something here.
(The link above is to a March 2008 version of the paper, the one I downloaded a couple of months ago was version 3.0 and dated October 2008, I’ve no idea what, if anything, changed in that time or where the October version is for download.)
In this paper the authors try and show statistically that organizational form does effect software architecture. Most of the paper’s 33 pages lays out in detail how they did this, and you could, if you so wish, poke holes in their techniques and question whether the analysis holds up.
Disclaimer, as is my usual practice I’ve read the introduction, the closing discussion and some of the material in between. Life is too short to read 33 pages of academic prose but it could be I’ve missed something, please let me know if I have.
That said, if you are willing to accept their hypothesis and method. There are two minor points that give me concern: 1) all the code they look at is Open Source, 2) we know little about the development practices employed by these teams. Both facts could skew the results, still I’m willing to give them a broad acceptance - then they show the hypothesis stands up.
The authors say:
“Our results reveal significant differences in modulality, consistent with a view that larger, more distributed teams tend to develop products with more modular architectures. Futhermore the differences are substantial - the pairs we examine vary by a factor of eight....”
And later:
“our study provides evidence to support the hypothesis that a product’s architecture tends to mirror the structure of the organization within which it is developed.”
There are many implications of this work and several are discussed by the authors. For me a few things come immediately to mind:
• The work of the architect and the manager are more difficult to separate than is commonly assumed. The organization will always effect the architectural.
• If you want a highly modular system distribute your team.
• This is why broken organizations usually have broken architectures too.
• When fixing a broken organization and/or architecture: fix the organization first.
This paper in common with most other work on Conway’s Law emphasises the way the architecture mirrors the organization. In so much as thisis true and it happens at first, organization determines architecture.
I also believe the reverse can be true. The organization can come to copy the architecture. In a world increasingly dominated by existing “legacy” systems it is common to see development organizations split up to model the software.
For example: Front End (UI) team, Business Logic Team and a Database team corresponding to three layers in a software. This may lead to a modular design in time but it complicates co-ordination. It becomes more difficult to make changes to the code and change the organization.
While most of the software world (including myself when I code) cling to the layered model it goes against object-oriention. In OO the object is should be self contained, too much layering and the object model breaks down. Layering is procedural not OO. The same thing happens in organizations. But all that will need to wait for another blog entry.
More on Conway’s Law:
• What do we think of Conway’s Law now? (EuroPLoP 2005 focus group)
• 2006 Blog posting: Return to Conway’s Law
• The original paper from 1968: How do committee invent?
So it was with interest I ran across a working paper from Harvard Business School entitled “Exploring the Duality of Product and Organizational Architectures: A Test of the Mirroring Hypothesis” (Alan D. MacCormack, John Rusnak, and Carliss Y. Baldwin). The author’s note Conway’s Law as one example of the mirroring hypotheses and cite several others too - which adds to the evidnece that there is something here.
(The link above is to a March 2008 version of the paper, the one I downloaded a couple of months ago was version 3.0 and dated October 2008, I’ve no idea what, if anything, changed in that time or where the October version is for download.)
In this paper the authors try and show statistically that organizational form does effect software architecture. Most of the paper’s 33 pages lays out in detail how they did this, and you could, if you so wish, poke holes in their techniques and question whether the analysis holds up.
Disclaimer, as is my usual practice I’ve read the introduction, the closing discussion and some of the material in between. Life is too short to read 33 pages of academic prose but it could be I’ve missed something, please let me know if I have.
That said, if you are willing to accept their hypothesis and method. There are two minor points that give me concern: 1) all the code they look at is Open Source, 2) we know little about the development practices employed by these teams. Both facts could skew the results, still I’m willing to give them a broad acceptance - then they show the hypothesis stands up.
The authors say:
“Our results reveal significant differences in modulality, consistent with a view that larger, more distributed teams tend to develop products with more modular architectures. Futhermore the differences are substantial - the pairs we examine vary by a factor of eight....”
And later:
“our study provides evidence to support the hypothesis that a product’s architecture tends to mirror the structure of the organization within which it is developed.”
There are many implications of this work and several are discussed by the authors. For me a few things come immediately to mind:
• The work of the architect and the manager are more difficult to separate than is commonly assumed. The organization will always effect the architectural.
• If you want a highly modular system distribute your team.
• This is why broken organizations usually have broken architectures too.
• When fixing a broken organization and/or architecture: fix the organization first.
This paper in common with most other work on Conway’s Law emphasises the way the architecture mirrors the organization. In so much as thisis true and it happens at first, organization determines architecture.
I also believe the reverse can be true. The organization can come to copy the architecture. In a world increasingly dominated by existing “legacy” systems it is common to see development organizations split up to model the software.
For example: Front End (UI) team, Business Logic Team and a Database team corresponding to three layers in a software. This may lead to a modular design in time but it complicates co-ordination. It becomes more difficult to make changes to the code and change the organization.
While most of the software world (including myself when I code) cling to the layered model it goes against object-oriention. In OO the object is should be self contained, too much layering and the object model breaks down. Layering is procedural not OO. The same thing happens in organizations. But all that will need to wait for another blog entry.
More on Conway’s Law:
• What do we think of Conway’s Law now? (EuroPLoP 2005 focus group)
• 2006 Blog posting: Return to Conway’s Law
• The original paper from 1968: How do committee invent?