Agile Engineering

I had the pleasure to meet Ryan Shriver last week at Tom Gilb’s seminar. Ryan had some very interesting ideas, experience and suggestions about Agile Software Development and Tom’s EVO process.

In fact, Ryan had independently arrived at many of the same conclusions I had about Agile, management and the future of development. He also gave me a new term, a new piece of the jigsaw which I think helps make sense of the world: Agile Engineering.

Ryan doesn’t claim to have invented the term but he did define it using seven points:

        1.        Deliver Value
        2.        Quantify Qualities
        3.        Reduce Feedback Cycles
        4.        Explore Alternatives
        5.        Last Responsible Moment
        6.        Front-to-Back
        7.        Test-Driven Development

This list will look familiar to anyone who knows about Agile Software Development but it is subtly different and allows for greater scope in the engineering, management and development of software systems.

Like me Ryan has seen the limitations of Agile Software Development and, like me, sees the need to go further. As it originated - in XP, in Scrum, etc. - Agile Software Development doesn’t go far enough to address requirements, design, large systems, and a whole bunch of other things you have to do to manage software development.

This is where Agile Engineering comes in, it includes Agile Software Development but is about more than just Agile Software Development.

Ryan Shriver has some presentations online which should expand on this. (He had a busy week last week, as well as presenting at Tom Gilb’s London seminar he also presented at Agile ITX in Reston, Virginia.)

So, Agile Software Development is Dead, long live Agile Engineering!

Uninsurable risks

Although my post about off-balance sheet risk was at the weekend I’ve already had a couple of responses by e-mail. Some of these comments are worth noting here:

“The insurance industry has long refused to take on unlimited liability on a number of perils (nuclear power, nanotechnology, terrorism). This is not only because you can't calculate exposure and thus risk-based capital, but also because the extent of the liability is fraught with political risk - and then there's always moral hazard in the picture.”

“This is the nuclear option. They can only ever sue a contractor once, unless they're shutting down. After that, noone sensible will work for them.”

Actually, Lloyds of London used to ask its “names” to accept on unlimited personal risk exposure but in the mid-1980’s this almost destroyed the Lloyd’s.

It has also been suggested that if one bank is doing this the others may follow soon. If this happens, then it will only take one bank to sure one contractor and everything will be exposed.

Of course, its entirely possible that this is all a storm in a tea-cup. The bank in question may have simply written the clause into the contract to give them a big stick with which to beat their contractors. So either the bank has an overly simplistic view of risk, or it has an overly simplistic view of IT. Neither option is particularly heart warming.

There might be a silver-lining here. If banks are really asking the IT suppliers to take full responsibility then it might alter the shape of the industry. In the short term the likes of Accenture, Logica, CSC, etc. will probably benefit as they are better able to take on these risks. It might also force IT suppliers to take on more responsibility and actually deliver systems which work, and which are usable. Unfortunately I’m not sure many, or even any, of the big suppliers are able to do this.

I’ve also been sent some interesting links which, unfortunately I’ve not had time to read in detail - nor am I likely to get the time in the near future. A quick scan did produce this interesting quote:

“The declining role of manufacturing makes physical activities and risks less critical to many firms, reducing their need for ‘bricks and mortar’ coverage. Meanwhile, new, harder-to-quantify risks have emerged for both service and manufacturing companies. Liability risk is one important example.” (Swiss Re Sigma publication 4/2005 - and in other languages)

Ever since I did my PRINCE2 certification earlier this year it has been clear to me that the IT industry does not have a good toolbox for coping with risk. Last weeks seminar added to that impression and gave me some ideas. However, what I perhaps didn’t appreciate is that even the present, limited, tools are wearing out too.

Off balance sheet risk

I spent most of last week at Tom Gilb’s seminar on Risk and Uncertainty. Tom expects everyone at this seminar to contribute, so, having heard several very competent speakers present some very thought provoking ideas I was, for the first time in a long time, a little bit nervous about doing my own presentation.

As it was my presentation - entitled Risk, Uncertainty and Money - was well received. Funny thing was, I had a much better understanding of what I was trying to communicate after I finished presenting! I expect a revised version of this session to resurface at some other conference in the near future.

A few days before the seminar I heard about a new risk management technique being employed at one London bank. The bank is asking its suppliers to take on the risk of system development. Nothing new there you might say, the ability to take on such risk is exactly why Logica, EDS, Accenture and co. exists and keep merging - you need a big company, with a big balance sheet to take on these risks.

But this bank had, as I understand it, three novel twists:

• The bank wanted the suppliers to take on the risk of errors made by the system after deployment. Not just the risk of development but the risk of using the system.

• The bank wanted the suppliers to accept unlimited liability - so an insurance policy wasn’t going to be enough.

• The bank was managing this project themselves and the suppliers were not Accenture or Logica or similar but IT contractors. Such contractors are individuals, trading as self-employed professionals or, more usually, limited companies with one employee.

Taken together what this means is: part of the operational risk of building an using the system is now taken by not the bank but by the IT contractors. So, on the bank doesn’t need to carry this on the “risk balance sheet”. If something goes wrong the contractors will pay up.

Except, the contractors are individuals. If the bank comes looking for £10m they aren’t going to be able to pay. The bank could get them declared bankrupt but it isn’t going to get its £10m.

So the bank isn’t off loading the risk, it is still taking the full risk but by a slight of hand it can claim it isn’t. This seems like a dangerous development, the bank are not only deluding themselves but are potentially deluding their investors and regulators about the risks they are taking on.

New training courses online

I’ve spent the last few weeks reviewing and revising the training courses I offer through my company, Software Strategy. The new courses are online and you can see them for yourselves on my website.

The unifying theme around these courses is the management of software development projects and products.

Not all the courses are fully developed yet so I have plenty of work to do. I would welcome feedback on these courses: are they the right thing? what needs adding? removing? and I’d especially like ideas for new courses.

(And I’d like it even more if you got in touch and asked me to deliver one!)

The courses are divided into three groups:
• Management of Agile software development
• Managing the development process and teams: this material can be used in Agile or non-Agile settings
• Design Patterns

As I complete the course design I’ll add the details and announce them here.

Please, let me know what you think, drop me an e-mail at allan@allankelly.net.

Agile failure modes: Potemkin Agile

One of the re-occuring themes in this blog is my attempt to capture “Agile failure modes”. For example:

• Failures in Agile processes?
• More Agile failure modes
• Agile failure modes - again
• And some in Reflections on XP Day

One I should have added a few months ago comes courtesy of Steve Freeman: Potemkin Agile.

Characteristics of Potemkin Agile include:
• “Show” teams in the organisation follow Agile but they are isolated examples and do not represent the organization as a whole (other teams are as disfunctional as ever)
• Teams who go through the motions of Agile without any real understanding
• Teams who do Agile, deliver software, get a big tick but the end product doesn’t deliver value

(I know Steve reads this blog so if I’ve missed anything, or misrepresented it I’m sure he’ll respond.)

The last one on this list is classic goal-displacement. People come to see “following the process” as the objective not adding value. Its been documented as happening in traditional development methodologies (specifically SSADM), I’m sure I’ve seen one project manager mistake following PRINCE 2 as success, I’m also sure it happens with Agile methods like XP and Scrum.

The more prescriptive the approach, the methodology, the greater your chance of this happening. My solution: don’t have a methodology, or at least, roll-your own. If you create your own your free to keep improving it.

That may sound radial and scary to some but it means you can focus on what you are really trying to achieve and then work out how to do it.

Q&A: Where do testers fit in?

I got the following question in my mail box a few days ago:

“I was reading the Powerpoint deck you prepared for the East Anglia Branch of the BCS and wondered what your thoughts were around Testing.

How should testers tackle verbal requirements – as you say Requirements shouldn’t be a document but Testers rely on such documentation as check lists for their testing?

I would be interested as I find myself in an ‘Agile’ environment and this issue takes up a lot of our time. Any advice would be great.”

I think this is an interesting question, and I think we often ignore the role of testers in Agile software development. This is for two reasons: first Agile came from developers, hence it ignores testers as it does Product Management. Second, Developers don’t really want to admit the need for Testers. Developers always think they can do it right and nobody needs to check on them. Call it arrogance if you like.

Consequently testers have problems like this. So I’m happy to try and suggest some answers.

(By the way, if anyone else out there reading this blog has a question please e-mail me the question, I’m more than happy to answer, especially if it gives me bloggable material.)

Short answer to this question is that Testers need to be involved in the same dialogue as the Developers and Product Managers. It is not so much the lack of a document that is the problem as unclear requirements, or requirements that are too specific, or Testers being cut out of the loop altogether.

People focus on the document because, despite its imperfections, it is the only thing that is available. The deeper problem is often that nobody pro-actively owns the business need. We somehow assume developers know best.

(I’m using the term Product Manager here. In most organizations this person goes by the name of is a Business Analyst, Product Manager, Product Owner (in Scrum) or even Project Manager. What ever they are called it is the person who decides what should be in, and how it should be.)

If we break the problem down it comes in three parts:
1. What needs testing?
2. How should it perform?
3. How should it be tested?

The planning/scheduling system should supply the “what to test” part. If you are using a visual tracking system what is being worked on should be clear, and it should be clear when it is ready for test. (I’ve described this in my Blue-White-Red process.)

If Software Testers are having problems knowing what to test it is usually a problem with the project tracking. In Blue-White-Red each piece of work is represented as a card, the card journey’s across the board an can’t finish its journey until a Tester approves. Testers might demand to see unit tests, see evidence of a code review or pair programming and may try the work themselves.

I think my correspondent is having a problem with the second question. After all, if you don’t have a written document how do you know how the software should perform? As I’ve said before, written documents are flawed - see Requirements are a Dialogue not a Document from last year.

Testers need to be part of this dialogue, they need to be sitting there when the Product Manager and Developer discuss the work so they can understand what is required at the end. If the work is contentious, or poorly understood, they may take notes and confirm these with the Product Manager later. They may also devise tests to tell when these requirements are met.

The Testers role is in part to close the loop, their role follows from the Product Manager role not the Developers role. They need to understand the problem the Product Manager has and which the developer is solving so they can check the problem is solved.

If Testers are having a problem knowing how something should be then it is probably a question of Product Ownership and Management. When this role is understaffed or done badly people take short-cuts and it becomes difficult to know how things should be. One of the reason many people like a signed-off document is that is ensures this process is done - or at least, it ensures something is done. (I’ll give an example later). But freezing things for too long reduces flexibility and inhibits change.

Finally, the “how should it be tested” question, well, that is the testers speciality. That is their domain, so they get to decide that.

That’s the basic answer but there is more that needs to be said - sorry for the long answer.

In my experience there are four levels of testing, most have more than one, two is the norm, having all four is probably overkill.

User Acceptence Testing (UAT)
Performed by Software Testers and Business Analysts, and/or Customer representatives. Formal test cycle using release candidates. Software is released with ‘release notes’ listing changes, additions, fixes, etc. Seldom found in ISVs (Independent Software Vendors) which produce shrink wrap software, conversely may be a regulatory requirement in environments like banking. Always conducted in dedicated UAT environment.

System / Integration Testing
Performed by Software Testers to ensure software works as expected when working with existing software and other systems. Usually, but not always, working with release candidates although these may not be as formal as in UAT. Usually conducted in dedicated system test environment.

Functional / Embedded / Close support Testing
Performed by Software Testers working closely with development teams. Tests any sort of build, usually from the (overnight) build environment, or potential release candidates, or even from developers machines. Helps to keep developers honest and bring testing skills and considerations right into the development team.

Developer / Unit Testing
Performed by developers on their own work before turinging over to anyone. Traditionally manual - perhaps using formal documents and “condition response” charts. Increasingly automated with tools like Junit and Aeryn.

Notes:
1. I’ve deliberately excluded regression testing here, although it is important it is not a level in it’s own right. It is an activity that happens within one or more of these levels.
2. Different organizations use different names and different descriptions for these levels. I’m not concerned what they are called, these are the levels I see.

My correspondent didn’t mention what sort of testing they were doing. If they are working at the UAT or System Test level they should be getting Release Notes from the development team to tell them what has been changed - not how it should work but what to look for.

One of the tenants of Agile software development is higher quality. If the code the developers produce is of higher quality then there is less rework to be done, so there is less disruption and overall productivity rises. This is the old Philip Crosby Quality is Free argument. Thus the overall approach is to inject more quality at the lower levels, i.e. Developer and Embedded Testing rather than UAT and System Test.

Good developers have always tested their own work. This has taken different forms over the years but is usually called Unit Testing. Nobody has ever recommended that developers slap some code down and pass it to testing. This might be what has happened sometimes but nobody has ever recommended it.

In compiled languages no developer would ever try to claim his work was finished until it at least compiled. And neither should they try to release it unless they have made some effort to ensure it does what it was supposed to.

What Agile software development does is suggest a set of practices - like TDD, pair-programming, code reviewing, etc. - which improves the developer testing side of the work in order to reduce the amount of work required later.

When this works code quality improves, Software Testers get to pass more work and raise fewer issues and the whole process works more smoothly. Testers can play a role here by keeping the developers honest, they should be asking: was the code reviewed? Have unit tests be written? Have the tests been added to the build scripts?

The pure-testing part of a Software Testers work will reduce over time thereby allowing them to focus on the really Quality Assurance aspect of their work. Unfortunately, in the software world the terms “Testing” and “Quality Assurance” are too often confused and exchanged.

So, as quality increases the amount of testing should reduce, and the amount of Quality Assurance increases - which again highlights the link between the Product Manager and the Tester.

One way in which Quality Assurance should work, and where Testers can help, is by defining, even quantifying, the acceptance criteria for a piece of work. Again this is picks up from Product Management.

For example:
In many “traditional” environments the Product Manager comes to the developers and says: “Make the system easier to use.” The Developers respond by putting in more menu items and other controls and the Tester is has to test the items exist and work.

What should happen is: the Product Manager comes to the Developers and Testers and says: “Users report the system is difficult and we would sell more if it was easier to use.” All three groups have a discussion about why the system is difficult to use, perhaps there is some research and user observation. Product Managers assess how many more sales there are to be had, Testers determine how they would measure “ease of use” - how they would know the system was easier to use - and the Developers come up with some options for making it easier.

Knowing how many more sales there are to be had determines how much effort is justified, and the test criteria help inform the selection of options.

As Agile development is introduced into an environment with legacy code it takes more time for this effect to come through. Even if Developers start producing higher quality code there is still a lot of legacy checks that need to be performed manually. Plus, as Developers productivity increases there is more to check.

The answer is that Testers need to bring in their own mechanisms for improving consistency and quality. This often takes the form of automated acceptance tests - perhaps using FIT or Selenium.

In the longer term quality all round should increase and - this is the scary bit for software testers - the need for testers should reduce.

If all software was perfect the need for Software Testers would go away. The presence of Software Testers shows that something is not right. However, the same could be said about many other industries which employ Testers but the reality is testing is a multi-billion pound industry.

So, should Software Tester be worried about their jobs when Agile software development comes along? - some are. I remember one Test Manager who like the ideas behind Agile but could bring himself to embrace it. He saw it as a threat to his job and to his people.

But No, I don’t think Software Testers should be worried about their jobs.

Even as quality increases there will be a need for some qualification that things are good. There are some things that still require human validation - perhaps a GUI design, or a page layout, or printer alignment.

In legacy systems it will be a long time before all code is unit tested, in fact I think it is unlikely to ever happen simply because it doesn’t need to. The Power Law tells us that only part of the system needs to be covered by automated tests. Still the rest of the system needs checking so there is work for Testers.

As Testers work more closely with Business Analysts and Product Managers their knowledge and insights into the business will become greater allowing them to add more value - or even more into Business Analysis or Product Management themselves if they wish.

And if this weren’t enough the success of Agile should make more work for everyone. Agile teams are more productive and add more business value. Therefore the organization will succeed, therefore there will be more business, therefore there will be more work to do.

The summary of the long answer is: Agile does mean change for Software Testers, it means putting more emphasis on quality and less on testing, I think it was Shigeo Shingo of Toyota who said there were two types of inspection [testing]:
• Inspection to find defects is waste [so should be eliminated]
• Inspection to prevent defects is essential.

Software Testers need to look reconsider their work as the prevention of defects, not the detection of defects.

I hope that answers the question. Without knowing more about the organization set up its hard to be more specific. And I apologise for the length, in Blaise Pascal fashion, I was working fast.

If anyone else has a question just ask!

Kanban: efficient or predictable, you decide

I have written a little about the Kanban method of software development in the past. And thanks to the comments from Karl and Wayne, plus a few on the side conversations with other people I have a better understanding of what is going on. (Although I’ve still to see it in action.)

Well after much thinking I’ve come to a conclusion, I’ve worked out why it is different to other Agile methods like XP, Scrum and my own Blue-White-Red. And this is is - drum role....

• The focus of most Agile methods is to bring order to the development process, one way they do this is by trading efficiency for predictability.
• Kanban - it seems to me - makes the trade in the opposite direction: predictability is reduced to increase efficiency.

I’m not saying Agile methods don’t care about efficiency, they do. And I’m not saying Kanban ignores predictability, it has an different method of delivering it. And I’m not saying Kanban isn’t part of the Agile/Lean family, it is.

But I am saying there is a trade off. And there has to be a trade off. Its not very nice to say but it is true.

It is true and can be proved mathematically through queuing theory. I can bore you silly with queuing theory but I’ll leave that to one side for now - if you want to know get in touch with me or read another site.

In summary queuing theory explains why in any production system you can have predictable deliveries up to 76% of utilisation. Once your system utilisation goes beyond 76% predicability is lost.

Just to be clear: You can have predicability or efficiency but not both.

The way most Agile processes work the highest priority items are more-or-less guaranteed to be done in the next time-boxed iteration, whether that be a week, two weeks, a month or what ever. Lower priority items might, or might not, get gone. But you can’t squeeze a pint into a quart pot, if you do bad things happen. People don’t sit around for 24% of the time but there is a buffer.

These processes reduce maximum theoretical throughput in order to increase predictability. There is nothing wrong with that, remember our 76% rule. Management might not like it but they are up against mathematics.

In Kanban the team remove (some of) the time-box constrains, planning processes and such that create predictability and go all out for efficiency. If you like they take a gamble, they aim to complete the work before there is need for predictability.

That in a but nut-shell is it. However Kanban does have one twist in the tail. As I understand it teams then apply statistical methods and observation to calculate the flow through the system. This can return some degree of predictability.

One more thing to note. All Agile methods, including Kanban and Lean are actually more efficient than traditional methods for another reason: quality improvement leads to less (re)work.

All Agile-family methods place a high emphasis on avoiding re-work, they are prepared to invest more in up-front quality in order to reduce tail end (bug) fixing. The net effect is to reduce the total amount of work. So, while Scrum and Blue-White-Red may trade some efficiency for predictability they do so and are still be more efficient than traditional approaches.

So it all comes down to choosing what is more important: out right efficiency or predictability. Then set up in your process appropriately.

Having written these words I can already hear senior managers responding: but we want both efficiency and predictability, why should we settle for one or the other? We need both, we are different.

Managers get brought up on these kind of contradictions: they are taught “the magic of and” to implement “simultaneously tight and loose controls” and so on. In many cases these are good ideas and putting the contradictions together leads to creativity but in this case it won’t end.

Still we all live in the real world so if you really need efficiency and predictability you can have it. The magic ingredient is illusion of control. Just d this and all things are possible - especially those “when will it be ready” and “why can’t IT deliver?” conversations.