Pessimistic about Agile in retail banks

Almost exactly two years ago I wrote a strident blog entitled “Agile will never work in Investment banks”. Its a blog post that keeps getting rediscovered by those who agree or disagree with my position and just such event has occurred on Twitter with Kevin Burns @martinburnsuk and several others @kev_austin @sandromancuso @gordonmcmahon 

During the intervening two years I have had many conversations which support my point of view. Most of these conversations have been with people who have more, and more recent, experience in banks than I do. In particular I found Peter Pilgrim’s blog closely aligned with my own views.

Conversations with the likes of Andy and Chris (you know who you are!) and others have led me to believe that while retail banking might be more hopeful it isn’t a massively better there. In part that is due to the colonisation of retail banking by investment banking that has occurred over the last twenty years.

Lets be clear: although investment banking and retail banking share the word “banking” in their title they have nothing else in common. I’ll leave it to John Kay to say something about the why they are in conflict.

The move by many investment banks to occupy retail space has been purely opportunistic, they have sought cheap funds and to embed themselves in the economic fabric so they become “too big to fail” and “systematically important institutions”, thereby guaranteeing Government bail out, thereby reducing their funding costs and increasing banking bonuses. (See The Bankers New Clothes for a discussion of the many problems with current banks and solutions.)

If you wanted to summarise my argument in one line I’d say: Too big to fail means too big to manage.

Now there are two caveats on my original post which I want to make clear before expanding on the discussion:

1. Agile is interpreted by many in banking to mean Iterative Development - see my recent “3 Styles of Agile” post. This can work to some degree. Simply adding the technical practices, improving software craftsmanship, can improve IT. However if an Agile ever becomes more than - incremental or evolutionary - there then conflicts will be created within the management regime which may lead to its own destruction.
2. Incremental and Evolutionary Agile can exist in banking, and have existed in many banks I know of but only in the short to medium term. This kind of Agile requires a guardian angel. Again this will create conflicts, while the guardian angel is strong enough the team can be protected. However given time either the angel is successful and will presented with a better opportunity (probably at another bank in which case he might have the team follow him) or he is not successful and will be pushed to one side and his creation will be dismantled.

Turning specifically to retail banks, and thereby extending my argument. On the one hand I don’t believe retail banks suffer with quite the same rent-seeking culture found in investment banks so I’m more hopeful of them. But on the other hand retail banks have an additional set of problems:

Legacy retails banks are dependent on Cobol mainframe systems. This brings several problems. Firstly while you can do some Agile like things in a Cobol environment there are many things you can’t do.

Specifically TDD: as far as I can tell several attempts have been made to create a CobolUnit (this CobolUnit may be the same to different) and they are all pretty much dead. No action since 2009 or 2010 on those links.

We can’t expect the OpenSource community to come to the rescue here. Few people write Cobol at home and the generation of people who write Cobol are different to the OpenSourcers who created JUnit, nUnit, etc.

Unit IBM or MicroFocus get interested in a CobolUnit then I don’t expect much to happen - it seems IBM might be moving in this direction. But generally I don’t see the incentive for either unless they can see money.

Second these Cobol systems are ageing. They have patch upon patch applied to them and by some accounts are reaching the end of their lives. Much of the technology they are built with just isn’t as flexible or modifiable as the current generation. There is only so much you can do with a hierarchical IMS database or a mainframe batch processing system. (If you could do these things, and you could do them as easily as modern technology then we wouldn’t have needed to invent modern technology.)

Since retail bankers were early adopters of computers some of these systems are among the oldest systems in use. Many of them have been patched together through mergers and acquisitions making things even more difficult.

True I’d a great proponent of never rewrite and I think it would be a mistake for banks to rewrite these systems. Indeed, given the chronically poor management skills in these banks authorising a rewrite would be a waste of money. Which means you are damned if you do, damned if you don’t.

Third, compounding the first two: the people who wrote these systems are nearing retirement age. Not only do they understand the technology but they understand the business too. When they are gone these systems are lost.

Banks could have alleviated this problem by training replacements but they don’t generally do this. Actually they have made things worse in the last 20 years by outsourcing and offshoring work thereby discarding their own experience and acerbating the generation change. Some of the knowledge might now reside in the outsourced and offshore but getting this into the bank will prove difficult and expensive.

The solution for the banks is to buy off-the-shelf systems. But this is incredibly difficult so...

Fourth, buying an off the shelf product means not only process changes but potentially product changes. That lands the retail bank with change problems. They will need to customise off the shelf software and that isn’t cheap either.

Fifth, new “challenger” banks like Metro bank are already buying off the shelf. As the costs of maintaining legacy systems or migrating to new systems escalate these banks will have a greater and greater advantage over the legacy banks.

And its not just challenger banks: Paypal, Zopa, Funding Circle and a host of other online payment and loans providers are applying technology to reinvent banking. In doing so they are eating the retail banks market. (And the banks can’t respond because of the reasons listed here.) In other words: the banks are suffering from disruptive innovation

Sixth - something I just hinted at - the quality of IT management in banks is absolutely appalling. Again investment banks have made retail banks worse because the big money was in investment, anyone who was any good and wanted more money had an incentive to move from the retail to the investment side. Retail IT has been robbed of its best staff.

Management quality is not entirely down to investment banks. Banks are about money management after all. Promotion to the senior positions comes from the banking side, not the IT side. Almost by definition the senior managers in banks don’t understand IT. It also means that IT managers sooner or later face a ceiling on progression.

Many of these problems have been made worse because investment bankers controlled retail banks. Investment banking is essentially a rent extracting activity, and they have managed retail IT just like that. The existing IT has been milked and without adequate investment. Hence problems like well the documented RBS outages and Lloyds payment problems.

To make matters worse banks are where the money is so they have been preyed on by major systems companies and tool vendors. These companies sell technology fixes which may - or may no - address a problem but do little if anything to build capacity. Put it this way: banks are major buyers of automated testing tools which aren’t used.

You see my argument?

I’m sure Agile could help with some of these problems - not all but some - but the way I see it the problems facing the banks mean Agile will have a very hard time.

Finally, as others have pointed out many of my arguments can be applied to other types of large corporations. I agree, I confine my arguments to banks because I think the case is clearer there, banks are all very similar, because banks exhibit more systematic failures than most and banks are something we should be worried about - as customers, as tax payers, as citizens in economies that depend on banks.