Banking systems stink - the pain of an botched HSBC release

I’ve had a very frustrating morning, and it all comes down the failure of bank IT systems - particularly a software release failure at HSBC. Gee, their systems are flaky.

I was trying to buy tickets at Eurostar.com for my trip to Lean Kanban France in a few weeks. First I tried to use my American Express card (I hate it, I only use it for work expenses) and I got the now traditional “Service not available -Unhandled Java exception …gavity.exceptions…” at the final payment stage - you know the pocky little box you have to put three letters into? - I’ve had this error before elsewhere and I think its because I block pop-ups on Firefox. Annoying but nobody at AMEX care so that is that.

So I repeated the transaction using Chrome and my Mastercard - issued by HSBC but branded John Lewis.

And again at the final security page - Arcot’s 3D Verified by Mastercard bit - the page errored. Not a horrible unhandled Java exception but an error just the same.

So I called Eurostar - because something similar happened a few weeks ago and they took my booking by phone. Only this time they insisted that it was the banks fault - like two banks fail? Didn’t anyone ever tell them “Select isn’t broken”?

This time they insisted the problem was with the banks, to make a booking over the phone would cost me £10 - something they waived last month. I even went as far as to speak to a supervisor who continued to insist he wouldn’t waive the fee - which I know from last month he has the ability to do.

Instead he told me it was probably my Mac that was the problem.

Seriously, he told me to use a phone instead. 10 years ago saying “don’t use a Mac” might have been acceptable but now?

Bad customer care on both points.

I gave up, I fell back on Safari and another card altogether to make the booking.

After several tries and a long wait I finally got to speak to HSBC Mastercard and this is what I discovered…

• HSBC Mastercard don’t currently use Arcot security but intended to start on 9 November
• But the release has been pulled because of technical difficulties, they don’t know when it will occur
• But the message had gone out to retails that they were to start (remember, today is 1 November so they change wasn’t scheduled for 7 more days)

This also explains a couple of other recent online purchase problems I’ve had with this card.

This implies that either:

• When a seller system (e.g. Eurostar) calls the HSBC system (“is_using_arcot_security()”) the HSBC system is replying “True” when it should say “False”

Or

• There is a piece of system configuration that HSBC publishes to sellers (“<is_using_arcot_security>True</is_using_arcot_security>”) which has been pushed out prior to the update and that a) this config has been released too early (its not 9 November yet!) or b) else has not been rolled back.

Either way HSBC have bungled a software release. Of course, it might not be HSBC, it could be Arcot, or rather their current owner CA - who were fingered in the RBS outage a couple of years ago.

Which also might explain why I spent over 30 minutes on hold to the call centre (plenty of failure demand).

And we still haven’t discussed by HSBC pulled their release in the first place, clearly something went wrong to cancel the 9 November release - my money is on low quality.

When will these companies learn: high quality in software is faster and cheaper.

Continuous delivery is about risk reduction.

(Well, they are probably mainframe systems so maybe I should be a little forgiving on the second point.)

Now someone at HSBC will probably get into trouble for sharing so much information with me. Well its a good thing they did because I wasn’t going to hang up until they did. Thank you, you did the right thing - unlike Eurostar.

I write this both to relieve my frustration and in the hope that it might make HSBC (and AMEX) do something about this.

(I should also point out: none of these companies have ever hired me. If they had I’d probably keep quite because I avoid blogging to specifically about clients.)

But you know the really worrying thing about all this:

Our modern banking system is based on the assumption that the Government will always bail out a failing bank - the fear of the payments system failing is so great governments will do anything, thats why we rescued RBS.

Now it seems banks are failing to maintain the payments system anyway. The banks are collecting technical debt the way they used to collect sub-prime. Central bankers throwing money around won’t work.

Who will save us this time?