BA’s “global IT system failure” highlights the importance of disaster recovery testing

The weekend’s ‘global IT system failure’ will cost British Airways dearly – both financially and in terms of reputation. With aircraft and crew in the wrong places, connections lost, luggage missing and so on, the chaotic ripple effect of BA’s failed IT backup and disaster recovery solution will probably take weeks to resolve. But could it all have been avoided if BA regularly tested their backup and disaster recovery solution?

The disruption affected 75,000 people, with hundreds of flights cancelled from Heathrow and Gatwick, and delays continuing for days. And it’s not the first time that BA customers have suffered as a result of an IT outage. Last September, thousands faced lengthy check-ins, delayed flights and cancellations at London City airport after an overnight IT glitch.

The cost of BA’s IT system failure

On the first day of the crisis, Alex Cruz, British Airways chairman and CEO said: “Today we have experienced a major IT system failure that is causing very severe disruption to our flight operations worldwide. All of our check-in and operation systems have been affected.”

While acknowledging that it’s too early to quantify the cost of BA of this IT outage, the Financial Times (FT) puts the potential financial impact in context, by comparing it to a similar problem Delta, the US airline, experienced last September, which triggered them to cut their profitability guidance for the third quarter. Delta then reckoned knock-on effects of the IT outage would cut pre-tax income by $150 million (£117 million).

So where did BA go wrong?

Back up and disaster recovery – how it works

At the time of publishing, BA is still investigating what caused the power supply problem, which triggered the outage – and what happened with its backup-system, which should’ve kicked in immediately, ensuring failover for near 100% business continuity. Unfortunately for 75,000 Bank Holiday weekend travellers, this was not the case.

Normally, this is how it would work. A business will write a business continuity plan, which will include an IT disaster recovery plan. The very process of doing this will highlight flaws in your IT and processes that need addressing.

For example, consider these 10 questions on disaster recovery planning every manager must ask. You may discover that your existing backup and disaster recovery solutions don’t fully comply with your industry regulations. The Financial Conduct Authority (FCA), for example, has stringent rules surrounding business continuity of banks.

Or perhaps, like Crondall Energy, you realise that hard copy tape backups just aren’t practical or secure enough for your business, and you should think about cloud backup. Or, like BA, the financial and reputational costs of business downtime might just be too much to risk not having a reliable disaster recovery solution in place.

Test, test, test: the importance of testing your disaster recovery plans

But it doesn’t stop there. It’s one thing having a top-notch disaster recovery plan in place. It’s entirely another thing to make sure that it works – and continues to work as your business and IT systems and processes grow or morph. And that’s where testing is business critical. After all, what’s the point in paying for disaster recovery if it’s not going to ensure business continuity?

Four key repercussions of failing to test your business continuity and disaster recovery are:

  1. You won’t achieve fast or reliable recovery
  2. You won’t be able to read your data
  3. Your initial test is likely to require hours of specialist engineering
  4. Infrastructure changes may render new devices or applications unworkable

We don’t yet know how many of these BA suffered, but I’m guessing all four. You can read more here.

Why choose cloud backup and disaster recovery?

Here are our top 10 reasons for choosing cloud backup and disaster recovery.

  1. Get secure, offsite cloud backup
  2. Free your business from manual and complex tape backup tasks
  3. Get predictable costs and simpler budgeting
  4. Count on reliable, guaranteed data recovery
  5. Minimise the risk and cost of downtime
  6. Restore data – fast
  7. Take advantage of specialists’ expertise and resources
  8. Offload regulatory compliance requirements to the service provider
  9. Rely on a well-managed cloud that is more secure than your own data centre
  10. Count on easy set-up and use. ‘Set-and-forget’, with no training required.

You can read the blog in full here – or go to this page to watch a two-minute video.

Click here for your business continuity pack, including a free template on ‘How to write a Business Continuity plan’.