CyXcel: Lessons learnt from the CrowdStrike outage
October 2024 by Jack Horlock, Principal Associate specialising in cyber risk and incident response at CyXcel
The Financial Conduct Authority has published a report today which provides insights, observations and key lessons from how firms responded to the CrowdStrike outage and their preparedness to respond to future incidents.
Jack Horlock, Principal Associate specialising in cyber risk and incident response at CyXcel, a leading cyber security consultancy combining security, regulatory and legal expertise, and Anthony Rance, a Partner at CyXcel who specialises in technology, data and cyber disputes, have commented on this further and have provided insight into why businesses should continue to protect themselves legally when it comes to contractual positions with suppliers and why it is important to understand your security infrastructure.
Jack Horlock, Principal Associate specialising in cyber risk and incident response at CyXcel, comments,
"This incident was headline-grabbing, and understandably so. One of the largest cloud hosting providers and one of the largest cybersecurity providers had gone down, creating a cross-sector shutdown. How? A faulty update was pushed out to user-end devices. Why? The error in the update evaded CrowdStrike’s validation and testing process.
"Although the impact of the incident was felt across the globe, the reality is that this could have been much worse. The consequence of the faulty update was that servers and endpoints “failed closed” – i.e. they became inaccessible rather than exposing or opening a vulnerability. Therefore, whilst warnings about scammers seizing the moment and capitalising on the chaos were necessary and timely, there was no question of malicious compromise. The fix came relatively quickly. There was variance, though, in different organisations’ efficiency and speed of recovery: those who had tested recovery plans and up-to-date infrastructure recovered quickly. Those who didn’t, didn’t.
"Businesses are increasingly reliant on a patchwork of suppliers and service providers: their supply chain. Organisational risk, for some time now, hasn’t been a question of what goes on just within the four walls of the company, but also a question of transfer and management of risk outside the bounds of a single organisation. Regulations across multiple jurisdictions are being updated to reflect the significance of supply chain risks because of exactly that: a chain is only as strong as its weakest link. Organisations must have a clear view of their suppliers and service providers which includes not just who is doing what, but how they are doing it, what the consequences are should there be a failure by the supplier, and how the organisation will respond in that event."
Anthony Rance, a partner at CyXcel who specialises in technology, data and cyber disputes adds:
"Whilst the precise legal fallout from the incident remains unclear, it seems likely that CrowdStrike will face increased litigation from contractual counterparties. Delta Airlines – one of the significantly impacted organisations, and the first to surface with a public dispute – has complained of a “backdoor” which allowed this sensor update to be pushed through despite Delta claiming to have turned off auto updates. CrowdStrike explain that Delta’s position is based on misinformation and lay the blame for Delta’s losses at Delta’s business recovery strategy. In either case the important question for organisations is: do you know who is doing what on your systems? If not, find out.
"There are lessons to be learned across the piece following this incident. Providers: test your products and updates on a “real life” basis. Users: understand your contractual position with all suppliers. Cybersecurity is not a “one and done” but requires constant and regular attention. Business recovery plans need to be tested and kept up to date. Regulators, customers, and insurers will expect organisations to know and understand their security infrastructure."