Why CrowdStrike Update Caused Unprecedented Microsoft Outage: ThreatLocker CEO
Danny Jenkins—whose company, ThreatLocker, secures the unaffected JetBlue Airways—tells CRN that the seemingly minor nature of the CrowdStrike update was responsible for the sudden widespread impact.
The seemingly minor, routine nature of a CrowdStrike Falcon update was a key factor behind the sudden widespread impact from the Microsoft outage caused by the defective update, according to ThreatLocker CEO Danny Jenkins.
The outage caused massive, unprecedented impacts worldwide Friday, including the cancellation of many medical procedures and flights for multiple airlines.
[Related: Analysis: CrowdStrike Isn’t The Only One To Blame Here]
ThreatLocker has been involved in responding to the outage because some of its customers use both its endpoint security tool and CrowdStrike, Jenkins said. Customers of ThreatLocker that do not use CrowdStrike, such as JetBlue Airways, were not affected, he noted.
Ironically, the fact that the CrowdStrike Falcon update was a minor “content” update rather than a full software patch was likely a reason that this outage was so widespread. Normally, full patches involving installation of a whole new software version would be “staged”—in other words, deployed on a staggered basis starting with beta and early adopter users, Jenkins told CRN.
“Because this is not a full software update, it’s not staged,” he said.
Instead, Jenkins said, this was an update to CrowdStrike Falcon likely targeted at protecting customers against newly discovered cyberthreats, which is a frequent type of update for an endpoint security tool.
To keep customers protected, CrowdStrike “wants to push those threat updates instantly, to as many people as possible,” he said.
As a result, a high proportion of CrowdStrike customers were likely to be affected by the update, which the company has said contained an unspecified “defect” for the Windows version of Falcon. CrowdStrike has reported having nearly 30,000 customers, many of which are large enterprises.
“We see software updates cause chaos all the time, but not at this kind of scale because normally they’re staged,” Jenkins told CRN. “Obviously, no one’s installing a [software] patch automatically on a server. But when you think about a threat update, they’re being pushed as fast as possible.”
As a result, “this is the biggest cyber incident in history,” Jenkins said, even though it did not involve a cyberattack.
‘Content Update’
In a statement provided to CRN Friday, CrowdStrike noted that “there was an issue with a Falcon content update for Windows Hosts.” MacOS and Linux users were not impacted, the company said.
“We understand the gravity of the situation and are deeply sorry for the inconvenience and disruption,” CrowdStrike said in the statement.
Jenkins noted that about 22,000 endpoints that were using both ThreatLocker and CrowdStrike were impacted, but nearly all of those have been recovered at this point by ThreatLocker personnel.
There are two different possibilities for why the Windows blue-screen issue occurred as a result of the defective CrowdStrike update, he said, though it’s not clear at this point which is most likely. “It’s really most likely going to be a bad signature in there, or the file just wasn’t readable,” Jenkins said.
Recovery Underway
Speaking with the Today show Friday, CrowdStrike CEO George Kurtz said that the issue has been fixed but “it could be some time” before a full recovery is possible.
The defective CrowdStrike software update led to impacts on real-world services including more than 1,000 flights reportedly canceled, health-care services such as surgeries curtailed and 911 system outages.
“We’re deeply sorry for the impact that we’ve caused to customers, to travelers, to anyone affected by this,” Kurtz said.
“As you might imagine, we’ve been with our customers all night and working with them,” he said. “Many of the customers are rebooting the system, and it’s coming up and it’ll be operational.”
Multiple airlines reportedly asked the Federal Aviation Administration for a full ground stop on flights amid the outage. Flights from American Airlines, United and Delta were the hardest hit, according to reports. Nearly 1,400 flights have reportedly been canceled globally.
A number of U.S. hospitals have curtailed services, including elective surgeries and other non-urgent visits Friday as a result of the outage. Mass General Brigham in Boston said in a post on X that “all previously scheduled non-urgent surgeries, procedures and medical visits are canceled today.”
Meanwhile, impacts to 911 systems in states including Arizona, New Hampshire and Alaska were reported.