The rise in cyberattacks has resulted in a seemingly endless litany of problems across all business types. Some of these problems are easily quantified but many are not. In the latter category, the costs of downtime stand out. Not only do cyberattacks take security analysts away from productive work and force them to deal with exigent problems, but they also disrupt the work of other professionals who rely on technological infrastructure to execute their jobs.
But it has proven difficult to assess the costs of downtime in a consistent manner. The academic literature on the subject is somewhat scant, so we must rely on reports from private businesses — that are understandably reluctant to share the financial impacts of these events unless required to do so. As a result, the figures are all over the board, varying according to industry type, length of downtime, and a suite of other factors.
What we do know is that these attacks are unlikely to abate anytime soon. Roughly 50% of organizations experienced more than 24 hours of downtime between 2014 and 2019. And large organizations are far from the only targets. Small and medium-sized businesses (SMBs) are increasingly on the radar of cyberattackers as well. Healthcare and non-finance type businesses have suffered more than three quarters of the brunt of this onslaught.
More than 80% of manufacturing companies have reported downtime in the previous three years according to a 2023 report, for example. But other types of organizations are hardly immune — over 40% of companies that use cloud services are seeing four types of attacks on the applications used to run their businesses on a weekly basis.
Here, InformationWeek explores the nature of downtime and how it can be mitigated with Michael Dickman, chief product officer at Gigamon, a company focused on deep observability; Corey Hynes, executive chairman and cofounder of Skillable, a cybersecurity training platform; and Charity Majors, chief technology officer and cofounder of Honeycomb, which also focuses on observability.
What Constitutes Downtime and What Causes It?
The definition of downtime is somewhat ambiguous. Generally, it refers to any time not spent on productive work as a result of a cyberattack — by both cyber analysts and other members of an organization. Thus, it can extend to nearly every aspect of operation. Some 78% of organizations report breaches as the main source of downtime.
However, sometimes downtime is also scheduled for maintenance or training purposes and may occur as the result of internal errors, power outages or disasters. It may also include time spent repairing systems and equipment that have failed on their own, either due to poor planning or to unforeseen circumstances.
For cybersecurity analysts, downtime includes time spent assessing the cause of a breach, containing the breach and recovering from its effects. According to incident management software company Blameless, responding to a single incident, even a minor one, may take up to eight hours.
For other professionals, it includes lost time managing operations, contacting clients and making sales, planning for future business, manufacturing goods and any number of other tasks that lead to profitable business enabled by smoothly functioning, secure technology. Regular business may grind to a halt if, for example, a payment system or software used to guide production are compromised. Cincinnati Crane and Hoist, a small crane manufacturer, was severely impacted by a spearfishing campaign that targeted their email and payment system in 2017, for example, losing a quarter of a million dollars in revenue and resulting in the company scrambling to save its operations rather than spending time on producing equipment. Production at Molson Coors, a prominent American brewery, ground to a halt for weeks in 2021 following a cyberattack.
Michael Dickman, Gigamon
Downtime is sometimes unpreventable, as in the case of power outages or natural disasters. But when downtime occurs due to a cybersecurity breach, it is typically attributable to failed cybersecurity procedures and deficient business continuity plans. Even very basic cybersecurity training for employees can help reduce the likelihood of falling for common tactics such as phishing scams. So too, failure to implement a business continuity plan that establishes back-up procedures can result in companies scrambling when an event occurs. Just over half of companies have a disaster recovery plan in place according to one 2021 survey.
“Of course, staff are diverted to focus on the immediate issue, but often the first question is where exactly is the attack happening, and what exactly is impacted? Being able to observe — with confidence — the answers can help speed up response, not only by quarantining and remediating but also by guiding decisions on whether or where to proactively shut any systems down,” Dickman says, emphasizing the importance of preparing for such events.
Average Costs of Downtime
The costs of downtime are difficult to estimate in a general sense. They are highly specific to business types and how reliant they are on digital systems. And there is no standard means of assessing these costs.
Below is a table of downtime cost estimates from various sources. This patchwork of data underscores the need for more extensive information sharing among industries and deeper academic work on the issue:
Other staggering statistics have emerged: some estimate that manufacturers lose $50 billion a year due to downtime. If even a fraction of that cost is due to cybersecurity breaches, clearly greater consideration ought to be given to downtime planning specific to hacking incidents.
Costs range across all aspects of the business, from manufacture and sales of products to lost administrative time to effort spent tracking down new clients and new employees to replace those who fled during the chaos. And stressed organizations whose cybersecurity staff are occupied with one crisis may be vulnerable to further attacks — leading to additional downtime.
What Downtime Looks Like on the Ground
During periods of stability, cybersecurity analysts are on the offensive.
“There are different ways to categorize an analyst’s time. In this context, we can think about time spent on active protection, proactive security preparation, and reactive response and remediation. ‘An ounce of prevention is worth a pound of cure’ is true in cybersecurity, so proactive initiatives are critical for the success and sustainability of a cyber program,” Dickman says.
When they are on the defensive in the wake of an attack, these responsibilities become secondary. Work on building defenses against potential attacks is suspended — that is, if a dedicated team is not available to man the ramparts while others head to the battlefield.
Corey Hynes, Skillable
“Responding to cyberattacks is different from most other incidents. Time matters. Like any critical response team, cyber teams must train not until they get it right, but until they can’t get it wrong,” Hynes says. “The tools, procedures, and practices employed when an attack occurs have to become akin to muscle memory.”
Other departments will be similarly distracted from their typical duties. Customer service representatives will be diverted from fielding everyday complaints and inquiries to doing damage control for hordes of unhappy customers. Finance professionals will be pulled from their bookkeeping duties and asked to investigate potentially compromised customer accounts. Public relations officials will turn from promoting the company to stanching the reputational damage that usually occurs when breaches become public. And the C-suite will pivot from making deals and managing operations to reassuring client representatives and board members.
The Parable of Hospital Downtime
Downtime in hospitals is among the better studied scenarios in the academic literature. Many hospitals are also research institutions, and the cost of downtime is not just financial — it can result in healthcare consequences, even lost lives. Further, hospitals are prime targets for cyberattackers, with some 166 hospitals attacked between 2012 and 2018 alone, resulting in an estimated 701 days of downtime. Healthcare institutions are thus incentivized to share their messy, ad hoc procedures — and the lessons they learned from them — in ways that some private industries are not.
Due to their heavy reliance on digital recordkeeping, these organizations must devise radically different means of communication if those systems are affected by an attack. Even when outages are not complete, they may pose a danger to patients. For example, if clinical decision support mechanisms in electronic health care records are compromised, treatments that might otherwise be flagged as potentially dangerous are implemented anyway.
As the digital generation comes of age and enters the working world, many professionals are unfamiliar with paper record keeping and non-digital means of communication, presenting an added challenge in devising downtime procedures. During one outage, physicians had to hand-write prescriptions and may have compromised patient safety due to their lack of familiarity with how to correctly indicate drugs and dosages.
Developing templates in case of such outages will likely be useful. One hospital pathology lab did so on the fly following an attack, which proved helpful, but the approach would have been more efficient if prepared ahead of time. Staffing had to be doubled to implement the approach, another consideration for downtime planning extending to budgeting and staff management.
During a 2017 downtime event at an Australian hospital due to the WannaCry attack, staff used a combination of email, text messaging, in-person meetings, paper records and public announcements via the loudspeaker system.
“Familiarity with medications and all of the things that are associated with clinical care is just so much easier with the electronic record and people become used to that. You don’t realize how automated things are now with the electronic record and the checking and all those things that go on within the record [that] are not available in the paper system,” one of the staff members interviewed by the researchers said.
While staffers were initially hesitant to use the PA system due to fears of disturbing patients and their families, announcements turned out to be key to ensuring smooth communication. In person meetings were thought by most interview subjects to be the most effective means of transmitting information. However, other research emphasizes that written communication is essential — verbal directives cannot always be verified due to the busy nature of the hospital system. Text messages protocols, runners to transfer paperwork, and even the use of large white boards have been used to ensure the accurate transmission of information.
How To Plan For and Deal with Downtime
Having some form of business continuity and disaster recovery (BCDR) plan in place is probably the best prophylactic against major downtime costs. More than 90% of managed service providers report that such plans are likely to reduce downtime according to one report. A study of healthcare providers indicates that incident response plans may reduce downtime by as much as 48% a month.
“Your systems exist in a continuous state of partial degradation, and you are positively swimming in failures,” Majors notes. All eventualities, from minor disruptions to full shutdowns, need to be planned for if downtime is to be used efficiently.
Charity Majors, Honeycomb
“IT leaders and other business decision makers must think critically about their support teams, identifying and encouraging continual upskilling via real-world scenarios to mirror the threats they’re likely to experience,” Hynes advises. “Staying skilled in parallel to increasingly complex and intelligent cyberattacks can make recovery more efficient and alleviate unnecessary downtime that puts the company reputation and stakeholder relationships at risk.”
This will often necessitate de-siloing an organization. As one paper observes, cybersecurity analysts are sometimes not looped into continuity plans, making those plans next to worthless when something actually happens. Conversely, analysts do not necessarily share the findings of their risk assessments with the necessary departments. So, nobody can plan accordingly.
As previously referenced, planning for alternate means of communication, whether it be in a hospital or in another business, is crucial. Ensuring that an immediate fallback to typical communication channels is in place will almost assuredly save time in the event of an attack.
Once plans are in place, they need to be tested to assess potential weaknesses. But planned downtime is a luxury most organizations can no longer afford, Majors says.
“Once upon a time, we had ‘planned downtimes’ — we’d take the system offline for a night while we repaired a drive or repointed the primary,” she recalls. “Nobody thinks this is acceptable anymore. The frequency and duration of our total outages have been steadily dropping for years. But this doesn’t mean things don’t break.”
“As the saying goes, ‘If it hurts, do it more.’ If it hurts, and is hard, and takes a long time to deploy some code, you should do it again and again and again until it’s fast and easy,” she adds. “When it comes to software, speed is safety, much like riding a bike or ice skating. Anything that gets done every day will be easy and effortless. Anything that you do only once a year under extreme duress is going to hurt like hell.”
Procedures for securing the environment following an attack need to be put into place as well to ensure that downtime is not wasted during recovery — it will be useless if another attack is just over the horizon.
The National Institute of Standards and Technology provides useful guidance with its Cybersecurity Framework. Of the five categories, identify, protect, detect, respond, and recover, Dickman says some are overlooked.
“Identify, detect, and recover have too often been neglected, which makes protect less successful, and respond slower and less complete. One important improvement that helps all five of these functions is creating the infrastructure ahead of time to observe the movement of data and the interactions of users and applications, in a way that is immutable and complete,” he claims. “Having this kind of deep observability will Identify threats faster, inform better Protection policies, Detect a greater proportion of threats, focus Response faster and in the right place, and give confidence that Recovery is safe and successful.”
