Novel Technologies Bolster Cybersecurity at Water Treatment Plants

At 8 a.m. on Friday, Feb. 5, a municipal water plant operator in Oldsmar, Florida, noticed the city’s water treatment control system was being accessed remotely. Assuming it was his supervisor, the operator saw no cause for alarm, until a few hours later when he witnessed the cursor moving across the screen of its own accord and adjusting the level of sodium hydroxide, or lye, to more than one hundred times what it should be.

Targeting public infrastructure, including water systems as observed in Oldsmar, is not an unfamiliar threat. The fallout from the recent Colonial Pipeline cyberattack has been a testament to the persisting risk.

Nor are these new threats to researchers at the Johns Hopkins Applied Physics Laboratory (APL), which — recognizing the vulnerability of public infrastructure — stood up the Critical Infrastructure Protection Group six years ago to study these issues and think through possible solutions.

“We see these types of threats to what is essentially invisible infrastructure — the things we take for granted,” said Tao Jen, the group’s supervisor. “You turn on the lights, the lights come on. You turn on the faucet, the water comes on. And you assume everything is going to be good.”

But what if everything wasn’t good? In the case of the water treatment plant incident in Florida, had the altered chemical levels not been caught, the attack could have resulted in a string of major ramifications.

To prevent such incidents, APL is designing cost-effective cyberdefense technologies that are strong enough to repel serious cyberattacks but affordable enough to be purchased and easily implemented by the nation’s diverse water treatment plant operators. Many plants across the country are municipal operations plants with constrained budgets and — sometimes — an IT department of one.

The water treatment testbed in the CYber Physical REsilient Systems Solutions (CYPRESS) laboratory at APL was used for a Department of Homeland Security-funded pilot of APL-designed and -developed research technologies to demonstrate a Resilient Industrial Control System (ICS). The results of the pilot, and other demonstrations of the Resilient ICS, found that the APL technologies were successful in prevention, detection and mitigation of cyberattacks on industrial control systems.

“In testing and demonstrations without these defensive technologies, the cyberattack works,” said David Halla, a cybersecurity engineer who manages the Homeland Integrated Cyber Operations Program in APL’s Asymmetric Operations Sector. “But when we turn these defensive technologies on, they prevent the attack from happening. Had the operator in Florida not seen or been able to see the chemical levels one hundred times greater than normal, our technologies would have blocked the attack anyway.”

“An attack on water treatment facilities does more than stop the water at your tap,” explained project manager and systems engineer Lauren Eisenberg Davis of APL. “There’s a domino effect, including no drinking water, surface water contamination, agriculture crop loss, loss of ecosystem protection, economic and financial impacts, public health risks such as cholera and dysentery, loss of basic sanitation and even loss of fire protection.”

The possible domino effect of cyberattacks poses a risk not only to civilian infrastructure, including buildings, schools, hospitals and governments, but also to the military infrastructure that relies on power and water.

The technologies implemented in the CYPRESS water treatment testbed include:

• Out-of-Band over Existing Communication (OBEC): Detects whether an adversary has changed the values, even when the expected values continue to be displayed to the operator
• Network Deception and Response Toolkit (Network DART): Diverts an intruder to a high-quality decoy, protecting critical equipment while gathering intelligence about that intruder
• Mitigating Incidents with Mock Industrial Control Systems (MIMICS): Transfers control of critical processes from an industrial controller to a virtual instance to maintain continuity of operations if the industrial control system is attacked

“The technologies cover a broad range of cyber threats and attacks on control system operational technology,” Davis said. “They can be combined with open-source tools to provide a robust, resilient approach to ICS cybersecurity. All of these resilient ICS technologies are high-Technology Readiness Level and low-cost, and have been integrated into other testbeds, including a smart power grid and chiller testbed.”

APL is continuing to closely collaborate with government sponsors on further applicable safeguards and is focused on sharing its technologies and approach with the wider community — which included delivering a presentation at this year’s Industrial Control System Joint Working Group (ICSJWG) conference hosted by the Cybersecurity and Infrastructure Security Agency at the end of April.

“Broadly championing these ideas across the community is an important part of helping critical infrastructure facilities achieve their missions, such as safe and reliable water delivery, even in the face of cyberattacks,” Davis said.