BsidesLV 2025 - IATC - Monday

[Music]

Wow. [Music] Heat. Heat. Heat. [Music]

[Music] Heat. Heat.

[Music] Heat.

[Music]

Hey. Hey. Heat. Heat. N.

Heat. [Music] Heat. Heat. Heat.

[Music] Heat. Heat. [Music] Heat. Heat. [Music] Yeah, [Music]

[Music]

down. [Music] Hey hey hey hey hey hey hey hey hey hey hey hey. Yeah, [Music] you [Music] Down

down down

[Music]

Hey [Music] everybody. [Music] [Music] for down [Music] behind. [Music]

Black. [Music] Hey. Hey. [Music] Hey. [Music]

[Music] Heat. Heat.

Heat. [Music] Heat.

[Music]

Heat. Heat.

[Music]

One minute warning.

[Music] Heat. Heat. [Music] Should [Music] we start? >> Let's go. >> All right. Welcome to this year's Bides. I am the cavalry track. Sometimes called IATC because acronyms somehow helpful. Um, I'm Josh Corman >> and I am David Bots. >> Uh, don't make me have to chase you. I rolled my ankle last night, so it'll be fun. Um, if who has not been to this track before? >> Awesome. Okay, good. So, we're going to do this track's different. Uh, appreciate you coming. The first talk just kind of sets the table for the next two and a half days. Uh, we're going to try to provoke you, uh, but not just for provocation sake. We're going to ask you

to be comfortable with some discomfort. We're going to tackle some pretty heavy topics. Uh, because I saw a lot of hands, I might do a very, very, very short what the heck is I in the cavalry. Uh, and what does it have to do with besides Las Vegas? I sometimes skip that. But, uh, we're going to mostly orient you to the track for a few minutes. Then I'm going to give a specific talk to try to challenge you. I want you to critically think and then we're going to give an overview of every talk for the next two days, two and a half days so you can kind of see what the flow is. We try to

say that this is not an alakart track. It tries to be a bit of a symphony. We very deliberately invite speakers. We very deliberately sequence speakers. We negotiate with them what we'd like to see from their expertise and they push back on us and it's a beautiful little tugof-war. uh but we believe that every talk is enhanced by the talks that came before and the talks that come after it. So everyone gets more out of the the sum is greater than the the whole is greater than the sum of its parts. So we'd encourage you to stay. All of these will be streamed and will be recorded. Uh so you can watch them later if you do have

to miss one. Go upstairs for like a sky talk that isn't recorded for example. Um, but you will get more out of this and we even sometimes pick the ones just before or after lunch on purpose to make sure that it's maximal discussion. So, be prepared to discuss things. Most of these um will leave room. We have a two-hour water block, for example. Um, but you want to do any welcoming before I say what the cavalry is and where it comes from. >> This is your MC for the next two days. >> Good morning. My name is Dave Bots. I'm I'll be your MC today, tomorrow, and Wednesday. >> Oh, that's right. We've got a rock and a

roll in set of sessions. Um, I will be the very mean timekeepery person. So, uh, when I stand up, that means we're close. And the closer I get to the whoever's speaking, what that means is they need to wrap it up. Um, but that's what I'm going to do. So, you'll see me and and there is going to be a lot of opportunity for audience engagement. And when we do that, I'm gonna play I'm of a certain vintage. I'll play Mr. Phil Donahghue where I'm doing this. >> Yes. >> To get your input. So it because we want your input and we need you to use the mic because otherwise our dear friends at home can't hear you and they'd be

like that might have been an interesting question but I do not know what it was. >> Yes. >> So uh help us help you help us help you. >> Okay. All right. So, I'll try uh a shortish uh cavalry story uh how we got here. Um so, first of all, let's all say happy birthday. Cavalry was born on August 1st, 2013. So, we just turned 12 years old. >> Happy birthday to you. >> Um so, I love Bides. Been coming to Bides for a while. Uh after rise researching like the rise of anonymous and activism and stuff, I kind of started getting pulled into uh the government circles a little bit, I was worried that activism

may turn into cyber terrorism, which it did. Um so at one point a bunch of hacker people, you know, and respect. We all went into Fort Me for two days to try to warn what's the hacker eye view on public safety, economic, national security so that we can give them some fresh ideas. And during those two days, um, we had breathtaking ideas from like HD Moore, Dan Kaminsky, Jean Kim, myself, Alex Hutton, and David Etchu. And we just, it was one of the most exhilarating things to take individual superheroes and put them together to form an Avengers to solve bigger problems. And it was one of the highlights of my life, right? I'm doing an abbreviated version of

this. What wasn't is that um in between day one and two, I went to my car and had a ton of voicemails because you can't bring a car into Fort Me. I mean a phone into Fort Me. And I a bunch of people were saying, "Oh, I'm so sorry. I'm so sorry. I'm so sorry." And what I uh couldn't tell my teammates is that my mom had had a stroke and we found out it was pretty aggressive terminal brain cancer. So like I'm at the peak of like trying to like speak truth to power with really talented people, but I'm also like demoralized that I know what's going to happen over the next couple months. So on that personal journey, we

close day two and we have our recommendations and they're amazing. And if someone presses me later at the bar or the pool, I'll tell you what some of the answers to the challenge questions were. Like if you could write one sentence of legislation that have the most material impact on the hemorrhage of intellectual property from the US to China and material weaknesses in public safety to human life, what would that one sentence be and why? So we had an answer. Uh so ask me later, I'll tell you the answer. But um at the end of reading these all out to General Alexander and his staff and Anne Newberger, the answer was I can't do that one. There's no political will for

this one. People have to die first for that one. We went the entire list. They couldn't do a single one of our ideas. And this isn't to slam on them. It's just, you know, things work differently in the beltway. So, um, we all went to the bar. We all flew there on our own dimes. We drank at the airport bar. No one said a word for like, I don't know, 20 minutes. I finally broke the silence and I said, "Guys, the cavalry isn't coming. Uh, no one's going to save us." That was half the thought. Uh, advance a couple weeks later. My we take my mom to uh her church one last time before she

has to go to hospice. really shitty luck. It happens to be the Sandy Hook shooting. And she was a superintendent. So all her students and teachers and principals were all crying and scared to go to school. And the preacher kept saying for mostly two hours straight, "Why is there evil in the world? Why is there evil in the world?" Uh, and never quite answered the question. And I think everyone was hurt and scared and my little girls were crying because they're they're going to lose their mimi. They're afraid to go to school. So that was an uncomfortable moment. And then eventually come January, she passes and we go back to that church and I have to say something

as her eldest and I'm looking at her parents and her siblings and her students and her grandkids and I felt angry like why am I feeling angry at my mom's funeral? And it was partly because like that nagging question of I didn't like the question of why is there evil in the world? And at some point I had to go inside. I walked up and something flipped in my head and it was something along the lines of, you know, the last time we were here I was really angry. It was Sandy Hook, etc., etc. The question didn't sit right with me. And I think I just figured out why. And it was basically that my mom uh said my mom was

my seventh grade science teacher because someone got very hurt. So, she got to substitute. And she was fantastic. But one of the things I learned in science was uh darkness is not a thing. It's an absence of light. And cold is not a thing. It's an absence of heat. So maybe it wasn't just the presence of evil. It was the absence of good. And I said, maybe something's missing. We got to put it there. So then I asked all the family. I'm like, what's the absence of Marie? And I didn't have an answer to my own question, but I'm like, we don't get to find out because now it falls to us to do what she was doing. And that kind of

squared the circle for me. And I said, okay, maybe something's missing. what can we do about it? So, I came here to Vegas. I asked besides can I stick my neck out. We said, "Hey, uh, if the cavalry isn't coming, um, what's the hacker community willing and able to do to maybe be safer sooner?" So, the problem statement, uh, was something along the lines of our dependence on connected technology was growing a lot faster than our ability to secure it in areas affecting public safety, human life, economic, national security. So I was really worried about putting software in medical devices when we can't protect websites or credit cards, why can we protect hospital equipment, cars, industrial controls,

oil and gas pipelines, aviation, highspeed rail. So I said, if there's a knowledge gap between how overdependent we are on dependable things, can we maybe not be a pointing finger, but a helping hand? Can we maybe lead with empathy instead of judgment? Can we maybe build trust and coalitions? Can we meet people on their terms, learn their love language, meet them where they are, and then crawl, walk, run together. So I said, we're going to try something different. You can do the same thing over and over expecting different results or we can try to work together. So about 50 people said, "Yes, let's do it." We went to Defcon later in the week in the main

stage and got a couple hundred more. Went to DerbyCon, said, "What's our mission, vision, goals?" And essentially said, "We're going to focus on where bits and bites meet flesh and blood." So, if you watch the launch video, we had a little bit more scope, but we narrowed it down to say what's an unimpeachable thing that we could show the vital unique contributions the hacker community can make for public safety. A nonpartisan thing lo it's hard to lobby against some of those things or harder, not impossible. Uh, and we said we have no idea what we're doing, but um we want to lay the groundwork and the trust that such that when there is political will, we can seize on it. So

over the last several years, without getting through a history lesson, you can watch the 10-year anniversary video from two years ago, um we have dramatically influenced regulations domestically, internationally through things like the patch act for medical device, cyber security, IoT laws, really took the teeth out of things like DMCA and CFAA for like punishing or stifling good faith research from people in this room. So we really tried to take the name hacker back to show there are helpful hackers that are protectors, puzzlers, prestige, profit protests. But that this group we really found that besides was the protectors and the puzzlers that want to make the world safer and want to tackle really hard problems. And this track is really our

neutral ground to get government people that we're building team teamwork with or industry people from the water sector or nurses and doctors or technicians or highspeed rail folks to come and learn from us and for us to learn from them and then we go change the world one year at a time. So this is now uh since 2013 the track. Did I hit the most mostly the high notes? Okay. So thank you for joining. Uh I'm not the cavalry, you are. Uh and the and the process has changed over time. We became trusted enough that whenever there was something big going on in the government that affected public safety, there was like a congressional task force for healthcare

and industry cyber security. Then when the pandemic was declared, some of us went in as emergency feds to keep hospitals and vaccine supply chains safe. Um and then occasionally, you know, there's executive orders for ESBOM or this or that. So we have become an honest broker, independent um voice of reason on public safety, public good and each of you get to contribute just like Fight Club, you choose your own level of involvement and uh some of you have had profound impact. So we appreciate that. The line is basically shifted into something like this. And if you were here last year, um we're going to crank up a topic called undisruptible 27. So, normally we're patiently impatient and

we know it could take nine years to pass a law or to reform some broken thinking. Uh, we're not being so patient anymore. So, we're going to get into some pretty heavy materials and I'm going to show a talk that David encouraged me to do for the hackers that I originally did for RSA. So, I want to see how you react differently and there's definitely audience participation. So, at this point, I will start the big chunk of the opening. Okay. So, my name is Joshua. Shall we play some war games? And by the way, a lot of people haven't seen that movie. So, if you haven't, please do. Um, this was mostly aimed to cause cognitive

dissonance to the RSA crowd. I'm just preaching to the choir here, but I I'm still mildly curious. I I agreed to David's challenge. I'm mildly curious how you all answered. So, this requires prompt feedback. Are we ready? Okay. So, you're also going to see we have stickers. This is a water hammer. We're going to talk about that later. We learned it from our trust building with water engineers. Okay. Who wants to play a game? >> Yeah. >> All right. All right. Sorry to yell. Okay. So, unlike chess checkers, the global thermonuclear warfare, here are the games we're going to play. And um maybe now that we have more funding from Craig Newark. So, what I probably should

say about Undisruptible is um I met Craig Newark. He had not heard of the cavalry. Craig Newark of Craigslist. He was very taken with the mission. He was very taken with the impact. He was mostly taken with our ability to do storytelling and to use empathy to not talk like beltway people to average everyday Americans. So he really wanted to fund the project. I wasn't I said Cavalry's never taken a penny. We're never going to take a penny. So we found a common ground with IST, the Institute for Security and Technology. Many of you might know it for the ransomware task force, but it's got a a much bigger portfolio. And we said, okay, why don't we use the 501c3,

the Institute for Security and Technology, and why don't we try a pilot? And my pilot was based on basically three things. One, uh, on January 31st of 2024, Jen Easterly, Christopher Ray, General Nakason, and Harry Koker, the four horsemen of cyber, said in an unclassified hearing to Congress, "Hey, uh, China has intentions towards uh, the Taiwan Straits as early as 2027. Volt Typhoon is an army unit that's uh gaining access to US civilian infrastructure like water and power and other things which is very transgressive. It's not normal spycraft of fair play. It's putting virtual bombs and detonation charges into civilian infrastructure. Um pretty scary stuff and they were saying it out loud. and knowing what I know from my 18 months at

CISA and working with the federal government, um we don't have a cyber defense force like military, we'll get into that on a different slide, but I'm like where's the urgency? Uh where's the collaboration? Where's the systems thinking? Because if we're still struggling on voluntary standards for credit cards and and business apps, how are we going to protect water and power and hospitals that are many of them are target rich but cyber poor? They don't participate in the public private partnerships. They're small and medium rural. No cyber security staff, no mandatory restrictions. So, um, we decided to make a pilot to say, let's look at when everything's critical. Nothing's critical. And we'll get into a little bit more with some specific

slides, but the basic thumbnail is we're seeing more disruptions. Our neighbors don't call it hacks or breaches. They say payroll was disrupted. Patient care was disrupted. My flights to my own wedding were disrupted by crowd strike. So, like these things were happening in a higher volume. So there's more of them. They were longer. They're more life safety. And they're starting to say, "I thought you guys had cyber under control." So with that disruption trend of being either accidents like crowd strike or adversaries that want money, what happens when this turns to weapons of war because the next conflict will be a hybrid conflict. So if we're going to have a war, you should expect uh people

are going to start throwing rocks and we're made of glass houses. So what can we do between now and then to fix things? But also, we have 16 critical infrastructure sectors. They're not equally weighted. We had 55 functions within those. They're not equally weighted. So I really said, let's focus on the lifeline basic human needs that if they're shut off for 24 to 48 hours, your family suffers. So water, access to emergency care, power, food supply, things like that. Uh we said they're also interconnected. So the federal government doesn't like to cooperate cross agency or cross sector. They say stay in your lane. Stay out of my lane. This is your lane, not your lane. So we

might organize the government that way but harm and disasters do not present themselves that way. So I wanted to say can we look at interdependencies on lifeline services as we approach 2027 or later and find a way to get to meet people on the ground. Meet them where they are. Use empathy. Use love language and see if there's some way to reduce elective exposure or to mitigate the worst consequences of failure. So this game is going to help us through that. All right. Asset prioritization. You have to pick one. What's more important? PCI data or PHI data? You got to shout. >> PHI. >> All right. Who says PHI? Who says PCI? All right. So, we think our healthcare

data is more important. Okay. What's more important, intellectual property for your employer or your PHI? >> This room. Okay. But don't you have day jobs? Do you know uh somebody else that you work for might disagree? All right. What's more important, your medical records or your medical records? This really matters, folks. Uh, the fine print is the confidentiality of your medical records or the availability of your medical records. Which one? >> Who? Raise your hand for confidentiality. Raise your hand for availability. >> Availability. >> Okay. I love my privacy. I'd love I'd like to be alive to enjoy it. And we only have regulations and laws for HIPPA for the confidentiality of medical records. You have more uh regulatory

incentive to have a corpse with your privacy intact than to keep patients alive. Your wallet or your life? >> All right. So, this room's, you know, I'm preaching in the choir. I told you they were going to get all the right answers. But, okay. The needs of the many or the needs of the few. Specifically at RSA, I meant your employer. What? >> Okay. >> The many depends on if you're a Star Trek fan. Okay. So, we've sort of constructed a Maslo's hierarchy of needs, right? Not everything's equally important. So I kind of heard the availability of your medical records so you can get your chemo treatment or avoid a blood type cross cross contamination that's more important than

the privacy and the privacy is more important than IP and PCI doesn't matter you know that much. I said, 'Okay, so RSA, why did we do the exact opposite prioritization for the last 30 years? Because we did. It's not trit. And you could feel the room go. And it's not just that we've had the the backwards priority. It's that we're usually there for our employer, for our enterprise to drive fiduciary value to shareholders. But every single one of us live in a community. We have a family. We have a town. We have a county. And if we are applying our unique talents merely to protecting data for an employer, who's protecting the water, the health care, the power for your

small, medium, rural community. Don't worry, we'll come back to that. Now, that's just assets of data types, but like I told you before, we have national critical functions. These are specific discrete services that we all depend upon in different parts of the federal government have custodial ownership for. Francis is the national coordinator across those if their siblings let them. So let's do that same game with functions. So what is a function? There's 16 sectors. Water and wastewater kind of got the EPA. Health and human services kind of takes care of this thing called healthcare and public health. Treasury has financial services. So they all have a custodian. But when SISA came along, they made 55 national

critical functions. are more discreet things like provide drinking water, provide medical care, provide electricity, distribute electricity. So each one is a service you can look at is is it available, is it degraded, is it down for how long? What's the meantime to repair? So you can treat this like a utility just like your your phone, your dial tone. Okay. Well, one of them called provide medical care belongs in p healthcare and public health which is HHS. And even though it depends on all sorts of other sectors, you know, it gets sort of badly tracked. So back to Mlaw's hierarchy needs, what I kind of said after the CIS co task force is when everything's critical, nothing is. If

you were to map these, you know, to the things that keep us from being lower the flies, it's going to be a much smaller list. So the stuff at the bottom is the no kidding, non-negotiable stuff, and the stuff up top matters to somebody eventually, but only if we can actually um have water and food. So, back to the idea of your wallet or your life. I think you all said pretty correctly. Um, your life. So, what's more important? Support banking or provide fuel for the eastern seabboard? I need answers. Huh? Fuel. Okay. I mean, the continuity economy is pretty important, but maybe not today. Maybe within a couple days, right? What's more important? Provide

electricity or provide food? Can I see some hands? Electricity first? >> Hands uh for food. Okay. How long can you go without food? >> Okay. What's more important? Provide water or provide food? >> Okay. You can go I was told three minutes without oxygen, three days without water, 30 days without food. I think it depends on who you are, but uh I don't think I can go 30 days without food. Um interesting. Okay, good answers. uh provide water or provide medical care. >> Hands for water. >> Hands for medical care. >> Yes. >> It's a trick question. What's more important? Protect sensitive information, your PHI under the HIPPO law, or provide medical care, timely access to care for you and your family

when and where you need it. Which one? I think HHS OCR would disagree with you. Okay, I agree with you. Okay, so let's figure out I'm pretty biased. I think timely access to medicalare is important. So, you're going to see the first of a few videos here. I did not make this one. We have drafts or two that we did that definitely need your help. But this one uh can help you understand something we learned from Christian Demf who's speaking tomorrow. Dr. Christian Nef is that time is brain. Uh so here you go. Emergency department. >> When it comes to a stroke, time is brain and every second counts. That's why at Northwell Health, we know a stroke needs

a different approach to time. So, we introduced Teastroke, a remote video consultation system that connects you to a worldclass stroke urologist instantly, day or night. It's another way we're not just raising our standard, but the standard of healthcare. Northwell Health. Look north. So with strokes, there's really two major categories. There's lots of subcategories, but it's either a clot choking your oxygen or it's a bleed. If it's a clot, and we can use imaging, you can take a clot buster and it'll save brain and save life, your save your motor functions. If it's a bleed and you give that, you'll kill them instantly because it'll accelerate the bleed out. So imaging is critically important for that timesensitive time is

brain. Okay, there are other time-sensitive conditions. So during the CISO CO task force, some of us that went into the government for I was there 18 months to the day, we started seeing um quite a bit of hospital disruption from ransomware activity and because of the circumstances, I'm not going to go through the longer you can see a longer video on all this stuff, but we published the first statistical proof of loss of life from a ransom event. Um we got to see with public health information for something called excess deaths, which are tracked all the time. difference between expected deaths and actual deaths by by month, by state, by condition, by g by demographic. And we

could see that in the same state with the same population adjusted hospital type and size, the ransom communities achieved stress levels sooner and stayed longer than their peers. And these stress levels were directly associated with excess stats two, four, and six weeks later for time-sensitive things like heart and brain and pulmonary. The math is all published through the CDC MMWR. So like whenever someone says well no one's going to listen until people died. We had to kind of prove okay yes we have proof that a nonzero number of people died we could use you know public health information and statistics to do so longer conversation go through this in fact maybe tomorrow uh Christian will

show some of that but there's a couple ways you die in this scenario number one is there's a disruption of patient care and the unavailability of certain technologies like electronic medical records or imaging which number two leads you to divert ambulances to the next nearest facility if those next nearest facilities are four minutes away might be okay. If it's an hour away, you might not be okay depending on the condition. If it's four hours away, you might not survive that delay. But the real deaths happen in columns three and four where we could see hospital strain levels of ICU bed count associated with excess deaths 2, four, and six weeks later. On the same day we published this

of October 1st, 2021, the front page of the Wall Street Journal also published a delayed story of the first named victim of a ransomware uh lawsuit. It was a 2019 ransom of an Alabama hospital that with a complicated birth. Um they couldn't use imaging. The the hospitals are ransom. Much of the equipment didn't work. And because of that equipment not working, um they still admitted patients. They couldn't use imaging. They didn't notice an umbilical cord wrapped around the neck, which is still a treatable and deliverable baby. But uh despite a complex and otherwise successful birth, the post birth monitoring was also compromised. In a typical uh natal intensive care unit, you could have a dozen pieces of

technology that force multiply the limited nurse to patient ratio and caregivers and none of them were working. So subsequently the the poor child perished. There's an ongoing lawsuit in part because the hospital and staff were communicating with the chair saying we should never have admitted patients under our conditions if the equipment was working. This was otherwise very treatable situation etc. After placing so between a statistical proof of loss of life and a named victim we started to get some political will and then subsequently medical health professionals started doing a lot more peer-reviewed medical journals. Last year if you were lucky to be here Dr. Dr. Christian showed one of his newest ones. He's going to show two new ones

tomorrow. Two that I believe maybe even a third. So this is one of the ones he shared with us which is not only in UCSD University of California San Diego hospital where they work. The Scripps Institute hospitals all got ransoms. Not them. They were fine. But they saw the blast radius of overflow of patients overwhelmed all other hospitals in the area. So they call it the blast radius. and they could quantify weight times, cancel surgeries, procedures, worsened outcomes at a macro level. But this one was even more stunning because he went really deep just into heart conditions. And when you have a heart condition, we track the survivability rate with favorable outcome, favorable conditions. And they saw a that even an unransed

hospital in a ransomed region had a t-fold drop in favorable outcomes for heart conditions. and that really got the attention of the cardio community. So, um he will share some of that tomorrow. Then we said it's not just the individual health because we know that when you're degraded and delayed and there's um a ransom making unavailable health records or unavailable workflow, it could be change health care where one payment middleware for the country at United Health shut down patient care and workflow for 75% of the nation's hospitals for weeks. Well, we also know that many of the nation's hospitals, we had 7,000 when I did the congressional task force in 2016. We have 6,000 now.

So, we've lost a thousand to mergers, acquisitions closures financial insolveny, some of which is exacerbated by ransoms. Why? Most small medium rural hospitals, according to Beckers that studies the space, have four to six weeks of their cash reserve on hand. So, their burn rate, if they're not getting income, is four to six weeks. If a ransom shuts down your patient care for four to six weeks, you could either close forever, like St. Margaret's in Illinois, or maybe be weak enough to be acquired and be lucky enough that the next time there's a a ransom in common spirit in Portland, Oregon, you'll shut down hospitals in the same network in Connecticut. So, we're seeing um you

either go down for the count, you're weakened enough to be acquired, and they take your doctors, your nurses, your services. Some of these acquired hospitals have bats in the fifth floor in Florida uh from just being under uh invested in. They're functioning hospitals with bats in them or uh you're lucky enough to be in extended blast radius for the next big ransom. So we are not comfortable with this. And when you really look macro, not just your hospital, but maybe nationwide, uh University of North Carolina tracks hospital closures. So Audi and I made a time lapse. Every single one of those dots is a hospital that's gone for good. So again, not all from ransom, but

ransom can make it worse. Uh so we're with 700 plus ransoms a year on hospitals, you're rolling a D20 each time and hoping that you don't actually close. Okay. So which is more important? You're protecting your HIPPA data or provide medical care? We already answered this, right? So why is it that every single time Children's Hospital of U Chicago gets breached and they call their instant response firm and they tell them what they should say to the press, they say something along the lines of in an abundance of caution and in accordance with industry standard best practices, we've decided to shut down operations to contain the breach. So what they did is they took a privacy

loss of PHI that was the horse had already left the barn and they voluntarily self-inflicted a denial of patient care on provide medical care. They also voluntarily shut down workflow for billable financial cash flow and if it was for longer than four to six weeks they might shut the doors forever. So our best practices are taking a HIPPA optimization over patient care. So back to this idea of May as hierarchy of needs, I started saying maybe we should look at downtime tolerance. So which of those 55 national critical functions, if you shut them off for 24 to 48 hours, do people die? And it was about a dozen of them. And not only are these particular dozen atomically

important, they depend on each other. So if you have no water, you can't provide medical care. If you have no chemicals to clean things, no blood supplies, no transport for pass uh patients. And when people die, it starts affecting your workforce for things like water and wastewater chemical treatment. So we saw this was a pretty dangerous cycle. Hence the undisruptible project was saying let's focus on those dozen or so lifeline critical functions. Okay. So let's talk consequences. I was going to give some freedom of choice but we're going to little is a little compressed. So power. So let's pick one. Somebody say hostiles. >> Hostiles. Okay. Picture a hospital. Picture your hospital. When was the last time you

were there? Was it to welcome a baby into the world or to say goodbye to a loved one? No one wants to need a hospital, but when we do, we depend on timely access to care. When and where we need it, irrespective of cause, delayed and degraded care for time-sensitive conditions can affect worsened outcomes and even loss of life. A 5minute longer ambulance ride has a significant impact on 30-day mortality rates. Time is brain where even an hour or few could determine if you walk again, if you talk again, if you even survive. Now picture your hospital. What if that hospital was not available to you? If your hospital was disrupted, where would you go

instead? Is it across town, more than an hour away? What if they are also down? The chance is not as remote as you'd hope. Hospitals have become a top target of ransomware, cyber attacks that [ __ ] technologies in the vital path of care delivery. Worse, your hospital doesn't even need to be the one attacked to endanger you or your family. We've seen a 10-fold decrease in favorable outcomes for heart patients, merely due to excess strains of a ransomware affected region. Now, back to your hospital, back to your family. You and your family deserve better. If we want timely access to patient care and more resilience in the face of accidents and adversaries, we're going to need to advocate for ourselves.

Now, as we head into an era of hybrid conflict with threats to water and power, these disruptions stand to get a lot worse, but we'll talk about that in another video. That is one of our draft videos for the first year pilot that ended about two weeks ago. So, we would love feedback if we're going to invest in making them bigger. But usually we think of a disruption as an inconvenience or a breach or a ransom or a fine. Uh, but if it's happened to your community, you know, it can dramatically affect your access to care. So, let's talk consequence number two. You're going to get stickers that have these on here. Anybody know what that is?

>> A water hammer. No, it's not an ice hammer that someone thought yesterday. It's a water hammer. I didn't know what a water hammer was. You probably heard one in your house, but the when they're much bigger, they're much more devastating. I learned of the water hammers from our teammates like Dean who presented on water. He's a professional water engineer. Part of this empathy and getting on the ground and meeting people where they are and learning their love language. When we started warning them about things like Volt Typhoon, we said, "What's the worst that could happen?" You know, there's lots of bad stuff that could happen. What's the worst thing that could happen? Like, well, bursting

of water means could be bad. So when you hear it in your house, it might sound like a little you're hitting like a radiator with like a little pencil or something. It's like a ting ting ting. But if you have a 24 inch water mane or a 36 inch water mane, they can be 30 40 foot high uh burst pressure loss. That's a truck driving underneath it. Uh they don't necessarily happen where you want them to. So the weak points in the line and aging pipes and the aging millions of miles of pipes that we have across the country uh could create a downtime event. So what's your downtime tolerance? And uh so water hammers could

be pretty bad. There's other scenarios they gave us as well like an overpressurization event, an underpressurization event, chemical adulteration, etc. So we started leaning into this water hammer idea. We'll come back to that. Who wants to see the power video? >> Yeah. Woohoo. Yeah, me too. But it was not in the scope of the pilot. Okay. All right. That's why we needed to get more funding. Okay. So, let's test some assumptions though. At some point, usually, especially a circumspect crowd like this one, because we are critical thinkers, someone's going to be saying this wouldn't happen. Why you talk about 2027, etc.? So, let's go through a couple of the t the top assumptions of the who, what, when, where, how, why.

So, who would do this? Well, remember we have accidents like the crowd strike thing from a year ago. We have adversaries that want ransoms and money or intellectual property. financial crime, right? But we also see hybrid conflict being used a little bit. Uh and there are four countries on our map. Um three of them have reached out and touched water already. Uh so there is Vault Typhoon, which is the one I I talked about that stimulated this conversation. So this is uh prepositioning an access campaign to get in and stay in living off the land on the nation's unguarded infrastructure. But there's also Iran, the cyber avengers that maybe you've heard of. um considered a activist group of some

sorts. They hacked Israeli made equipment in Alipeka, Pennsylvania and other states, I think 21 other states. Uh so that we know that they can reach out and touch things. They defaced the interface as opposed to did damage, but that same level of access could be used for something else. And then Russia, the cyber army of Russia Reborn, not the actual army, but a criminal group um hacked equipment in Texas and overflowed two two reserve tanks. So they were doing it for money. They weren't trying to cause the physical damage, but it's pretty easy. And as Bo Woods likes to say in the cavalry world, malicious intent is not a prerequisite to harm, right? So you don't even necessarily uh

I'd prefer they don't have access and they aren't trying, but we have wide open water infrastructure. So, all three have demonstrated the means, motive, and opportunity to reach out and touch water. So, this is a office of the the director of national intelligence put out a two-page PDF. It's public and shows some of the states and dates and campaigns where Iranian hackers hit US infrastructure. Um, I'm shocked that they did it in public open source as opposed to classified. There's other stuff if you're in that world that's more serious than this, but this is the part they were saying out loud. Um, okay. So, why 2027 is usually the second question. Well, um, I hinted Jen

Easterly, Christopher Ray, Nakason, and, um, Harry Coker pretty clearly, like pretty resolutely, pretty consistently warned of Volt Typhoon. Uh, Xiinping says he wants to unify China with Taiwan um, in his his term. Um, he told his military to be ready to do so as early as 2027. If you're following people like uh our guest speaker in a minute here or Dmitri Alovich, some people think it may be a year or two later, but we're in the vicinity of um declared intent to take Taiwan or reunify with Taiwan. And they said to the US uh we'd prefer you stay out of it. So part of the stated objectives of Volt Typhoon are to undermine public support for our

intervention and or uh if we do interfere they could um do allout chaos on US infrastructure uh in retaliation. Perhaps there's a baby step in there like a brushback pitch. But uh it doesn't take a lot takes two towers for example to have us go to war for 20 years. So even a demonstration of force could be pretty devastating to otherwise unwitting civilians. So, this became uh if you're what's that Maya Angelou quote? If someone if you're someone tells you who they are, believe them. So, we at least have their stated doctrine and we've talked about it in open source uh public hearings and you can watch them. This one can take a while, but I'm going to go faster,

especially because uh Bryson's going to do a guest sit rap. But someone says, "Well, don't doesn't the military protect us? If this is a war, aren't they going to protect us?" And without being mean, I'm just, you know, been up close and even at times in the belly of the federal government. Um, what people don't realize is the in other countries in Europe like critical infrastructure lifeline stuff is publicly delivered and publicly owned and operated. Uh, whereas we delegate that to the private sector. 85% was a wild guess with no actual math behind it, but it's actually come up and they did the math and it's about 85%. Uh so 85% of owners and operators of

lifeline infrastructure are um private. Some of them are price fixed at the state level. Might cost you more than a dollar to give you a dollar of water. Um so they're really prone and you might have heard the term we coined but target rich but cyber poor. You kind of building off Wendy Nathther's notion of living below the security poverty line. Um then people said well don't we have cyber command? Doesn't cyber command protect us? Well cyber command's an offensive unit. It's not defense. And then someone says, "Well, I heard that CIS is the nation's cyber defense agency, right?" Well, they don't have authorities to go into your networks that are privately owned and operated.

They're an inform and advise and assist function and a national coordination function. Uh, but they're not actually authorized to go do operational security and patching for you, and nor would you want them to, per se. So, that doesn't really happen. And that was always kind of true. People didn't know it. It got worse uh with executive order 14239 in March, I think it was. Um where there's a strategic shift whether you like it or not. Um maybe it's the right call eventually. I think the timing is probably pretty unfortunate, but the shift is saying we want to shift the bulk of the work for cyber security resilience from the federal government to the states. So, um, if we're going to shift that,

um, one of the very important institutions that could help is the MSIAC, which stands for the multi-state ISAC. And yet, maybe independently that one lost its funding, too. So, we both are doing a strategic shift on the when you can hear the drums of war. Uh, bad timing and some of the graceful transition has been hampered by defunding of the multi-state ISAC. So, what's an SRMA and CPAC? Um well SMras are the sector risk management agencies. So each one of these 16 sectors has a custodian. Many of them had significant budget cuts uh uh voluntary attrition some Doge effect and or their committees of jurisdiction and oversight are mad at them for some reason or another. Uh that includes SISA

and then CPAC. CPAC is the way that the public private partnerships are lawfully legally allowed to talk to each other without breaking other advisory committee laws and lobby lobbying things. And so CPAC got suspended back in February and I don't think it's been turned back on yet unless someone can tell me. So CPAC is what allows them once a month or upon emergencies to talk to each other uh under TLP red and amber and they can't actually do that. So even the free collaboration that had existed at least is temporarily still turned off. That was not on purpose. It was by accident I was told but so this is not a good situation. And then SISA as I said

it's it's advertised as a cyber defense agency but it is an inform advise assist not operate function. Uh, and then Congress, um, even if Congress is pretty bipartisan on the China Volt Typhoon, salt typhoon stuff, which they kind of are, um, if you pass a law tomorrow, the normal cadence of how a law metriculates into a notice of proposed rulemaking and commentary period and all these other things, uh, it probably wouldn't manifest any direct impact on the affected time zone for which we speak of 2027, which is now under 18 months away. Okay, insurance. The next thought turning cliche we tend to get as I try to go a little faster is but I'm insured. Um, I'm working with Cyber

Accu, which is the antitrust collective of the top 20 underwriters or cyber insurance in the world. Most of them are the big ones that do other forms of insurance. Uh, have you heard of acts of war exclusions? Um, so none of them believe that you are covered for the things that you believe you are covered for. Um, but there are some edge cases and some nuance and I've been collaborating with them on clarifying to their policy holders what is and isn't covered in the case of uh the People's Liberation Army destroying water in the community and having downstream and secondary effects of other things. So, they have some public statements they're working on. There's going to be some

tabletop exercises. Um they also even without acts of war exclusions they don't cover business continuity disruptions instituted by a service disruption for infrastructure. So if you get ransomeds they'll cover the business loss for a certain amount of time for the ransom but if your water goes out uh not usually covered. So it depends may v uh mileage may vary and there's talks for years and years about a cyber security backs stop. Some of them mean a vertical one for the whole industry. on the mean horizontal for certain topics like we have a TRIA backs stop for terrorism after 911 so people would build property that gets renewed every couple every 10 years or so but

there isn't really a cyber backs stop but there's on and off again discussion so maybe Congress could do something there or the White House could do something there but don't look for a lot from the federal government and don't look for a lot from insurance what we're really looking to do is clarify your expectations about what is and isn't covered and then the last one is Well, in the world of national incident management systems and NIMS, which you're going to hear about later today. And in the world of disaster science for wildfires, we don't have to have enough firemen for wildfires. We can call on mutual aid, mutual assistance agreements with other states, and we can snap into things like

NIMS. Well, um, mutual assistance, uh, is predicated on a concurrency capacity assumption. So Wisconsin firefighters are very happy to help LA fire uh firefighters unless Wisconsin's also on fire. So when in these attacks are everything everywhere all at once um most of the concurrency assumptions fall apart real fast. And by the way so do the insurers. The insurers said we'll still help you if we're not covered. We'll do the instant response. We'll we'll assist you. I said what's your maximum concurrency to assist your install base? Do you prioritize based on first come, first serve, on declared state of emergency, on biggest customer? Do you get to decide or does mania get to decide? So, there's a lot of

unanswered questions on concurrency. So, most of our mutual aid things are pretty much going to fall apart within the first few hours. So, those are fun. Uh, but let's pull some of these things together. You have the hammer time stickers we're going to give you. You saw the hammer, the water hammer. So, let's play another video, shall we? This is the first video we made.

We are too dependent on undependable technology. The systems that we rely on every day for everything from water to food to power and emergency medical care are subject to escalating harms by accidents, bad actors, and nation state adversaries. These attacks could quickly move from disruption to destruction. For example, an intentional water hammer that abruptly stops or reverses water flow, sending a shock wave through the system. Attacks on our water systems would be devastating, not just for lack of access at home. No water means no coffee, no toilets, no laundry. No water also means no hydrants to put out fires. No water means no healthcare. The hospital can't run without clean water. No water means no sterilization, no

surgery scrubbing, no laboratories, and eventually no access to life-saving care. Our dependence on connected tech has grown faster than our ability to secure it. And there is evidence that foreign actors are already weaponizing these vulnerabilities. But who would actually do this? In public hearings, Congress and US government cyber security leaders have warned the public of Vault Typhoon, an ongoing campaign of successful attacks on US water facilities led by a People's Republic of China state sponsored cyber actor. But China is not the only aggressor. We've seen cyber attacks on our water systems from Russia and Iran. These attacks pose a broad and unrelenting risk to critical water infrastructure and could escalate to large-scale destructive attacks on

our water systems as early as 2027. The good news is we have time to make changes. We must strive to make our lifeline basic human needs undisruptible and where we cannot ensure that our communities are more resilient under fire. This means divesting our reliance on connected technology, better securing our existing systems where we cannot disconnect, and ensuring analog solutions are in place when those systems fail. If this sounds overwhelming, remember if you can't afford to protect it, you can't afford to connect it. Undisruptible 27 will prioritize the safety, security, and resilience of three lifeline basic human needs, especially at the local level. >> Okay. So if you take the your hospital video of there's it's not your privacy

there is a time and space risk of diversion to next nearest facility and then now we combine that a disruption of water shuts down hospital operations in 2 to four hours and then if you try to go to the next nearest facility and they are also disrupted do you think the body count's going to be zero so um we're pretty concerned concerned about hammer time. We're pretty concerned about this. So then it comes what can be done. If we already told you the federal government's not going to help much and insurance will be after the fact, but probably going to exclude things. We started leaning into what is the art of the possible. Just like with Y2K, we

had to work backwards against a certain date and said what's doable. So is the best defense shields up. You want to add cyber. They don't have budget. They don't have staff. they've got about 12 to 18 months by the time we get to them. Or should it be connections down? Maybe it's being less dependent on undependable things until we can pay a responsible risk mitigation. It's more likely physical mitigations. And this is why I'm really trying to get a a lot more heat and light on some friends in the room. Uh things like Idaho National Labs has consequence inform engineering. We'll talk about that in a second. So maybe I can't disconnect and maybe I

can't cyber up, but I can make sure that a compromised system can't blow up a water man on the hospital network. So the best defense against hammer time is not cyber. It turns out it's really unintuitive. So uh Virginia, can you raise your hand? All right. So we have a a speaker later today and doing some free training over Padium today and tomorrow. is going to talk about uh consequence of foreign engineering but I'm going to grossly oversimplify their massive body of work and say if you want to reduce the probability of a compromise in cyerspace you add cyber but if you want to reduce the consequences of a successful attack you add engineering and we realize we are not going to cyber

our way out of this in the next 12 to 18 months so what can we do is say what's the worst that can happen and are there available familiar engineering mitigations for something like a water hammer And the answer are yeah, there are uh if you Bryson may mention critical effect, the conference we ran in June in DC, but we had a water engineer come and say, "Here's a $2,000 pressure sensor with a physical wire back to the pumps that can notice you're out of acceptable uh pressure ranges and disable the pump." It's like a circuit breaker for water. So for 2 to 10,000 for that pressure zone, maybe you can get punched by the Chinese military, but you can

take that punch without a burst water mane. Now I'm not trying to say it's that simple in every case, but this is going to be the mission. Cyber exposure with life and limb consequences and available familiar tangible engineering mitigations. So let's try that prioritization game again as we round it to in five minutes or so for our guest on sit. Okay. What's more important, provide water or provide medical care? >> Yes, water. >> Trick question. No water, no hospital. But now it gets harder. Do I restore water to the dialysis center on the on the west side of town or the trauma center on the east side of town? These are hard choices. These are probably these are probably

not your choices. Okay, next one. It's harder. Do I protect the town with one and only one hospital or do I restore one of the three hospitals in a town with three hospitals? >> This might not even be your town's choice. This might be county choice. There's only one level two trauma center

>> that that matters. But these these should be answered left of boom, not under fire. And we're going to have a talk closing and the closing block today from people in public health and emergency management, disaster management who know how you do these things. But have we asked those questions or posed them to our leadership yet? So which is most important removing exposure, adding cyber or engineering down consequences? >> Engineering baby, >> you're cheating, right? Okay. So I'm going to mostly compress this part. This was mostly for the RSA audience, but just if you're wondering what I I said I'd come back to it. Um, the reason we have our priorities backwards, it's not all their fault. Let's give each other

some grace. We've been incentivized to do so. Part of that is, if you know the William Sutton quote of why do you rob banks? It's where the money is. Um, forever prior to ransomware, attackers, the shields, excuse me, attackers, the sniper rifle scopes, and defenders, the shields focused on the Fortune 100 or 500 or 5 or 2000 because that's where the money was. Um, ransomware was not a technical revolution. It was an economic one. And what they found is um the unavailability of anyone can be monetized. So we had an unmititigated feeding frenzy on the cyber poor. The RSA crowd still has not figure out how to monetize down market. So they don't try. So we're kind of doing what

economics tells us to do, not what our community needs us to do. Uh because this target rich cyber poor stuff is not participating in public private partnerships. I'm going to skip this from the Cisco CO task force, but let's go look at the three. When we have public private partnerships, what that means is there's a government custodian, a sector risk management agency like HHS or EPA for water, there's a sector coordinating council, which is the voices of the private sector defending their honor and doing the trade-offs and there's usually an ISAC for the technical exchange of indicators of compromise. They said usually water emergency care and energy and power. There are 151,000 water plants in

the US, not not even including water and wastewater treatment. 151,000. A third of them service homes. So let's just say let's focus on 50,000. Do you know how many participate in the ISAC? Somebody not you. >> 650. We have 0.4% of the nation's water plants even in the ISAC. Do you know how many hard cyber security requirements are thrust upon water operators? Do you know that when the White House and EPA asked the water sector that at your next annual sanitation survey, please indicate which of the 38 SISA cyber performance goals you do and don't have in place. Inventory them. You don't have to do them. Just tell us which ones you have. We want some ground truth. You

know what the response was? >> We're suing you. Uh and these three states and several trade associations successfully sued the federal government for daring to ask. Yes. Uh so when I talk about participation um the water is sac whom I love and I'm working with and I'm trying to get them more more love they've got 650 of the halves not the have nots awa's um American water and waste water they have 4,000 members so they're a little bit bigger so they're at 1.4%. uh hospitals. There's 7,000 hospitals down to 6,000. There's about 300 in the ISAC. It's the halves and have nots. The national footprint needs to go further down below the poverty line. Energy and

power is doing much better. You got Edison Electric for the bigs. You got uh NECA for the tiny rural co-ops. You have APVA for the public power. They have a lot more participation in part because they have Nerk and Furk regulations. They have Caesar from Department of Energy. They have CEOs coming to the sector leadership instead of IT staff, but they're still, you know, in the crosshairs. So, what I'm trying to say now that we've done a year of undisruptible pilots is let's look at the nexus of these. So, we can't fix them all. Out of the 151,000, there's 6,000 that service a hospital. So, let's make a hospital town the center of our bullseye and let's look at

the dependencies it has on weak water and power and other things. So the undisturbable project was looking at this as we close the track you're going to see different talks throughout the next several days on pieces of this project but we have changed our theory of change and I'm going to outline how we can directly get you involved and maybe even your community involved because we learned a lot in the last 12 months but it was originally going to look at these four Nasier needs just the life safety stuff just water emergency power uh and food supply We started with the nexus of water and emergency care for the first 12 months. And if we were going to get more

funding, we're going to weave in power and food supply. Uh this is too much of an eye chart, but the theory of change was let's go to owners and operators of water and healthcare first. Then a year later, we're going to go to the municipal town leadership, city planners, so that when they freak out, the water people already have a great answer for the risk that we identified. We're giving people time to go through their stages of grief and be the hero of the story instead of flatfooted. And then maybe a year after that, we might go to the public. And Bryce gave me lots of great feedback on maybe we shouldn't go to the public and

maybe we won't have to go to the public. But I reserve the right to go create public demand for their to leadership and their owners and operators if it comes to that. And then we also need that long band of helping the helpers be helpful. So I need you left a boom and right a boom to start learning more about consequence informed engineering. How to talk to national incident management systems and your state local public health people how to be helpful instead of ignored and uh maybe the questions you can ask and the offers you can make to your local teams. So we don't get to sit this one out, right? I can't say not to play the game, but

maybe the safest move is not to connect. There are towns that aren't yet connected. Maybe they should stay that way. There are towns that more recently connected. Maybe they should maintain the continued ability to do manual operations, which is increasingly disappearing. There are new valves that one of my college roommates helps make that don't have a wheel on them. So if the computer that powers it goes out, there's no wheel for the human to turn. So maybe we can play some chess. But once again, in a much higher stakes than when we first said this 12 years ago, the calvary isn't coming. Like you may be the one that introduces Volt Typhoon or consequence form engineering to your

town. And what I really would like you to understand is uh the way that you could be most helpful and we're going to pivot to this towards the last day is knowing how to protect your household because if you're looking over your shoulder for the water needs, the medical needs, the food needs of your household. We don't want you to be doomsday preppers, but we do want you to be prepared. And the UK government's already asked their citizens to have three weeks of food and water on hand. They didn't tell them why. So, there's a way that you can make sure that your household's okay, maybe your neighborhood's okay, and then you can start to help your town. We're going to

equip you with critical questions to ask and suggestions and resources to provide. And maybe instead of top down central push from the federal government out, we can get to individual targeted cyber cyber poor communities and raise up Craig Newark's family of um grantees have kind of decided we all take volunteers. Maybe we can make a unified volunteer platform where people can say I'm willing to volunteer on the following topics in the following places and people who increasingly want volunteers can look one-stop shop to say who can help me on water, who can help me on power, who can help me on hospitals. So, it's not just about cyber physical systems, but the cyber resilience core CRPS is a way that

people can start to volunteer. And uh Craig's in this whole cyber civil defense idea of he wants you to do your part just like Rosie the Riveter or like they had to do when he was a kid. Um and if the cavalry isn't coming, same kind of sentiment. So that is mostly the talk from RSA. I would like to have a guest who knows way more about national security and war fighting doctrine than I could ever know. And if you were lucky enough to see him upstairs, he was the morning keynote. So I'm going to try to transition to a 15minute sit wrap on that you are national security >> and we'll take questions after him.

>> Yes. Yeah. >> Did you do at all? >> Okay. because I saw some of your slides. >> I didn't answer that. >> Okay. Um, but it's uh hopefully you're getting some stickers while they're switching laptops here. Um, I'm not saying every single one of you has to take responsibility for national security, but I am encouraging you to take some interest in your household and your community. And then Bryson's gonna give his perspective. Okay. >> I think we should clap for you. >> Oh,

so I actually uh got started as an army officer in tanks. I was a qualified tank commander. I commanded a tank platoon. And ladies and gentlemen, there is nothing better on this planet than commanding a tank. I'm not kidding. just sort of you look and let's make this real. We had a name for everybody that wasn't in a tank. Anybody know what it is? >> Crunchy. Not kidding. That was what we called dismounts. Um Josh talks about how the US government had a different philosophy on defense. And let me take that to a very pointed time. 2015 until 2015 the US government said this is not our problem cyber security for me good luck we're not even talking critical

infrastructure we're just talking the concept of cyber security was not even something the US government looked at domestically until 2015 I co-founded the IC village with Tom Van Norman and we've been doing this for a long long time and until Colonial Pipeline, no one paid attention. Nobody cared. So the US government didn't care about cyber security until 2015. And until Colonial Pipeline, going back to Josh made the comment about educating the public. Well, that was an incident that educated the public because it wasn't about cyber security. It was about the fact I couldn't get gas in a hydrocarbon economy to live to do what I want to do. And so I work with Josh and

Silas who was just here um at the Institute for Security and Technology. So how does the electric grid work? This highlights the shift. 50 years ago, electric grid was simple. Generate electricity, send it over there, somebody uses it. Easy, no problem. We don't need computers to do that. Well, and then something changed. The electric grid no longer went that direction. We have renewables. We have market economics that requires computers to work. And all that means is surface area. Anytime we add a computer to equation, we have exponentially increased the surface area. Turns out that's what this whole cyber security thing is about. How can I do something that I want to do generally unauthorized on your computer? There's

no computer. I can't do that. There is. I now got a shot. So, what is industrial control systems? It's any computer that's at least 20 years old.

And this is what we're up against. So, at the start of the keynote, I anchored it with we don't work in cyber security. We all work in national security. And that applies whether you're in critical infrastructure or not because our opponents are the proverbial nation state. That's who we're up against. And that's what makes this discipline so interesting is every day we wake up and go, how do we stop the Russians? How do we stop Russian sanctioned ransomware gangs? That's our lives. That's what we're dealing with. And the problem is is it morphed from 2015 where it was going to businesses. And this is where uh Krebs and I when he was the director gave a keynote at RSA talking

about the coming scourge of ransomware because while ransomware had happened, it hadn't yet tripped to being a national security problem. And it is now. Current estimates are that there are over 100,000 trained cyber operators in China. That's one country. And you think that number is static. They are investing. You hear about asymmetric warfare all the time and it's kind of this buzzword and I don't know and I don't want to get into the rabbit hole of whether we are at war or not but we are definitely in something and that is what we face every single day. That's the pit in your stomach when you try to take a vacation because you don't know. So why are we here? We're here because

we have a simple system that is never in isolation. Turns out pretty much everything isworked. And so when we start putting those systems together, whether it's direct, whether it's through something else or whether it's the supply chain because there's a dependency on the back end. This is what starts to make this problem. So what aren't we accounting for? Nobody expects Elon Musk people. Remember Oldsmar that got so much attention. But what actually happened? Nothing actually happened. It was a human error. But the problem is that human got really afraid when the FBI showed up and was like, "Ah, I think I've been hacked." Human error is still most of the challenge that we have. You

see this every day. In fact, I joke that's the reason we work in cyber security is because we're bad at it. We need to account for the human in this. Computers didn't get there by themselves and they don't continue to operate by themselves. So why are we here really? Because it's cheaper to bring computers and to remotely manage things. A lot of what Josh was talking about goes back to that because this is delegated down to private industry. Efficiency is the driver. We got to hell in a hand basket because it's cheaper and it's convenient right there. So what is a threat? It starts with somebody with a motive. You have to want to do something. Nobody

accidentally attacks an electric utility. It's not an accident. It's on purpose. But motive isn't enough. You waking up in Moscow and saying, "I'm going to get you drink a fifth of vodka for breakfast and throw it in the direction of America." It only goes so far. You need capability. You need capability. Capability is tools, access, tradecraftraft infrastructure trained operators. That combined is the threat. And this is where I flip the trope because there's this we talk about the defender always needs to be right and the attacker only needs to be right once. That's wrong. That only means what it looks like for them to break in. Once they're in, they need to be perfect or

they will get caught. That's your opportunity. That's what you control. They can't hack what they can't touch. Which means they're constantly looking how do they get that? How do they get in? Well, those are your computers. You put that computer there. That's your computer. You decide how that computer speaks, what protocols it uses. And then the attacker is constrained to only being able to work on those hosts that way. We have the power. It's not meant to be a pun. We are in control of this. And I wish Casey Ellis was here, but he summarized this so well. Threat actor, someone who wants to punch you in the face. A threat is the punch being

thrown. Vulnerability, your inability to defend against the punch and your risk, the likelihood of getting punched in the face. So, who is the threat? Long time ago, when I first started public speaking, I went out of my way to name the countries that were behind things. That was not common. We didn't do that. Attribution does matter. We should understand because again this isn't a cyber security question. This is a national security question and who's doing what matters. I'm really short on time. I can't go into detail here. Um Jay Healey um who's a department chair at Columbia University has a really good article um with the Atlantic um the Atlantic Council and he talks about the spectrum

of state responsibility because again there is almost nothing that is happening today in offensive cyber operations that does not have some level of a adversarial state that is involved. These are not happy accidents. So, I'm gonna pull up some quick case studies here. So, last year, um, this might have been RSA or Blackhead, I forget which. I I had a video interview and I was talking about, and a lot of people didn't understand because I was like, look, we need to take the temperature down. I had just met the week before with a small municipal um, and they were like, oh my god, the Chinese are going to get us. And here was the thing. We had turned up the

temperature of attention which led to fear. And all we had given them was fear, not direction. And so I met with them and they had this list of priorities that they were going to do. I was like, well, let's back up because what I'm about to do in these slides is I walk through, well, what exactly does the threat do? Because that's theoretically what you should be improving against. And it wasn't until the sixth thing on the list that it even affected what anything that the Chinese might do at all. We had made them afraid. They were spending money. And this is one of our challenges and part of where Josh and the community, we take

this so seriously because when we get it wrong, when somebody spends energy and time in the wrong direction, that costs us more because it's going to be really hard to earn that trust again. It's going to be really hard to go back to the well to try again. We cannot afford to do this wrong. We don't have the time for it. So the reality, it's now been zero days since Stuckset's been mentioned. I think you laughed before I even finished the joke, David. Have I said it that often? This is not our problem. Uh Dale Peterson published uh um actually this ties back to the congressional testimony. So there are a few folks who uh testified to Congress on stuckset 15

years later and pretty much all of them were like we were really surprised there wasn't more stuckset because there isn't. We aren't up against the Hollywood military weapon. We've seen a few of them but it's not the primary threat because the reality is let's look at the Purdue model. It's the nominal enterprise architecture for industrial control systems. I've got it at the top. I then cross over into an OT boundary where I typically want to have some kind of shared service level, routing, file servers, remote access, and then we get down to the individual segmented control zones. And here's the thing, the same way that this works for all of us in real life, it's job is to be on the

internet, which makes it the easiest surface area to access. I realize you are an industrial control system asset owner. I pivot through your network. I cross over and all the hard work I had to do to get through your IT. I'm now on a high level industrial control system that is a three generations old Windows or Linux and I don't need to do anything other than to tell it what to do. That safety instrumented system is alreadyworked with all those devices it's monitoring all those really unique weird this version of Modbus or Profet. It knows it. I don't have to build anything. Hackers are lazy. I will tell you as a former intelligence community guy, we were lazy, too. I only did what

I had to do. We will use whatever you have. And you already have all of this. That's the common threat model. So, we're not alone. What is the red line? You haven't heard of the Talon Manual? The Talon Manual is the utopian answer to how we should be looking at this problem. But here's the thing. It's academic. >> Being right doesn't make it so. We have a political willpower problem, not an idea problem. But we're not alone. There's a reason there has not been substantial impact to all of these operations. And one of them is that if you do something, the US military will say hello. We will not cyber you back. we will do what we're best at, which is

not nation building. I was I was an officer for Afghanistan and and Iraq, and trust me, it didn't make sense to us back then either. We're good at blowing things up, not keeping it. I got it. Um, so the US military is still a deterrence factor, but this leads back to what are their motives. So PW Singer came out with a book called Ghost Fleet, which was how the uh Chinese were able to directly hack military ships to support the invasion of Taiwan. I don't think that's how it's going to go. So back to the tank that we started with, the best tank to fight is the one that doesn't show up on the battlefield.

And the US military has yet to figure out how to teleport tanks. So we still need to coordinate and to ship them, which means that's our weakness. And that's the wobble that they're going to put in the system. And we did a project at the Army Cyber Institute called Jack Voltaek where we looked at the force um uh the interdependence of civilian critical infrastructure and force projection. Um there's public reports you can read about this. And what we we showed was how easy it is to throw that wobble off in a water plant near Fort Liberty and then those tanks don't deploy. So in summary, what do I see? And this is what Josh hinted at. I think

Taiwan will be invaded in the early 2030s. There is certainly a cultural reason for 27, which is why it is a consensus opinion, but as somebody who has experience doing combined arms, trust me, it is not so easy. The reason that the Russians have struggled so much in Ukraine is a demonstration of how hard it is to send people, even in that case, just pretty much on land, and it's still difficult to coordinate a combined arms effect. Hollywood cyber weapons aren't the problem. Risk is still a basically three to five on the list of how anybody is looking at this. Surface area is increasing faster than control control structures. We're viating our way to the next

generation critical infrastructure. We're shifting this to states. You are the cavalry. So how you can help? Um Josh mentioned the CRC. I will throw in also there's another one standing up called the civilian reserve ISAC which we're partnering with at the village um at Defcon they're going to have a booth if you want to talk to them I am the cavalry of course you can read this stuff another thing is our nonprofit the IC village is doing workforce development so people don't need a degree to get in the game and some more resources the one I just want to highlight here is because I've got Krebs Easterly and our next director Sean Planky that is Shawn uh trying to

cook >> on Unicorn Chef and if you Go watch the episode. You will understand why I say trying to cook. >> Thank you. [Applause] >> Thank you. All right, so home stretch. David and I are going to rapid fire tell you what you're in store for for the rest of the next two and a half days. Okay, so you just saw setting the table uh for the most part. Um, we're going to break for lunch. There's some other good content upstairs. The next talk is a two-hour block on water. We're going to have Virginia Wright from Idaho National Labs and Consequence for Engineering. Andrew is going to be the primary speaker and we have a

unfortunately Dean has a crisis because he's a water professional and water professionals have crises. So, he recorded something and if it works, we'll play some of that in that mix. This is one of the ones where we want a lot of discussion because of all the weak links, water is the weakest. So our highest consequence is denial of patient care in the nation's 6,000 hospital communities. The weakest link to cause that is likely water, but not exclusively water. So I would highly recommend you stick around for the water talk. That's why it's so early in the flow. No water, no hospitals, no kidding. >> Any comments on that one? >> It's important. >> You have your uh lab or no?

Do you have the the oneliner? >> Yeah. >> So for water and we're also going to talk about a training. There's uh uh consequence informed training that is going to be available as to water. Take a tumble down the rabbit hole of the wa water sector cyber landscape and emerge with real world strategies to outsmart today's most dangerous cyber threats. This session blends high stakes insight with a touch of wonderland whimsy. >> Wow. uh showing how cyberinformed engineering can create resilient water systems. If you work with water IC or just want to avoid hearing off with their heads, then you definitely want to show up to this session. >> So, we asked each speaker, can you give

us a 140 character description? Not 140 words, but so we'll we'll go a little faster, but it's going to be a great session. And um can you two raise your hands? Okay. So and uh professor kitty as well right uh the training is in the platinum you had to use eventbrite there might be some seats left there's a a session today and a session tomorrow for free training four hours on consequence of horn engineering for water and there's a panel so try to find them if you want to get into the training we're going to go a little faster on the rest of the sessions the afternoon block is a couple different talks uh Blake please

raise your hand is your co-presenter here >> okay We have a happy warrior we met through the launch of Undisruptible who does public health and emergency management in in a county in Arizona and has been incredibly helpful to helping us understand that if there's a disaster, how can we best snap in? So, we're going to have cyber incident command systems and there's lots of names space collisions like ICS is industrial control systems. It's also incident command systems. It's super confusing, but they are happy ambassadors and translators and that's one not to miss. Immediately following them, we have cascading failure unified defense with an honest to God EMT and some emergency 911 stuff. So, not only maybe could you

learn how to keep first aid for your family if healthcare was down, you can also look at like the cascading failures across these sectors. And then this is anybody have one of these or something like this or know what this is? >> All right. So, we're also going to have a small talk. We were hoping to do training. We will definitely next year. But meshtastic type, Laura type, non ham radio type. If 5G goes down from Salt Typhoon or your phone lines or your internet, can we still have nonzerocoms with each other and with our community that we wish to serve? So, there'll be a small session on that to close out today. Any other comments for today?

Okay. tomorrow. This one's stressing me out a lot, so I'm I'm not gonna miss it, but I am losing sleep over it. Um, Dr. Abba Stewart from INL, chief scientist for the grid, uh, I think is how she sometimes describes it. Uh, returning hero for the third time in a row, I think third year in a row with Manish. Uh, they're going to talk about power and the power block, but they chose to go the AI data center route, which stressed me out. And the more we talked about it and thought about it, we have to talk about it. So, as bad as things are now, um, with the frailty and fragility of no water, no hospital, no kidding, no

power, no pumps, no pumps, no water, no water, no hospital. And even circular, you need power to have pumps to have water to have power generation. So, it's very uh interconnected. All these rapidly built AI data centers are going to make it 50 times worse. So, you both need AI to fight the risks introduced by AI, and we're putting 50 pounds of [ __ ] in a five pound sack. So if you want, you know, nightmare fuel, come tomorrow morning. Okay. Um Joe Slowick in the same block is going to show how ransomware was a good training lesson for critical infrastructure disruption. These are not confidentiality of data per se. These are availability of lifeline services

and ransomware is the unavailability of many of those lifeline services. Backed by popular demand, after the lunch break, we are going to have hackers. Oh, excuse me. Emergency care. So, this is gonna be a trio. Dr. Christian Nef has come back with two new peer-reviewed data science things and a cyber crash cart overview from his ARPA h funded project at UCSD. We also have Bo Woods recruit and MVP number one from the I am the Calvary movement 12 years ago uh who started in healthcare and he's going to bridge the gap between Christian Meth as a cyber hacker doctor and a nurse that we met and spoke to last year at cyber meds summit in DC. This is the head nurse at

Ascension Health and Mlullen Health McLaren Health who got ransomed hard and they wrote a demand letter for their union saying we are not trained for this. We don't know how to do our job. patient safety is at risk. So, she's going to talk about how haring it was to be a nursing professional amidst these. We often look at this as an IT problem or a data problem, not a staff and a patient care uh problem. So, the three of them are going to walk us through a lot of conversation, everything from peer-reviewed science to how disruptive this was to care delivery. Uh so, please um try to give Dena, our nurse, uh your full attention and empathy. And then, uh

and she's really brave, too. She doesn't know cyber at all, but she's a sponge. She sucks it all up. She asks great questions. She gives good feedback. She's awesome. All right. And then who likes to eat? Okay. So, we're gonna have a hackers like to eat session. So, we have some people from the bioacc um and they do lots of different intersections of things that mostly get orphaned by other sectors. And they're both going to tell you some of the food supply constraints and choke points, but also maybe how you might think about your own food stability for your own home because it's not the way you think. So I'm pretty excited about that hourong

block. Anything to add on those? Just trying to go fast because of the time limit. And then uh to end the day, Silus was in the room, but Silas Cutler uh who works at Census IO now, but is also faculty for IST uh he published. So you saw the Purdue model that um uh was just shown as to how many owners and operators segment and isolate their OT control systems and ICS and and everything. Well, he found 400 uh OT controls naked directly accessible on the internet with no passwords or validation. He worked with the EPA to get them offline quietly and that happens pretty often. So, you know, we like to have this multi-layer defense.

We don't always get it. So, some combination of Silus with some of his findings for his current ongoing projects to scour the internet for the lowest hanging fruit, but also Paul Roberts with several nonprofits like Right to Repair uh and other things and Stacy Higin Bothotham from Consumer Reports Advocacy. They're kind of looking at how end of life software policies or SUAX and end of life software can lead to end of life for humans. So some combination of we know some of the problems, we have some policy proposals, none of them had had political will and we wonder if this typhoon suite could create political will somehow. So they're going to wrestle and talk through EOL software

should not be EOL for humans and that will close out day two and then why don't you start day three because you're one of the speakers. >> Okay. So Wednesday um we open up with cyber civil defense volunteers to the rescue. Volunteers are the backbone of cyber civil defense. If you because you're here I think you are ready. If you're ready to join the community cyber defense fight but don't know where to start, this talk is for you. This is Wednesday morning. We'll map out the current volunteering efforts, pinpoint the crucial coordinated strategic actions still needed, and be your onestop shop to identify which volunteer groups to join in. So that leads us off on

Wednesday morning, followed by neighborhood and household resilience, a month without external support. So I will be talking about things that you can do at low cost, relatively low effort to make your household and I would say even more importantly your neighborhood resilient to crisis. We're going to talk about it. There's don't feel overwhelmed. Feel simply wellelmed because you can definitely do it. And then finally, Josh is going to close us out with tying it all together and trying to bring forward lessons and vignettes from each of the sessions to to build a single unified model and really a a course for action moving forward because we want to leave everybody with some homework to do when

you go home because you're really your work is not done here. The work starts here, but it continues when you go back to your home, your community, your school, your work, the people you interact with every day. So, um, part of that session is the slides that he encouraged me to do from RSA for you this morning were the first year's pilot. I am very happy to say part of what we're going to do to close this out besides synthesize everything is we're going to dig into what is the next two years. So, uh, to his credit, Craig Newark didn't only fund the first year. He announced and we announced on stage at Critical Effect in DC. Um, he's

provided $3.2 million to fund the next two years for a much more aggressive, robust project plan and roll out that you could participate in. And we're going to show you how you could snap in the uh, without killing all the cool details. Part of the idea is our theory of change is we had an information gap. We did not just have an information gap. We had a motivation gap and an enablement and power gap. Uh the other thing is with so many to reach and so little time, we're going to innovate narrowly and replicate widely across a two-year project plan. What does that mean? We are going to have funding for 12 cities, 12 towns, 12 communities. is

diverse a composition of philosophy, budget, red states, blue states, urban, suburban, rural, near floodwaters, near cities, near ports, whatever. We're going to pick 12 really diverse communities that have a hospital in them. And we're going to meet the hospital, the power, the water, the municipal leadership, and emergency disaster management for the state and county all together. And we're going to cross through the motivation and the enamel. will even pay for some of those mitigations and we're going to make playbooks and capture their stories before, during, and after. We believe the best ambassadors for their peers of people who look and talk just like them. So, the co-creation, the storytelling, and they spread to the other 6,000 and

then the other 151,000 of water plants across the US. So, people are never surprised and they have proven playbooks from people who look and talk like them that are practical and deployable using consequence informed engineering. We're going to do a bunch of tabletop crisis simulations, but also some physical damage demonstrations and including using mainstream media sources and congressional teammates. So, we want you to help us pick over the next two days. Think about what you're hearing and say, "My community might be one of those 12, Josh." So, we're in the target selection mode right now, like right now. We hired an amazing project lead today and we're still hiring some more. So this has gone from a concept to

aggressive campaign and I want the people in this room to help us target those. So we're going to get into more details on how that can happen to close out the session for our two and a half days of awesome sauce and then we're all going to go upstairs and hear Casey John Ellis close the keynote. So if you're interested come back. It is lunchtime but I will talk to anybody for as long as you want >> the training. >> Yeah. >> On the training. >> Give him the mic. Yeah. on the training emphasized water, but in reality, engineering applies all over the place. And so, you might learn some water if you come, but if you're at all

interested in how to engineer this problem out that part of that slide and you're not a water person, come anyway. >> Yes, the water hammer is not the only threat. It may not even be the right threat, but it's a great storyline to get people to the table, solve something, learn CIE, and then rinse and repeat on the other hazards that we may encounter. All right, give yourself a round of applause for hearing for an hour and a half, and we hope to see you at the water block >> after lunch. 2 p.m. >> 2 p.m. 2 p.m. Thanks, everybody. [Music]

[Music] Heat. Heat. [Music]

[Music] [Music] Fire down.

[Music]

Heat. Heat. [Music] Heat. Heat.

Heat. [Music]

Heat. Heat. [Music] Heat. [Music] Heat. Heat. [Music]

Heat. Heat.

[Music]

Heat. Heat.

Heat. Heat. N. [Music] Heat. Heat. N. [Music] Heat. Heat. N. [Music]

[Music]

[Music]

Yeah. [Music]

Woo! Wow! [Music] Heat [Music] up [Music] here. [Music] Heat.

Heat. [Music] Heat. Heat. [Music]

Heat. Heat. N.

[Music] Heat. Heat. Heat. Heat. [Music]

Heat. Heat. [Music] Heat. Heat. [Music]

Heat. Heat. [Music] Yeah, [Music]

[Music] down. [Music] Hey hey hey. [Music] down. [Music] Down.

[Music] Heat. Heat. [Music] [Music] [Music] down. [Music] Hey [Music]

Down. [Music] Down. [Music]

[Music]

Heat. Heat. [Music] Heat. Heat.

[Music] Heat.

Heat. Heat. [Music] Heat. [Music] Heat. Heat. [Music] Heat. Heat. Heat. [Music]

Heat. Heat. Heat.

[Music]

Heat. Heat. Heat. Heat. Heat. [Music] Heat. Heat. [Music]

[Music]

[Music]

Yeah. [Music] Heat. [Music] Heat. [Music]

Woo! Wow! [Music] Heat. Heat. [Music] Heat. [Music] Heat. [Music] Heat. Heat. [Music] Heat. Heat.

Heat. Heat.

[Music] Heat. [Music] Heat.

Heat. [Music] Heat. [Music] Heat. Heat.

Yeah, [Music]

[Music] heat. [Music] Black. [Music] Yeah, [Music]

down. [Music] Heat. Heat. N.

down.

[Music]

Hey. [Music] Hey. Hey. [Music]

[Music] [Music] Corn [Music] baby. [Music] Fire.

Home. [Music]

[Music] Heat. Heat. [Music] Heat. Heat.

Heat. [Music]

Heat. Heat. [Music] Heat.

Heat. Heat. Heat. [Music]

Heat. Heat. N. [Music] Heat. Heat. Heat.

Heat. Heat. Heat. [Music]

Heat. Heat. [Music] Heat. Heat. N.

[Music] Heat. Heat. Heat. [Music]

Heat. Heat. [Music]

[Music] Ooh. [Music] Ooh. Heat. Heat.

[Music] Heat. Heat. [Music]

Woo! Wow! [Music] Heat. Heat. Heat. Heat. [Music] Heat. Heat. [Music] Heat. Heat.

Heat. Heat.

Heat. Heat.

[Music]

Heat. [Music] Heat.

[Music] Heat. Heat.

Heat. Heat. N. [Music] Heat.

Heat.

Yeah, [Music]

[Music]

yeah yeah. [Music] Hey hey hey hey hey hey hey hey hey hey hey hey. Yeah, [Music]

down. [Music] Down

[Music] Woohoo! [Music] Woohoo! [Music] [Music] Baby, [Music] baby. [Music] Hey, hey hey.

[Music] Down. [Music]

[Music] Heat. Heat. [Music] Heat. Heat.

[Music]

Heat. Heat. Heat. [Music] Heat.

Heat. Heat.

[Music] Heat. Heat. Heat. [Music]

Heat. Heat. Heat.

[Music]

Heat. Heat. Heat. [Music] Heat.

[Music]

[Music]

[Music] Heat. Hey, Heat. [Music] Hey. [Music] Heat. Heat. [Music]

Wow. [Music] Heat. Heat. [Music]

Heat.

Heat. Heat.

[Music] Heat. [Music] Hey, heat. Hey, heat.

Heat. Heat.

Heat. Heat.

[Music]

Heat. Heat.

[Music] Heat. Heat.

Heat. Heat. N. [Music] Yeah, [Music]

[Music]

[Music] heat. [Music] Hey, [Music] hey hey. [Music] Yeah, [Music] you [Music] Down

[Music] Heat. Heat. [Music]

[Music] by [Music]

[Music]

Heat. Heat. [Music] Heat. Heat.

[Music]

Heat. Heat. Heat. [Music] Heat. [Music] Heat. Heat. [Music] Heat. Heat. Heat. [Music]

Heat. Heat. Heat.

[Music]

Heat. Heat. Heat. [Music] Heat.

[Music]

Heat. Heat. [Music]

[Music] Ooh. [Music] Ooh. Hey, [Music] hey hey. [Music] Heat. Heat. [Music]

Wow. [Music] Heat. Heat. Heat. [Music]

[Music] Heat. Heat.

[Music] Heat. Heat.

[Music] Heat. Heat.

Heat. Heat.

[Music]

Heat. [Music] Heat.

[Music] Heat. Heat.

Heat. Heat. N. [Music] Yeah, [Music]

[Music]

[Music] heat. [Music] Hey, [Music] hey hey. [Music] Yeah, [Music] down down.

Down

down down down.

[Music] down. [Music] D. [Music] D hey.

Heat. Heat. [Music]

[Music]

Heat. [Music] Heat. [Music] Heat. Heat.

[Music] Heat.

Heat. Heat. [Music] Heat. [Music] Heat. Heat. [Music]

Heat. Heat.

[Music] Heat. Heat.

[Music]

Heat. Heat.

Heat. Heat. [Music] Heat. Heat.

[Music]

[Music] Hey. [Music] Heat. Heat. [Music]

Wow. [Music] Heat. Heat. [Music]

Heat.

Heat. Heat.

[Music] Heat.

[Music] Heat. Heat. Heat.

[Music] Heat.

[Music] Heat. Heat.

[Music] Heat.

[Music] Heat. Heat.

Heat. Heat. N. [Music] Heat.

Heat.

Yeah, [Music]

[Music] heat. [Music] Hey hey hey hey hey hey hey hey hey hey hey hey. Yeah, [Music] down.

[Music] Down

down down down down.

[Music] Heat. Heat. [Music]

Heat. Heat. [Music] Baby, [Music] baby. [Music] D hey do [Music] down [Music]

Down. [Music] Hey. Hey. [Music] Heat. Heat.

[Music] Heat. Heat. [Music]

Heat. Heat.

[Music] Heat. Heat. [Music] Heat. Heat.

Heat. Heat. Heat.

Heat. Heat.

[Music] Heat. Heat. [Music] Heat. Heat. N. [Music]

Hey, [Music]

[Music] hey hey. [Music] Heat. Hey, Heat. [Music] Hey, [Music] hey hey. Heat. Heat. [Music]

Wow. [Music] Yeah. Heat. Heat. [Music]

[Music] Heat. Heat.

[Music] Heat.

Heat. [Music] Heat. Heat. [Music] Heat. Heat.

Heat. Heat. [Music] Heat. Heat. [Music] Heat. Heat. [Music] Heat.

Heat.

[Music] Yeah, [Music]

[Music]

down. [Music] Hey, [Music] hey hey. [Music] Yeah, [Music] you [Music] Down

down down down down.

[Music] Heat. Heat. [Music] Heat. Heat. [Music] Wonder.

[Music] [Music] Baloo. [Music] There you are. [Music] You [Music] do.

[Music]

Down. [Music] Ah. [Music]

Heat. Heat. [Music] Heat. Hey, heat. Hey, heat. Heat. Heat. [Music] Heat. Heat. [Music] Heat. Heat. Heat. [Music] Heat. Heat. Heat. [Music]

Heat. Heat. Heat.

[Music]

Heat. Heat.

Heat. Heat. N. [Music] Heat. Heat. [Music]

[Music]

[Music] Hello. Wow. [Music]

[Music] Heat. [Music] Heat. [Music]

Wow. [Music]

[Music] Heat. Heat.

[Music] Heat. Heat. [Music]

Heat. Heat. [Music]

Heat. Heat. N.

Heat. Heat. [Music] Heat. Heat.

[Music] Heat. Heat.

Heat. [Music] Heat.

Heat. Heat.

[Music] Yeah, [Music]

[Music] down. [Music] Hey hey hey. [Music] Yeah, [Music] down. [Music] Down

[Music] Woohoo! [Music]

[Music] By far. [Music] Heat. Heat. N. [Music] Fire.

Home. [Music]

[Music]

Heat. Heat. [Music] Heat. Heat.

[Music]

Heat. Heat. Heat. [Music] Heat. [Music] Heat. Heat. [Music] Heat. Heat.

[Music] Heat.

[Music] Heat.

Heat. Heat. N. [Music] Heat. Heat. [Music] Heat. Heat. N. [Music] Heat. Heat.

[Music]

[Music]

[Music] Hey, [Music] Heat. Heat. [Music]

Wow. [Music] Heat. Heat. Heat. [Music]

[Music] Heat. Heat.

[Music] Heat.

[Music] Heat. [Music] Heat. Heat. N.

[Music] Heat. Heat. [Music]

Heat. [Music] Heat. [Music] Heat. Heat. [Music]

Heat. Heat. [Music] Yeah, [Music]

[Music]

down. [Music] Hey hey hey hey hey hey hey hey hey hey hey hey. [Music] Yeah, [Music] you [Music] Down

[Music] Get away [Music] Thanks everybody for being here. I I have a boat book full of notes from Josh's and uh Bryson's presentation this morning. So, we'll try and attempt to continue to weave some of those themes in. And um anyways, appreciate being here. Thank you all for being here. Uh my name is Andrew Ort. I am with West Yoast Associates, which is a uh engineering consulting firm, Water Wastewater Only. Uh we've got about 250 people now based in California, although I live in Northern Minnesota. If I go down the street a little bit and look, uh, I can see Lake Superior off in the distance, which is 10% of the world's, uh, fresh surface water. And, um,

anyways, it's a pleasure to be here talking about cyberinformed engineering with you all with Ginger Wright. >> Thank you all so much for coming this afternoon. Uh, I'm Ginger Wright and my work is in cyber security for operational technology systems and my current passion which you will learn a lot about today and have an opportunity to learn even more is cyberinformed engineering and we'll talk about what that is as the story goes on. But essentially, it is a secret weapon in defending critical infrastructure that we can use more to make an adversar's action not work to avoid the worst impacts of a cyber attack. And I think that's something that we all want to know about, especially since this

morning Josh raised our awareness of some things that are pretty scary and concerning. And so we wanted to come back right after lunch and at least reassure people that there are solutions. There are ways that we can help our industries solidify their defenses and some of it uses engineering, not just fighting cyber with cyber. >> So, uh, we did have a third panelist. His name is Dean Ford. Unfortunately, he had a a work emergency that he had to attend to. So, I just want to give him a shout out. He provided some slides. We'll go through those sorts of things. I'm curious. He gave a really excellent presentation at Bides Las Vegas last year. Were any of you two

>> last two years? >> Last two years. Thank you. Were any of you present for that? >> Couple. Okay. Wonderful. So that's a little bit of background. I'm curious who here works in the water wastewater sector. >> Anyone? Okay. Wonderful. >> Who here is dependent upon the water wastewater system for your work? >> Oh, that gentleman is not. >> Sorry. All right. And then uh who here has heard of cyberinformed engineering before? Couple of hands. Anybody practicing CIE? No. Okay. Well, the goal is that you can walk away from this with some ideas on how to apply CIE and then also uh maybe attend the trainings later today and tomorrow morning uh where we're going to

do a very detailed deep dive um before hours in each session. And um unfortunately there is a little scheduling overlap today. So Ginger will be hopping out midstream um from this panel discussion. So, what I would ask is that as Ginger says something, as I say something, if you've just got questions, just interrupt, please. Um, that's just going to make this much more uh interactive, and that's really the goal. So, with that, you know, I if you know Josh, uh you probably know that he's a big Avengers and and comic book hero fan. And so, he he said that the Avengers had assembled um to think big thoughts and do big things and and help

the country and the national security of the US. So, I was thinking about like, okay, well, what's my role? like who do I you know think of myself as in that universe and I landed on Happy who if you know drives Tony Stark around for two reasons one I love cheeseburgers and two it's really my job to get the defenders of our infrastructure to go where they need to go to do whatever they're going to do so very similar in roles and uh there was also a lot of discussion earlier about how we don't have a cyber defense force so I used to get up in front of you know rooms like this and you know primarily ly water

sector people, engineers, operators, IT, OT, you know, all of the different roles. And I would ask a question. I'd say, who here considers themselves to be defenders of their infrastructure? And I would get two hands up. So let's pause here. Who here is defenders of our infrastructure in this room? Okay, wonderful. when I used to ask that question, you know, five, six years ago, get two people and turns out that, you know, the reason that they had that kind of hallowed look in their eyes was that they were from it, right? And they were just getting hammered day in and day out. Now when I ask that question about a third to half the room raises their

hands which is really wonderful because from my perspective and I think from a CIE perspective our cyber defense force is the operators and the engineers and the IT staff and the OT staff and they really need to adopt this perspective and we're starting to in our sector. So uh those are you know kind of some some starting thoughts to bring us back to the uh the prior conversations. So, we've all relied on water at some point today. Um, we probably drank some water. We probably made a contribution to the wastewater system. And yes, I use that term very specifically. Um, a friend who helps uh run the city of Sacramento, Department of Public Utilities, taught

me that it's all a contribution. It's full of resources. Let's treat it accordingly. So, if we think about where our water comes from, right? There's a watershed somewhere. When I go and I do risk and resilience assessments in the sector, the best days are the ones where I have to go into the coastal range of Oregon and look at the source water. Um, drive through all of the national forest land, which most people never get a chance to go to. So, those are the best days. But anyways, generally high elevation up in the Cascades, something like that. Um, we have a natural reservoir of water. That water is then brought into a water treatment plant.

It's treated and it gets pumped out into a distribution system. In that distribution system, there are going to be pump stations to make sure the pressures are right because, you know, there's always changes in elevation and of course there are going to be water reservoirs to store water. Um, and this is going to include, you know, large reservoirs that are on the ground and also the water towers, right? And uh, you know, a funny joke from the the sector is it's funny how uh, towns always name themselves after their water towers, but a bunch. Yes. Thank you. That was a joke grenade. >> It was. There we go. And then we have all these mains. You know, some large

water manes can be uh 10 feet in diameter. That's really big, right? Um you think about the valving and this the design and the structural engineering that goes into those. We haven't even gotten to the control system yet. Um it's really impressive. But most mains are probably in the uh you know 2 inch to to 48 inch depending on their function and how many people they're serving. Water is conveyed to the hospitals to your homes. Um and then of course we use it and we put it into the the wastewater system and it's conveyed away from us to a wastewater treatment plant where it's treated made safe and then discharged into uh a receiving water body whether

that be groundwater or surface water. Now, we're going to be very focused on water today, but we don't want to lose sight of waste water. And I'll bring that up a couple of different times in a couple of different ways, but um yeah, we just want to make sure. >> Speak for yourself. I'd like to lose sight of wastewater. >> Fair enough. All right. So, we'll come back to this a little bit later. So, what we want to do is is kind of give a little bit of background. And this is very much based on the content that Dean provided based on his presentations the last two years. So where does water come from? We talked

about that a little bit uh already. Who or what uses water? What are some of the threats? And we're going to start from an all hazards perspective and then questions questions questions. Please don't hesitate. So some some of Dean's foundational principles, right? This idea that there's no accidents. Um certainly there are probably more failures of imagination than anything. uh people are very much centered to this. The executives have to be bought in, the engineers have to be thinking about it, the operators have to be aware and they all really have to be defenders. Um cyber can really be a wonderful unifying conversation especially within the context of CIE which we'll get to. Cyber of course is one of the many risks

that we manage. A lot of my clients convey water in seismically active zones. A good shake is going to be really hard on all of those linear assets, right? And um it's pretty hard to replace those. Costs a lot of money and takes a lot of time. And then of course I think probably just like every sector, Ginger, technology is outpacing our ability to maintain it and possibly secure it. >> Everybody wants the benefits of technology, but very few of us are willing to pay the price. >> So briefly, where is the Earth's water? Primarily in the oceans. Um there's a little bit in fresh water. When we think about fresh water, we've got uh about

2/3 of it in the glaciers and ice caps, one/ird of it in groundwater, and a little bit of uh surface and other freshwater sources. When we think about those surface and other freshwater sources because those are what we can easily use. Um a lot of that is in ground ice and perafrost about 20% in lakes. The affforementioned uh Lake Superior is a big one of those. And then lots of other places, swamps, marshes, ponds, etc. So um we're very focused on the easy to get water because that's the least expensive. Now of course there are some desell plants or desalinization where they can take ocean water, treat it and uh provide drinking water based on that. In the US that's not something

we do much of. That's primarily in the Middle East and Israel. So I think I've talked about this already but it's a really nice slide kind of showing the the water cycle. So, of course, you know, water flows downhill. Um, and we get evaporation out of lakes and and oceans. Uh, wind car the weather carries that upstream, creates snow pack, glaciers. Some of that water that melts goes into the ground, goes into um surface water bodies like streams and I don't remember, Ginger, was it creeks or creeks? Probably depends on where you're from. >> Creeks for me. >> Okay. Creeks for me. All right. So, it does differ quite a bit from electricity and and uh Josh and

Bryson talked a little bit about the what the water sector really looks like. So, depending on how you define a water system, there's 151,000 water systems. Now, if we look dive into that number a little bit, there's going to be about 500 of those systems that serve a 100,000 people or more. About 500 systems serve 50,000 to 100,000. And then about 9,000 systems serve 3,300 people to 50,000 people. Right? So we have this huge diversity of scale of operations. And you might be thinking 3,300 is kind of a weird number. Why would we pick that? That's going to represent about a thousand connections. So you say one connection is about 3.3 people on average. Now those systems serve about 80,000% or

sorry 80% of the population and we have 16,000 public water systems that serve about 75% of the population. Um water systems are very local. They are not widely connected. In the off chance that there is a connection, it's very rarely sized for anything other than a very uh limited emergency um support. And often times water systems do not like to test those connections which are usually you know there's two pipes coming in there are two valves and then potentially a third valve in the middle. Um they don't want to test it because it can change the direction of the flow of water which um can cause all sorts of sediment to flow in ways that it hasn't flowed

before cause all sorts of issues. So um that's not something that's widely done or widely used even though in some cases those connections exist. Um the Bay Area is actually a really good example where there are lots of connections. They just aren't exercised or used. Um let's see. We do have to of course treat waste water to prevent sanitary and uh you know pollution from getting into primarily our surface water bodies. So uh two quick stories on that. One is um if we have uh any Chappelle Show fans from back in the day, of course there's the Charlie Murphy skit where he talks about the purifying waters of link Lake Minnetonka in the one of the Prince

skits, right? So Lake Minnetonka is beautiful. If you ever have a chance to go, it's it's wonderful. However, um Met Council, which was which is the regional wastewater provider in the Minneapolis St. Paul area and the surrounding seven county area. They um had a operational mishap and they had to release a bunch of waste water into Lake Minnetonka. It lost that sort of purifying status briefly and they had to just let nature kind of do its run its course. And the reason that they had to do that was is they couldn't convey water to the plant effectively. They had two options. One was to let the wa waste water flow into the lake or two let people's basements

fill up with waste water. So they chose the lake. That's a pretty bad day for a wastewater utility. Um, similar things have also happened in Hawaii along Wiki Beach. So imagine you take your family to Wik Ki, there's a wastewater release. Nobody can go in the water even in the ocean, right? So um, pretty bad days. It sounds pretty shitty, said Josh. So there we go. All right. Who's using it? Right. Residences, of course, we're using it. uh the casinos using it as a commercial entity. Uh we use a lot of water for cooling and heating. Um one of the things that is really emerging is the concern around water resources and data centers. I don't know if have you

looked into this at all Ginger? >> Um some of my peers have looked into it and I believe there will be a presentation on that tomorrow morning. >> Ah wonderful. Ah thank you Emma. So everybody uh go to that presentation tomorrow. So, um I actually have done work for a utility in the national capital region that serves a lot of data centers and they um they actually had a mishap where a data center was about this close to shutting down for you know one of the big cloud service companies and it caused them to hire me for a year full-time to help them figure out emergency management. It was not cyber at all. It was just how do we

communicate? How do we take all this data that we already have and get better? Um energy of course you there's a lot of places where you can't um generate energy without water public safety fire hydrants and sanitation um fire hydrants right that water doesn't need to be safe to drink right so there's always a little bit of tension in do we provide fire water if we can't provide safe drinking water healthcare which we'll talk a little bit about more food transportation and then recreation which we've touched on. All right. So, um, you know, this is a kind of a complex graphic of the source and use of fresh water in the US in 2015. Um, about twothirds of the water

came from surface water. About a third came from groundwater. Um, in places like California where groundwater resources are dwindling, there's a huge movement to create new surface water based utilities. So, there's kind of a the the pendulum swings back and forth a little bit over the decades. And right now we're swinging towards surface water. And you can see, oops, not sure how that did that, but you can see that there's lots of uses for water. And of course, it was touched on this morning a little, the food, energy, and water nexus. That's a big um topic of conversation. And Emma will cover some of that tomorrow. So Dean's, one of Dean's clients, I don't know who it was,

but they do serve um nearly 800,000 people, provided this quote. So really wonderful quote about the importance of water and hospitals and that connection. So hospitals are the most critical customers that we serve. Even a few minutes without water is detrimental and prevents a major life safety threat. Our hospitals are some of the most significant water users in our system. ABC health system is a as a whole is our largest customer by volume and revenue. Each hospital is also in the top 15 users monthly report. So that really illustrates the importance and the connection of hospitals and water. So if we look at how a hospital is actually using that water. So 42% um sanitary HVAC 23% all that

heating and cooling. Interestingly medical processes 14% right. And my guess is that associated with this is there's probably a lot of on-site treatment systems to get that water chemistry just so and then we have laundry cafeteria and miscellaneous and uncounted for um water usage. So that just gives a sense of how water is actually used by a hospital. Now interestingly this is from the Massachusetts uh water resources authority and um they are one of the larger utilities in the c country. they actually serve the greater Boston area. So, domestic water use in gallons per day per person and the growth of between 20 uh sorry 2000 and 2020. So, you can see um we're here in Nevada. So, there's

a pretty high water use compared to other states. Um domestic water use here is uh between 101 and 125 gallons per day. And the population here is growing like crazy, 55% in those 20 years. So one of the things that um we you know are observing here if you you know take some time and look at this people are moving out of some of those northern states and going to some of the warmer states and western states and that's causing water stress in different ways. So some threats that we always have to be concerned about is you know concerns around the quantity and quality of water. And we'll get into some of this when we talk about the cyber portion a

little in a little bit. droughts, quality impacts, wildfires are rampant, of course, and those it just causes chaos for water quality. Um, physical threats, you know, contaminant inserted into supply. There was, I think a 2013 uh CBS ran an episode of NCIS where the plot of the story was that um a bad actor injected some bad ethylth ethylmethyl bad stuff into uh the water system upstream of a military installation. And they were able to do this because they just connected a pump um up to you know the uh the backflow um in the uh basement of the house, there was no valve protecting that backflow. And so they were able to pump it in. And

AWA, the American Waterworks Association, which is really the premier industry organization, actually sent uh CBS a, you know, a letter saying, "Hey, this can't be replayed. You need to take this episode down." Because it created such a detailed map on how somebody could take uh conduct this attack on a water system. um aging assets of course uh everything's getting um you know quite aged but I will say that utilities are really doing a much better job of this than they were um 10 years ago and I was the American Society of Civil Engineers has a in infrastructure scorecard that they issue every year and I want to say that in the water sector we've gone from

like a D minus to a C minus in the last 10 to 15 years which doesn't sound like a lot but think about maybe uh if you were in high school and you brought home a C minus instead of a D minus to your parents. That'd feel pretty good, right? Right. So, as a sector, I feel pretty good. Yes. >> Don't you think they're worried more about lead line replacements than cyber >> Yes. into the mic. Don't you think they're more worried about lead pipe replacements than cyber security right now? That's where the money's going. So, we'll get into what they're most worried about, but I would say that you are absolutely right. The money is flowing

to lead line replacements and it's a huge concern. Um, especially when you get into, you know, schools that still have all that old plumbing and that sort of thing. Uh, natural hazards, of course, we always have to be worried about those. Now, one of the interesting things that h is happening is regionalization of water treatment um and water systems. So there's um oftentimes a large wellunded system with less well-unded smaller systems around it and um the big system buys the little system, right? And it's kind of funny, you know, if the water systems are mostly municipal, you wouldn't think that they would be um you know uh acquired like that, but that absolutely does happen. It creates some unique

challenges from a technology perspective, especially as you you know get multiple SCADA systems with different technologies, different levels of security, etc. and you have to combine all those. So that's a real challenge. State and local control, we have tons of regulations. Same um at the federal level, but from a cyber perspective, there really is no federal uh we'll use the term Dean chose here, control on cyber pollution. I'm not going to spend a lot of time at uh most of this, but you know, we have forever chemicals, PAS, PAS, uh storm water, and climate change is a big deal. We're looking at people at aging workforce. I would say that a lot of my clients have have sort of

gotten through the silver tsunami and they have uh the average age of their workforce has probably dropped about 10 years in the last five years which is a huge win for them. Um funding of course is always a challenge. Uh just to kind of touch upon chemical costs. So I think between 2019 and 2022 when um when I had the opportunity to present with the director of Minneapolis water she had said that the uh disinfectant costs had gone up four and a half times in three years. So that was really just you know over co um let's see so from a cyber and operational technology perspective um you know there's lots of different things here but you know it's OT right

if it works don't touch it I almost grabbed a a meme of a an old network switch you know covered in about half an inch of dust and it's like it's still working don't do anything so um you know the good and and some of these do cut both ways for sure gravity is our And right once you get that water up to an elevation, you can leave it there and it'll drain out as people use it, but you have to take a lot of energy to get it up to that elevation. Often times um our product uh production process is relatively simple. It's relatively slow and it doesn't really go boom compared to other sectors. Uh the distribution

process can be relatively simple. Although I would say that if you had to sit in one of these control rooms and look at eight screens with some of them very focused on, you know, 20 um remote sites with changing pressures and everything, it does get fairly complicated. The government finally recognizes water is critical. Um for the emergency managers in the room, right, water didn't have its own emergency uh ESF. Um we were emergency support function. Thank you. we were actually buried under public works and we're finally getting out from that which is a really good thing because what does public works mostly care about? Roads. Um regulatory agencies are definitely engaging and of course we're all of our

technologies are converging. All right. So uh to get to an earlier question here, what do we in the sector really think about cyber security at this point? So, every year on around June 1st, the American Waterworks Association releases the state of the water industry report. Um really, really um interesting data. They um they pull, you know, thousands of utilities. Um and the top issues facing water utilities are listed here. You got the top 18. Of course, money is the first one. Do we have enough money to do the things we need to do? But if you go down, number eight is cyber security issues, right? So that is above droughts. It's above um a lot of

other things that you would think that they'd be very concerned about. And this is a really substantial change over the last few years. So the text here is very small. Um I will just sort of explain this. So if we move from let's see right to left on this slide and I grabbed this out of the report. We start on the right with 2020 and we move on the left as we go left up to 2025. And what we find is that for the first three years 2020, 2021 and 2022, cyber security was not included in this list of top issues. Then cyber pops up at the same level for 23 and 24 and now it continues to move

up in 2025. Now, as we look at the different sizes, I kind of talked about how utilities um are are sized and there's different ways to do it. So out of the 2,000 that responded the the medium large system and very large systems this is their most important um priority right now is investing in cyber security. If we look at small systems it drops way down to seven and if we average it out across all of them it's second which is really wonderful. Now this is probably the most important thing that I take away from this sorry about that is that utilities are saying hey what do we need to innovate around cyber security and from

my perspective cyber informed engineering actually provides a lot of that possibility now Josh presented this earlier this is a list of attacks and 12 of them are actually water wastewater utilities and Josh you mentioned earlier that you're surprised that this didn't stay classified you know I am as well. So, briefly regulatory uh background, the Safe Drinking Water Act does now require cyber security assessments. Uh you have to do it every 5 years. More and more states are requiring them every year. Josh mentioned that there was earlier today that the sanitary surveys did have a cyber security component and Iowa, Missouri, and Arkansas a couple along with AWA and the National Rural Water Association did sue. And a big

part of that was is that it didn't matter if you served 8 million people like New York City or a 100 people like you know the trailer park down the street. They had to adhere to all of the same constraints and requirements and that you know really just isn't an equitable approach. One of the things that's been proposed is the water risk and resilience organization which is a functionally a water nerk. So if I say nerk does is everybody generally familiar with that? getting some nods. No. So, Ginger, do you mind doing a quick 30 seconds on Nerk? >> Sure. In the energy sector, NERK aids the utility population to set and abide by a set of requirements for high and

medium criticality assets and the asset owners agree to abide by those standards. Nerk helps to enforce the agreement for that standard. Um and ultimately the policymaking body that controls all of this is an organization called FK or a commission called FK. >> Thank you. >> So we would have an industryledd organization that's providing cyber and physical security standards and auditing of the application of those standards in the water sector. All right, let's get on to the main event. Ginger, >> it's not here yet. >> It is not here yet. legislation has been introduced and we've been waiting for it to be passed for several years now. >> EPA. >> So the question is is would this WORU uh

organization report to the EPA? So the expectation is that um similarly to Nerk and Ferk which report to DOE as the sector risk management agency for energy that theRO would report to the EPA as the water wastewater sector sector risk management agency. Yes. All right. Okay. So I work for the Idaho National Laboratory. One of the things that we are privileged to do is security assessments on critical infrastructure. Many of those security assessments focus on the cyber security of that critical infrastructure. And as we looked across all kinds of asset owners from all of the 16 different sectors, we saw something missing in almost every assessment that we did. The IT and the cyber teams were over here and they had

policies and procedures for how they looked at the digital layer of that asset owners infrastructure. The engineers and the operators were over here and they had policies and procedures for how they ran and controlled and designed the engineering and the physical part of that asset owner's infrastructure. But there were very few places where there was an admission that what happened on the digital side could affect the physical side. And inevitably the digital side just wanted to tell the physical side what to do. Here's what your password length is going to be. Here's how you're going to log in. Here's how your remote access policies are going to work. But they never said, "Hey, you all run the plant 24 hours a

day, seven days a week. What's the worst thing that could happen that we on the digital side of the world really ought to know about and that ought to motivate what we do? We noticed that conversation was missing. And so we started to pull the thread of what would happen if we could get engineers more involved in assuring the the physical side of reliability from digital ad digital adversaries and digital adversary effects. What would that look like? What would an engineer need to know? Right now, according to AET requirements, and AET is a certification body that certifies universities to teach engineering, we send an engineer to go take a cyber security class. Does anybody here know what they learn?

[Laughter] >> In short, yes. They learn about adversary threats. They learn about the attack chain, but only from a digital standpoint. They learn about how protocols work and how protocols are taken advantage of. But nowhere, nowhere, nowhere do they learn how to apply engineering risk management to what would happen to my production capability if an adversary got hold of critical equipment. and of all the critical equipment that's part of my production capability, where could that adversary do the most damage? So that when I then reach out to these IT and cyber people, I'm having them focus on the thing that my company needs the most. It's very very strange to have a group that is trying to protect a set of

assets that they know very little about and that they don't ask the owners and arbiters of those assets, what do these things do anyway? How do they work? What is the the manager in charge of your thing care about the most? How can I help you achieve that mission? Instead, it's, hey, I've got a checklist of things on the cyber side that must be done for me to get out of here. So, can we just do my checklist? So, with this idea in mind, oh, do we have a question? Yeah, I was going to let you finish, but since you asked, um, I work with with a with a water company and one of the and

then I listened to a presentation to the water isac group from the EPA and they sort of said they were in charge and somebody asked them, well, what about CES? Do they like Bigfoot you or you know, how does that work? So my question is, how many organizations out there that want to regulate a particular water district and how many organizations are out there that want to sort of help at the same time. >> So the answer is different >> and well and the answer is changing in time. So you are right. Um, a couple of years ago it was likely that there would be regulation at the EPA standpoint and at the CISUS standpoint and maybe

several other organizations involved with setting how an asset owner might comply with the regulation. What's the kind of compliance model? Does an asset owner set a riskbased approach and then demonstrate how they're achieving it? Um, as we move into this federal cycle, there seems to be less interest in the federal government driving very harsh regulatory standards. That doesn't mean it won't happen. And I certainly do not have a crystal ball to predict where regulations going in this administration, but I perceive that there is going to be much more localized effort to help asset owners with security needs, but much less force coming down from the federal kind of top if that helps.

>> Yes, they do. And so for Uh the question was do states also regulate water utilities? I'd say absolutely. So the privacy agency which is going to be the one who has the most strict regulations can either be the EPA. So in a place like Idaho, you know, to some extent it's probably the EPA. Uh but if you go to California, they very much want to have primacy over whatever goes on in their states. So u their state. So they they will have the cala and uh the division of water resources I believe and DDW division of drinking water who are all sort of engaged in different ways. Um what I would say like a simple answer to your question is is

in five years there will be one federal entity that is responsible for cyber for all utilities and then every single state will probably have their own perspective to some extent and that can include um you know do an assessment every year. um hopefully that that um type of a requirement which currently exists in the state of Minnesota I believe Indiana, New York um also will come there will be some funding that comes with that but that's TBD a lot of times that money comes from the feds and it's not going to you know that funding is not going to be available for a while >> so that we have a question here uh or did

>> oh Okay, >> thanks >> Josh. Did you have a question or a comment? >> Okay, so with all of this in mind, the cyber threat environment getting more complicated, the technology environment getting more and more digital, and engineers getting not very much closer to being able to exercise good risk management over the kinds of assets they were deploying. We knew that something had to change. So the department of energy brought a group of actually a group sorry go back a group of experts together and these were asset owners. They were from um electrical utilities and other utilities, universities, um federal organizations. We tried to get the largest cross-section that we could of critical infrastructure and we talked

to them about this thing called cyberinformed engineering and got their advice on what would a strategy look like that would allow us to begin bringing engineers through their university training or their technical education into being a part of the cyber discussion. And there is a link to that strategy here. But one of the things the strategy makes clear is at the end when we have this cyberinformed engineering thing and it's cooked into a university degree so that no engineer engineering in today's modern world can graduate without having some ability to do riskmanagement thinking about digital appliances doesn't mean they need to understand every bit or every bite that goes through it. It doesn't mean that

they have to be an expert on the protocol, but they do need to understand what could go wrong if an adversary had full control over it and how might they prevent that. That would look like engineers incorporating cyber security practices into their body of knowledge, including engineering minimum requirements and specifications for physical infrastructure systems that incorporate digital controls. So if you're working in infrastructure and you've got digital controls, the engineers have to come in and determine is this going to work effectively and what do we need to add more risk management and maybe even physical controls to okay >> sorry >> I'm just not water is new to me as a critical infrastructure. So, you might

have mentioned this in the beginning, but how do people's like wellwaters uh kind of factor into some of those things in terms of like who supports those? Like we're talking about the rural areas that have like their own things, but obviously there's probably a bunch of people that have their own wells and so like sitting with groundwater, well water, whatever that is, like is there a support system for those groups as well? And I'm sure that we're going to start seeing IoT into weird spaces as well like that. >> Oh yes. >> Yeah. So I actually I live in a fairly rural area. I have my own well and my own septic system. And so from that

perspective um both of those are governed by either the state or the county respectively. And um unfortunately um well for better or for worse right my well is only dependent upon power. It has a very very simple um controller. It's just like if the pressure goes down, the the pump turns on and once the pressure gets to a certain point, the pump turns off, right? There's no electronics beyond that um simple mechanism. Now, from a septic perspective, very much regulated by the county. They do inspections, all of that sort of thing. Septic systems are a large source of contaminants to the groundwater um aquifers and it's and in some places around the country they have

programs that are called uh septic to sewer sort of the you know the alliteration that they use and that those are programs just to like close down those septics install wastewater lines so everything goes to that plant and becomes a point discharge. But when we think about modernization in especially residential water systems, I work mostly with energy systems and there is a profit motive and an environmental motive for many people to install digital energy systems in their actual residence. For anyone who has a home water system, typically those are electromechanical, as Andrew pointed out, and there just isn't that profit motive unless you are the person who really wants to know everything that's in your groundwater at every time of the

day to put a lot of automation and technology in it. So, we're not seeing cyber creep into the residential water systems to the degree right now that we predict in the future or the the degree it has in energy, but we're certainly starting to see it in municipal water systems. Every organization is looking for automation to tell them how to do things better, faster, cheaper, more. Um, every organization is looking and and you even got a question this week about use of AI in water systems and especially at a distribution level. Everybody wants some autonomous thing to make it faster and easier for them to do the work they have to do.

Okay. How CIE works is we first leverage an engineer, not a cyber engineer, not a cyber safety engineer, but an engineer who has a context and a body of knowledge around a physical or process discipline. So whether it's space or whether it's agriculture or and software engineers um you really do matter but not for this part of the conversation specifically. We're looking for that that physical context area and discipline. We provide a framework that that engineer can use to engineer out the impact of a cyber attack. And if you don't believe that's possible, come to the training this afternoon and I will show you how we do that. Um, and hold me accountable. If I don't deliver on that

promise, let me know. We're focused on engineers and operators, not to make them a secondary or shadow IT or cyber staff, but to use the context they have to harness our cyber security protections to the most important systems and the most important functions of our organization. Because often our cyber teams are trying very hard to do a checklist of requirements across the whole system. They don't have time and they don't have the understanding to understand where to apply the most cyber security or where if a a requirement is implemented with laxity where it has less impact or more. And we're leveraging things that engineers already understand that are not usually part of our cyber discussion

that include functionality, safety, and reliability. Most engineers are conversant in these three things. It's how they make engineering decisions and it's how we help them make cyber decisions. So in this example, and Josh alluded to this earlier, if an adversary is breaking into a critical infrastructure asset, first they attack the digital domain. And the way that we can limit the probability of that attack being successful is to leverage what we think of as traditional cyber security. Absolutely. Traffic controls, routing controls, segmentation, all of the things that cyber people do every day. However, ultimately if those asset or if those adversaries are looking to attack the functions of that critical infrastructure, they are ultimately going to have to levy some sort of

effect on the physical infrastructure that runs it. Either to diminish the amount of control, to diminish the the visibility of that asset, to take control away from the operator and do something else. And we certainly saw that in the Ukraine attacks of 2015. Um I think everybody now has seen the YouTube video of the mouse moving and the operators looking you know distractedly not knowing how to stop that. Um those are exactly the things that an adversary would want to do to the physical equipment. And the way that we stop them is putting engineering in place. Engineering in place that limits the impact that the payload can have. Certainly, we don't want the adversary going loose in our networks and

wandering around until they find something. But if we have limited the worst that can happen, that traffic through the network is less concerning. And there's time for the IT and the cyber teams to figure that out and to fix it while we're ensuring that our operational technology is reliable and functioning as expected. Okay. Cyberinformed engineering has 12 principles. And uh if you're thinking what I've been told already, I've been told too many. So I'm going to focus on just the two most important and we'll talk about what the rest do. If you come to the training this afternoon, you will get a deep dive in each of these 12 principles and a chance to actually play

with it and work with it in a real use case. So I always like it better when I'm applying something rather than having someone just bloate with me. The most important principle of cyberinformed engineering is consequence focused design. We are looking at our system hopefully from the design phase forward and trying to design in those engineering controls that limit the potential that digital technology could do harm to our process. And if you're thinking, "Wait a minute, that might protect you not only from an adversary, but also from a user who did the wrong thing or a burp in the system that just sent the wrong packet." You bet. It will protect you from both. It

has the potential to protect you from AI that made a decision that would have a negative impact on the system. And these are all great side benefits, but we're talking about this for cyber security. And this consequence focused design is the question that the engineer understands. How do I understand what critical functions my system must ensure and the undesired consequences it must prevent? I taught a cyberinformed engineering class to a group of cyber professionals and they were cyber professionals in the power industry. So I asked them this question about an advanced distribution management system. What is the worst thing that could happen if cyber adversaries had total control? And they looked at each other. And I'm really

sorry to tell you, they didn't know. They didn't know. And then when we pulled someone who was an engineer, it was like, well, okay, you could kill somebody if you violated lockout tagout, and these are all industry words in the right way. um if they didn't have their physical lockout tagout mechanism, you could reach through this digital system and cause someone to get hurt. Um you could also do a number of other things, but it was critical for my understanding that the cyber people installing the system, protecting this system had no idea what an adversary could do to that in terms of the process that their organization was running or safety, reliability and performance. So that's

why this is our first question. Our second question also appropriate for an engineer is how do I implement controls to reduce the avenues for attack or the damage that could result. Understand this is not about perimeter defense. I hope the cyber team and the IT team can help us with perimeter defense. The engineers job is to look at the process. What is being produced? what are the very important aspects of production and ensure that we've got the right protections that even if the digital layer goes completely haywire or is controlled by an adversary, we can manage the negative outputs that occur. The rest of these principles all help those engineers talk to the cyber teams

and think broadly about digital effects. So in reality there are two big principles and 10 small ones. So if you want to hit the other side of it um although one of the most important principles that is maybe it is a big principle is cyber security culture. What we are trying to do in cyberinformed engineering is to ensure that our culture for cyber security extends all the way through the organization from the engineers from the operators from the cyber security professionals so that we all have a similar understanding of the consequences that we're trying to prevent and the systems that we are trying to protect. That unified culture ensures that we're getting investments in the right place and making the right

decisions.

I kept looking backwards at one of the doctors in the room. Um, how many people have ever said, "How do I make the business care about cyber security?" Right? Pretty much everybody here. What if you flipped the script and said, "What does the business care about?" And then, "How do we help secure that?" I think it's not just the culture point you made. It's that you're meeting them on their turf in their love language at their level and you have handed on a silver platter how you can actually drive value instead of pushing a rope. But the second and we did this with the medical field. We said uh hypocratic oath for connected medical devices.

>> Caregivers already care about their profession. We're just trying to support enable that. The last point I'll make and I almost asked earlier is on that chart where cyber security was rising. I think there's two hazards in that. It looks good at the 50,000 foot level, but how much of that is a a NIST cyber security framework without application and prioritization or a CISA checklist of controls or an EPA version of the CISA checklist of controls and how much of it is maybe on data privacy like credit card numbers or billing information instead of things that can go boom. So, the reason I love this is we're not giving them new things to care about. we're taking the things they

already care about and making it safer. >> So, I agree with that. Um, one of the things that we often find, um, when we talk to asset owners who are interested in this idea is that when we look at the things that they've done to their systems from a safety perspective, they can already take credit for a lot of cyberinformed engineering. They've already made good decisions that control what bad things can happen in the system. And so what we're asking them to do is okay now think just a little bit more about how the digital technology works and let's build on that design work that you've already done. Um Andrew participated in or developed a paper

that helps to link cyberinformed engineering with your enterprise risk management. So if you work in an organization that has a mature riskmanagement strategy, this helps you lead that conversation with the board or with your executives about how this practice will influence the bottom line and the things the company cares about. And so that's a resource that's available to you. And I'll note that I believe our training starts in eight minutes. >> So should we do an interim Q&A right now for Ginger? All right. >> Well,

I was going to ask, is your training filled up already or is there any open slots? >> So, this afternoon's training has some gaps in it. Uh, tomorrow morning's was a little more full. When they talked to us about it, they said there were 22 folks there. So they will if you have not used Eventbrite and paid your $663 they're happy to help you process that even if you're standing in line. So even with eight minutes there's time to get in there. Um and I have Professor Kitty warming the room up with the slides so he'll get us started and then we'll we'll come over after that. >> Um great presentation. I have a question. Um it seems like one of the

most uh elementary or first steps into to understanding this kind of cyberinformed engineering consequences to ask the engineers and other folks building these systems to uh report to you what they think the worst situations could be and I would agree that's a very important first step but it is also the case uh possibly that they don't know and a kind of classic case was you know when we first started hacking cars some researchers um fuzzed a a car and unlocked a mode where the brakes um the brake fluid could get drained. Right? So they the engineers had anticipated an adversary in the system had anticipated making sure that they didn't have any connection to the brakes but did not

account for this one edge case where if you entered that mode and then started your attack, you would essentially disconnect the brakes. Right? So in what sense do we have the ability to trust humans to report what they think the consequences could be versus a more rigorous approach that would account for maybe all possibilities in the system because but if you ask a bunch of these folks they don't know and they even if they are very well verssed in the system probably would not be able to report all the actual consequences. >> That is a really good point and can bust for the win. Uh love cars. Um, so the best of these conversations are the

engineer with someone who has a cyber bit who can talk about, hey, you think that that's what's going to happen. How do you know that that's what's going to happen and what information is passed by this system and what guarantees that information didn't get in subverted before it got to the point where this next system acted on it? And you're right, a vanilla engineer may not have the fullness of imagination to think that through. That is one of the risks. And it I don't want to make this like cattyshack where the engineer just goes no no no no no no no because that's not exactly how this works. But getting the engineer first to not trust the

technology to start asking the question about wait a minute what is the data that comes to my system and makes this next process initiate? What where did that data come from? How have I verified that data? How do I know that it hasn't been subverted by an adversary? What do I have that would alert me? These are all the questions that our framework helps them provide. But most engineers are much better at the framework when they do it with a cyber person at their shoulder. And I will add to that and say that um the level of creativity that engineers and operators are willing to apply to you know thinking about the worst case scenarios now is uh much greater than it

was 10 years ago. And part of it is the requirement to do these risk and resilience assessments every five years. So they're having now regular conversations about what's the bad day. And so um for example um we probably did about 80 to 100 cyber risk and resilience assessments in 2020 and so far we've done about maybe 25 for this new round and um you know we would go in and we would talk to people about you manipulating the control system to cause physical damage and they'd be like no that can't be done. And uh as a really good example of sort of the change in perspective about three or four months ago we were having another similar

conversation with a large water utility. They um get water from a you know a goodsized river and they have some really important mains running under the river. And so we reviewed all the data and we said hey we think we can damage the mains under the river and that's going to be a really bad day. And they just looked at us and said yeah we think so too. And just the fact that uh they didn't push back, that they were already there is such a huge conceptual leap for some people um in our sector. >> And I will add it would be ideal if asset owners buying and investing in critical equipment were allowed to look

inside it and actually had the right to inspect what they buy and to hire people to look at it and to know what it was. Um, that would be ideal because that's how that engineer and that cyber person could start really looking at what could possibly happen. In most commitment letters, whether it's a license or a purchase document, um, when you buy this critical equipment, you sign away the right to inspect. um because that you will lose either the support from your vendor or you will face legal liability if you take the thing apart and attempt to understand how it works other than functional testing which is usually allowed but often even that is controlled and that is one thing

standing in the way of getting really good engineering understanding of these systems as well as good cyber understanding >> and before we move on to the next question so um are there any professional engineers in the What kind of engineer are you? >> Electrical. Wonderful. So I'm a I'm also a licensed >> uh engineer and um if we look at the engineers code of ethics, it starts with you have to hold paramount the health, safety and welfare of the public. And so if we are doing that right, we're willing to have some of those uh bad day type conversations. And it's been an evolution right? >> What's number two? >> Oh, Ginger. Number two is that you may

not practice engineering on any system you don't understand. >> Raise your hand if you feel real comfortable with that requirement right now. Me either. >> All right, question. >> So maybe a good segue to that last point, but what's the Hollywood movie that you guys refer to and said this is actually the best one for the water industry? Like if we have to go like advocate to the public, if we have to go create a more un uh uh more in-depth understanding of what actually is occurring, you know, there's all kinds of, you know, kind of crazy stories that we see in the in the Hollywood and everything else, but what's the one that you guys kind of refer back to or like

in terms of how it presents attacks to water? >> So that the NCIS episode is the one that really stands out to us. Um, I think Bryson earlier brought up the August Cole and PW Singer book. Um, but they uh they have another book called Burn in which is a I think actually a better book than Ghost Fleet. Super easy to read. I recommend everybody read it. It'll take you a couple hours. Um but what it does is it articulates um kind of a multiaceted attack on automation across some sectors where it's like you just sort of give a system a nudge get another system to react and then there's this cascading failure and um it's not

AI induced in the book but it's you know very plausibly could be done by AI and when I talk to utilities that's generally what I say hey go read this it's water specific and you know it's just a fun burn in. >> Yes. >> Yes, absolutely. >> Okay. Should we go into practicing CIE? >> I think we should. All right. So, let's talk a little bit about actually practicing CIE in the water wastewater sector. So, as Ginger mentioned, I do get to help her and the CIE team at INL develop resources. Um, one of the other things I do get to do is actually go with out and work with water wastewater utilities and do cyber informed

engineering. So, you saw the 12 principles, right? And those are really well made. One of the challenges though is that we kind of have to um we have to boil this down to make the messaging just a little bit easier, right? So, um, three assumptions that we always start with is the systems are vulnerable, right? the the digital systems are our adversaries are well resourced with time, money, and expertise. And if we are targeted, we cannot stop them, right? They will get access. And this always this used to put people in a really bad spot mentally, but now they just kind of like, yeah, yeah, okay, we get it, right? And I I do want to just take make a note here is

that I have u maybe what would be an optimistic borderline naive view of our sector because I think of the utilities that I work with tend to be very forward thinking and so there is a certain amount of bias that I do have but it's also a really good set of examples on how any utility can move forward. All right. So in our water in our sector you know similar to every other sector or um industry there's a way that we do engineering. So we have different milestones we have uh the prelim preliminary 30% 60% and 90%. And just to talk a little bit about what those that means. So at the preliminary stage you

usually have a a PDR or preliminary design report. You might have some high level schematics. Generally, you know, the the project has been budgeted. Management has said, "Engineers go forth." And the engineers say, "Okay, we're going to have a reservoir. It's going to be about this big. It's going to be located approximately here. There's going to be pipe going that way." Wonderful. At 30%, now you're starting to actually get process flow diagrams. You're starting to get uh specifications at least at the table of contents level. Now, 60% is where the rubber really hits the road from a CIE perspective. Now, if you've done your job really well and had the opportunity to do it at the

preliminary stage, you can say, "Hey, we're going to do CIE in this project." When you get to 60% now, we're actually starting to have electrical, mechanical, and very high level control system drawings, which is where a lot of the work is done. So, we're going to have drawings. You're going to have partial specifications. Uh, has anybody here ever like created engineering drawings or written specifications? Thank you. Yes. All right. So, it's a it's a lot of work that goes into all of these and it's a you know there's a very rigorous system to do these. You're also starting to get estimates and schedules. 90% you continue to build this out and eventually you get to 100% and you know

in some places we call this issued for construction. Now Ginger and I and a good chunk of the CIE team have been working on a project around CIE adoption. And one of the people that we interviewed for this um you know I was talking about how you know once you get to 90% you can really do good review and you can have lots of comments but his point was by the time you get to 90% most of the money is spent. So if you want to make changes you got to have change orders and this has been a little bit of a challenge for some of our clients. So um one of the things that we have

done is we have gone and we've essentially redteamed designs from a CIE perspective and what we've done and is to create a a role of um you know really a commander type role. Now if you were in the military commander intent is going to be a very familiar term but the team that I work on none of us were in the military. Um so this was something kind of a new idea. Um, we actually got the opportunity to review a 60% to 90% um, water uh, treatment plant for a pretty goodsized utility in California. And, you know, when we started, we were just kind of engineers doing engineering review things. Like, it kind of turned

into a peer review. And it was a little unsatisfying if I'm going to be honest. So, what we did is we kind of got to 90% and we were like, you know, this just isn't working. And I said, "Okay, everybody, I am now the mission commander and you are the attack team and you are going to, you know, looking at the process flow diagram, you're going to attack here and you're going to attack here and you're going to attack here." And they said, "Okay." Right? And then all of a sudden, we had this organizational structure emerge. And it became a really powerful way for us to communicate to the utility and the utility to communicate internally. And

since then, uh, this utility has actually included cyber informed engineering requirements in all of their engineering RFPs that they've released, which is pretty amazing. So, in order to really do this review that I'm talking about, we have to have a certain type of skill set. So, um, you know, Josh didn't explicitly, uh, say that I could talk about my kids, but I'm going to because about a year and a half ago, I got a call from the principal at Bay View Elementary School, and he said, "Uh, hi, Mr. Ort. Um, your son Anders was uh, caught hacking into his um, fellow students uh, reading accounts." And I said, "Really? Did he social engineer his way in?" And then the

principal goes, "What do you mean?" And I was like, "Oh, wait. I have to be the concerned father this time." Because I was actually in the process of doing a social engineering project for a client. And it turned out that he had just sort of gotten bored and he had just figured out sort of the this very simple algorithm for passwords. And you know, he finished his work and then he went to finish their work and by virtue of doing it, he was able to like go and buy little widgets in the program, right? That's all he wanted. But this was a you know, a pretty bad day for for him and I was really proud. So um you have to have

that type of a mindset and that type of a skill set ready. You know that's uh the gentleman who asked about like you know how do you get to that worst day is it's it's like that level of creativity. And so you got to have the engineers right you you probably have that person in your life who's a little bit like my son who's willing to like just try things and mess stuff up and and go for it. Um the operators right dayto-day they know what's going on in the system. they in the water sector are also licensed and we have run into challenges where we do uh what we call day without SCADA exercises and I'll get into that a

little bit later but the operators actually push back on that sometimes because their lensure is on the line it cyber security it's a wonderful time to really build some relationships there uh communications and emergency management I will say emergency managers love this stuff right their whole lives are based around thinking about the worst day, right? They eat it up. Um, targeting expertise. OSENT is extremely valuable. One of the things that I've gotten in the habit of doing is I'll go to an LLM and I'll just type, hey, what kind of PLC does utility X use? Right? And sometimes it tells me, sometimes it doesn't. Um, but I always go and I always report that. I actually at S4 um

gave a similar presentation to this and I I you know told told people a story about it and this gentleman from East Coast came up and he said you know I went to Perplexity and I looked up what kind of PLC I used and it was right and I had to tell my IT manager to get on that right and that was a pretty bad day for him but of course and then you've got the people who know how to break stuff and really enjoy it because they also inject a lot of energy engineers We can be a little stodgy at times, especially electrical engineers. I don't know where he went, but I do like to

pick on them. All right. So, who's really ready to go and adopt CIE? Now, I will say, and Ginger alluded to this earlier, is that every organization by virtue of doing engineering well, by doing operations well, is already doing CIE to some extent. And that's one of the ways that um you know CIE is easier to talk about with a broader audience than just cyber security because if we go in and we do a cyber security assessment and we're looking at this large group and we're saying oh your networks aren't segmented. They're kind of like I mean I conceptually get it but I don't really know what that means. But if we go in and we say hey are your

operators able to operate without the control system? They know what that means right and they always have an answer. That could have been last Tuesday, right? Because the SCADA system just doesn't work very well or it could have been 10 years ago or 20 years ago. So, um, you know, it's easy to build on those sorts of things when we have a certain level of organizational awareness, right? Leadership is very concerned. OT is out growing, it's protective cocoon, which is something that we see a lot more of. It's really important. and when engineering is a little bit more resilience focused in general but also open to the cyber conversation because they've recognized to some extent that it is partially at

least their problem and then of course it and OT they have to get along right this is really really important now this is probably the most um impactful term um when I do presentations that that I introduce engineers for as long as engineers have been around have been very accustomed to this concept of a failure mode. And if you like look at a water pump, you know, especially um some of those at older utilities, these pumps can be a hundred years old very easily. I mean, they're just like huge chunks of steel. I mean, really impressive. And they've lasted that long because they were engineered to not catch on fire and not blow up and, you know, not do all those sorts of

things. And they've been really wellmaintained. So engineers understand failure modes and we understand uh you know how to prevent them in in many cases. What we haven't often done though is say hey there's somebody else on the other end of that wire that wants to intentionally make this system fail. So when you couple that idea of cyber enabled and failure mode, you start to really get engineers to perk up, right? Then they start to recognize some possibilities that perhaps they didn't before. >> Can I add one more idea? Yeah. So we're using the word failure here, but we have failed to define it. >> Often we think failure mode means the system is off or rendered unusable. It

it just no longer is available. For an engineer, a failure mode is a disruption in safety, reliability or functionality. If I mess with one of those three characteristics, that is a failure mode. So understand that a cyber enabled failure mode may make the system work faster, work better in some aspect so that it hastens on its way to burning out a pump or causing some other downstream aspect. So it's very important for us to understand what failure mode really means from an engineering context and that it's not just I made the pump go boom. It may be I made the pump run a lot faster and vibrate a lot more and that caused the pipes to burst and now we have a real

problem. >> Yes. Thank you. >> All right. So, I talked a little bit about engineering in the water sector and one thing I didn't do is I didn't give um context on sort of the volume of content at each one of these stages. So, preliminary design report we're talking order of magnitude 50 pages maybe 100. 30% design review, we're talking maybe 200 pages or so. 60% design review, um, you know, probably about 300, 350. Now, we really get into specs and we're probably talking over a thousand pretty easily. And then once you get to commissioning, right, that's a whole other story. That's uh tons and tons of checklists and that sort of thing. Now, Ginger, I don't think I've actually told

you this yet, but um we've been working on a CIE project for a water wastewater utility who is in the process of install uh you designing and soon to be commissioning a new treat um pump station for the water system. And what they have realized is that their engineer was um sort of in uh unintentionally delegating all of the responsibility for cyber security to the integrator which was you know another contractor and the the asset owner didn't really have any visibility on what the integrator was doing what kind of capabilities they were doing. So what the the asset owner actually asked us to do was to write into the engineering specifications language around how they

can operate this pump station without the control system and then also how will we actually protect this control system from misuse and this is a pretty big sea change I mean it's not a lot of text right this is less than a page of changes in a thousandpage document but very very impactful and from a commissioning perspective now that utility has to go and test the system without visibility of you know the normal um uh you know HMI or human machine interface views. Uh they have to test it with a lack of communications. They have to test it without you know access to the servers and then they have to test it without the PLC's and we're

going to see how that goes. That's usually where things get a little dicey for these utilities. So very exciting. All right. Do you want to start this one? >> I would love to start this one. >> Go for it. >> So, there's a classroom. Oh, go ahead. Ask your question. >> Totally. Okay.

So I work in rail, light rail specifically and one of the biggest challenges that we have is there are so many vendor managed environments to where like you mentioned before and some of those things like that ability to actually see and understand the technology that you're purchasing is kind of like a real problem in some of those areas. And so in that same way of you know let's say you have a PLC that goes down or some other type of device do is that is there a correlary correlary moving on uh in terms of like that there's that vendor management component or is the water system basically cut off and there isn't as much connected there or you have to

fight vendors to be able to manage and and operate that technology. >> I think everybody has that same issue. >> Okay. Um I will say you mentioned rail. So there was some very salacious vulnerability information published I think two weeks ago about a head of train end of train protocol uh that's used for connecting trains and moving them around. And the idea was that someone with a communications device wireless radio I'm sure none of you have any of that. Um could broadcast that protocol and potentially man-in-the-middle that particular protocol. And not only was it vulnerable, it was known to be vulnerable and had been reported ages and ages ago. So that was when the cyber

people reported that vulnerability, it was reported as if the physical part of the train didn't exist and it was the vulnerability of this protocol that was the key thing to look at. As we started having conversations with train experts around this protocol, there are other things that are on the train, physical protections, engineering protections that protect that train even if that digital layer is subverted. And that was one of the examples that we cheered about where we're able to say, "Hey, yeah, there is this critical vulnerability and it does exist in the digital layer, but because the engineering and operational layer was somewhat aware of it and was concerned about it for other reasons, that's a

very highintensity safety operation for any train yard. There are engineering controls that ensure that that the impact of that vulnerability is not going to be met." So that was at least a win on the train side that we had. So in the classroom that's doing the training, they're going to do a deep dive with cyberinformed engineering and we wanted to give you at least a taste of what they're going to do. So I'm going to have you leave your ordinary work, whatever it is, and come join me at the municipal water station. I've hired you all. Thank you so much for saying yes. Um your paycheck is coming. It will have a lot of zeros in it. A really a lot of

zeros. um your job. >> Yeah, no other numbers, but a lot of zeros. And you can tell your parents, I got a job at Bides and I'm making a lot of zeros. Um so we have these water booster pump stations. They are dotted out across the city. Um as Andrew said, they keep the pressure in the system at the right level for the environmental conditions and our systems. and they ensure that if a subscriber purbles a tap or operates the system, they get the water they expect in the right amount. They also discourage because of the pressure in the system, the growth of biologic agents that would make our water undesirable and potentially poisonous. So, we got a grant from the

state and we're so excited. We right now we maintain these booster pump stations by driving to them. We have two engineers in the truck and they go from pump station to pump station and they reset the pumps to be right for that day or that week and then the next week they're doing it again. The good news is they do a lot of physical work in that booster pump station when they're there. The bad news is that's really expensive for us and we'd like our engineers planning other growths in the water system that we'd like to do and focusing on making the system overall better. So, we got a grant from the city. We are

going to install cloud-based monitoring and control on our water booster pump stations. Who here is excited and wants to be part of this project? Okay, I got a couple of people tenatively like maybe maybe just with all the zeros I'm not going to write that on my resume and it's totally okay. So, when we look at this cloud-based monitoring and control, our focus is on the pumps. There's a lot of other systems in the water booster pump station, but we wanted to just put control on the pumps right now and we'll grow this if it works. So, with the pump applying cyberinformed engineering, what's the worst that an adversary could do to one of those pumps through this

cloud-based monitoring and control system if they had the magic keys to the kingdom? What's something they could do? Come on, anybody in the audience? shut off the pumps. >> Okay, that's great answer. And what would happen if someone shut off the pumps? >> No. The water pressure in this area would would fail and then we would have to kind of raise pressure in other parts of the system and we would have to send our engineers in a truck and they would have to get the system going again. And they think it would take about two hours to restore somebody who cut the pumps off. Okay. What's something else someone could do? Yes, >> they can over pressure the system.

>> Okay, they can turn the pumps on and make the pump do more than it's supposed to do. So, over pressure the system. Yes, they can do that. The good news is we have some protections in the system that allow us actually to operate safely in an over pressure zone for a while. We have a bladder system that will contain some of that. And so again, our engineers think they can get out in a truck and repair that in the amount of time before damage is incurred, >> assuming the alarm is not also compromised. >> Fair enough. >> And that only happens to one station. >> Yes. And we do have some contractors that we can put in trucks. But yes, once

they if they do all of them, we are a little bit strained. Okay, I've got somebody in the back. Okay, the good news is this the suggestion was that someone could take over our chemical doping system and affect how much chemical we're sending in. We are not going to put that under cloud control right now. We want to try out this software first. And the other control we have in place that we did look at is we have a pretty narrow pipe that allows the chlorine and other chemicals to go into the system. So even if you turn the system up to 11, very little material would flow through the pipe at any one time. So we're we're not

automating it, but we think we'd be okay. Maybe. Maybe. >> Can I brick them? Well, if I could turn the pumps off or I could turn the pumps on. What else could I do? Okay. Yes, I can turn them on and off and on and off and on and off. And if I can do that, what happens when I turn an electrical system on, especially something that rotates? >> Say that again. >> It uses a lot of power. And when it uses a lot of power, does it stay at the same temperature? >> No. It heats up. That inrush of current causes an inrush of temperature. And if I caused that inrush of current to come

and to come and to come, would it feel like Las Vegas inside that pump? It would. It would not be able to release all of that temperature because it uses convection. And so eventually the heat in that pump would get to a point where I am degrading the insulation around that pump. And that is assuming that it is so well anchored and so well engineered that the actual on and off activity of the water flow isn't damaging it. But just from an inrush current, I can on and off and on and off and damage the pump. So that's a pretty bad consequence. We went back to the engineering team and they said that that would take if you damaged

the pump, especially if you damaged the housing around it, made it hard to just do a flat replacement, that could be as much as 18 months. Back in the back.

So creating a vacuum. The good news is this large pressure tank right here has a bladder system in it. And so it is designed to absorb some of the fast pressure changes that our system has. And we think we can ride out what an attacker could do using this particular pump given its power compared to the the tank. So we think we're okay in that. So, this on andoff thing, if I can focus you in on that, the engineers have said 18-month outage, potential meeting the mayor and having to say boil water order, potential EPA involvement. They're really concerned about that. That could be a really, really bad day for our engineers and operators. So, the

cyber team, of course, has lots of ideas for how they would like to change that. But one engineer raised their hand and said, "There's this thing called a time delay relay." And I know it's kind of silly. It looks like a kitchen timer. And when I have a time delay relay, I can only issue commands once per timing window. So let's say we set it for five minutes. I can turn the pump on and five minutes later will open again for a command and it will accept the off. And so my adversary who's trying to ram that pump ultimately gets on and then we get to some hysteresis and then off again. How much do you think they cost?

>> 20 bucks and we can put them on as part of a regular maintenance in the water booster pump station. So should I do some monitoring on the system through the sock to make sure that people are not issuing commands at the rapid rate? Yeah, I should do that. And potentially I should drop some commands if it's just looking like there's no way this is what should happen. I should probably talk to an engineer before I set that system up. Anybody with me on that? Yeah. Okay. But would a $20 time delay relay? That means that even if the adversary blows past my monitoring, my sock is asleep, they've all gone home for Christmas, the

adversary gets in there and says, "Okay, on and off and on and off and on and off, and my system goes on, off, and then within the twohour window, the engineers are there. They take it off the system. Um, and we did do engineering analysis. And even if they did this to all the systems, with a time delay relay, we can keep things going until they take everything offline and can run it manually. So, I will admit this is a kind of a happy movie story. We picked an example that works, but this is an example of how cyberinformed engineering works and how it can work with cyber security. And we wanted to at least give you the taste

of thinking through that. So, thank you for coming to the water booster pump station with me. You all get a raise double your salary. >> Twice as many zeros. >> Yes. >> I I Yeah. Um, when you go to owner's operators and you're working with them to incorporate cyberinformed engineering, does it cost a lot of money? Does it cost extra? And I asked this because, as you know from our secure by design work, right? The question is, how much is this going to cost and who's going to pay for it? And then of course our answer is always like well let's look at all the costs that everyone is is paying right not just uh the manufacturers but you

know what are the costs that the uh customers of software are bearing right uh both left and right of boom so I'm just curious as I am drawing parallels here the questions that you've received on this and how you're answering them >> so I will say we get that too we get this is gonna slow us down this is gonna cost money or it's going to be extra design work that we have to do or this is one of my favorites. We're past the design phase. We couldn't possibly implement this now. Okay. Um and what we find as we work with people is that typically if we can get the culture of the team working

well, it actually doesn't take them a lot longer to include this in the hazard analysis that they do or the risk planning that we do. Now, we are developing some tools that help people who have uh on the engineering side, we often think about hazard analysis, and it's typically they have tools that are used for safety. And so, we're trying to find some ways to add these consequence-based ideas into their ordinary hazard analysis that allow them while they're under the hood, while they're thinking about safety and reliability, they can just add this cyber security risk in. But we are getting that. What are you finding as you work with asset owners? >> So, everybody asks that question,

especially even internally at West Yos, we get that question because we have to competitively bid for projects. And so, we're saying, hey, if we have, if we're automatically going to do this and it costs more, then we're going to win less. Um, so that's a big concern across the, you know, the spectrum of project delivery. What we're finding though is, you know, applying the 8020 rule, like that first 80% is really pretty inexpensive. you know, it's a little bit of additional engineering review. Um, and u that provides a ton of protection, especially in the two principles that Ginger highlighted earlier around consequence focused design and engineered controls. Um, you do wind up needing to re-engineer some things, but

um I'm also a big advocate of, you know, using common sense, right? you you have to like we're not going to go and redesign systems out of you know just applying CIE unfortunately but uh there are lots of good things and you know we've got a little bit of time left today and we can get into some of those and some of the things we've heard from early CIE adopters but I I would say that the cost compared to the capital cost of implementing these is you know less than 1% I mean it's small >> uh just a add on that. Like when we talk about a the water hammer, the way to absorb a water hammer, there's a couple

ways to do so. You know, the pressure sensor artor concept of an analog pressure sensor that shuts down the pump like a circuit breaker would for electrical surge in the house. It's like 2,000 or 10,000 depending on who you're talking to for the pressure zone. But what we also say is if you have 10 pressure zones, you don't have to do this everywhere. Like what's the highest consequence failure for downtime? is the hospital one. So if we bring a level of ruthless prioritization andor downstream stakeholder analysis as to what's the highest cost of failure, it's not every leg of the pumps and the pressure zones. It's just the ones that the town decides it's worth a little bit

of insurance on. Then we can maybe prove it's a high value thing and then it's net new. every new green field power water might say that was a worthwhile investment. So we have to maybe pilot this narrowly and then see where we can put it in going forward. >> Yeah. And to that point so this was probably 8 10 years ago a utility had decided to kind of do something like this on their own without the terms of CIE or anything. And what they had realized is that some of their most important uh influent mains were in really poor condition. And so they started to get concerned about over pressure, water hammer, blowing these mains out, causing damage. And I I would

say that to replace these mains in total was was probably at the time a high eight to low nine figure type of investment. Um you get into all kinds of property rights issues when you try and do these things. I mean, it's really a quagmire. And so what they had done in that situation is exactly what Josh was talking about, right? you get these um pressure sensors and they shut the pump down if you know if anything occurs to drive that pressure up and you the important thing is to wire it correctly right you don't wire it into the PLC you wire it directly into the controller for the pump and um in right directly into

the motor excuse me and that provides that input and that way aworked adversary cannot access it >> I think there was a question over there yeah >> yes um So, I know when I've dealt with municipals, they've been small and they're run by not the town, but a board of commissioners. >> Yes. >> And so, a lot of them drive cost benefit, meaning we're going to lower the cost and it'll be to our benefit because we'll get elected again for another term. And they kick all the projects down the line regardless of the fact that doing it now given time value of money, etc., etc. Does it ever have any benefit to create a worstc case scenario in order to drive

CIE or or really I mean I guess any any change but in this case to really change the way they're they're looking at this. >> Do you mind if I talk about Idaho >> please? >> Okay. Um I've got to leave for the training so this will be my last part but before I left um I've got an alternative idea for you. So um sometimes we can create a worstc case scenario but let's face it how many of us have seen a whole lot of FUD and how much of us are becoming a little bit resistant to FUD. Uh fear, uncertainty and doubt does drive action but sometimes it drives action without reasoning being engaged. So we had a

wonderful experience in the state of Idaho. The state environmental organization had was doing a call for their yearly grants and loans for municipal water entry entities or anyone else who wanted to put an update in their water system. And they put 20 points of the scoring of this grant towards that utility having something with cyberinformed engineering as a part of their endeavor. and they're starting to define some rules about what they meant, what that meant. But each asset owner was encouraged, even if you don't do everything, if you try to employ cyberinformed engineering, we will then be more likely to accept your proposal. I loved what that meant from the state organization as kind of the owner and

the responder of worst resort to a municipal water problem. they were putting their money where their mouth was and they were encouraging asset owners to develop a mature strategy towards engineering cyber resilient solutions. Um it may not be the only way to do this but I am so excited about the way that's working and I hope that it pays out. >> Yeah, it's really exceptional. Um and Idaho Idaho Department of Environmental Quality will be putting out some guidance for water systems um on C applying CIE which is going to be a really amazing resource. Uh Ginger and I have seen an early draft of it and it's fantastic. Um, one of the other things that we've done to your point is, um,

when we do these exercises, we always invite the executives and we put them in the middle, right? They don't play, but they observe. And when they get the chance to observe and hear the back and forth and see the challenges, um, it really drives home the challenge that their staff are faced with. Um, you don't, you know, do the first exercise that way, right? you let the engineers and the operators kind of build up, you know, some knowledge and so that they can demonstrate to their leadership everything they knew and some know and some of the challenges that they have. But it's a really good way to do it >> Andrew. I'm going to duck out. Thank you

for bringing this home. >> Thank you. [Applause] >> And I'll be over there soon. >> I have to grab one. >> Yeah, the camera guy. Um, so I've never had the privilege of running security operations in a municipality or critical infrastructure. I'd like to, but I just do business, you know. So, u, but I have a lot of friends who are operators at, uh, at independent energy stations. >> And so, I hear from their point of view the operator way of thinking where if if I if I fail at generating power, I'm not going to worry about what the state's going to do or anything else. it's that I have a contract with someone else that

will find me a bajillion dollars if I don't generate this power and that's their underlying principle of pretty much everything. Um, but from a cyber perspective, they pretty much have no input. So, I'm wondering if you see the future of operators um having an interface with the security operations center to where they actually the operators start thinking cyber rather than the IT guys and the operators, two different worlds. Where do you where do you see the intersection? >> Yeah. So, one of the things that uh we've been working on is um a paper on how early adopters of CIE are picking it up and running with it. And so, we've talked a little bit about how you can

apply CIA to new infrastructure that's in design, right? Of course, that's where you want to do it. But let's just be honest, we've I don't know untold trillions of dollars of existing infrastructure that we need to modify in a cost-effective and meaningful way. And so what the early adopters told us is that if we're looking at the different CIE principles, there are a couple that you prioritize for existing infrastructure. One of those is going to be planned resilience, right? To some extent, this could be just really great emergency preparedness and it and that's certainly part of it, but it's also going to be, hey, under a focused cyber attack, what are we actually going to

do? And uh the other one here is active defense, right? How do you know what's going on in your system? What do you actually do? How do you consume threat intel and make changes? And what we think that those two principles actually give us and give operators like your friends is the opportunity to really kind of take CIE and run with it in their own way, building on all the good operations that they're already doing. Now in the water sector right it's relatively rare that there are contractual obligations to the extent to what you described but what we do have is state regulations right and we do care what the state says because they will authorize you know continued

operations and that sort of thing. So um you know if you have existing infrastructure existing systems and organizations right focusing on those and those can be some of the lower um cost type things that uh an entity can do to adopt CIE. So that's what we say and if you go to the um CIE implementation guide there's I think shoot it's like 1,200 questions right and and the the guide isn't meant to tell you how to go and do CIE. to get you thinking about how to do CIE. And if you think about it, so there's 12 um principles. I think there's seven engineering life cycle stages. So you divide 1,200 by 12 by seven and you wind

up with 14 questions on average per principle per life cycle stage. And so it's really a great, you know, way to start adopting CIE and picking it up. Um if you go to the training and you know, we'll get into that. Uh, it's way too much to talk about here. So, does that answer your question? >> Good.

>> Yeah. >> Okay. So, I do want to come back to this. We've got about 20 minutes left, but before I do this, are there any lingering questions or topics that we haven't touched on? Um, anything, you know, you were expecting that that we haven't provided yet? Josh, >> it's given away a little bit of tomorrow's talk, but um Emma and Manish are going to do AI. Data centers >> are both a threat and a fix >> and a threat and a fix. Um power generation requires water. Data centers require water. Um, so when we're looking at how fragile the interdependencies are right now between water, no water, no hospital, for example, most of our strategies we've been

discussing aren't factoring in that we're building a hell of a lot of AI data centers with really high capacity needs and we're not making more water. So since we may not get a chance to ask you that tomorrow morning in advance of their talk, do have you been thinking about how we might better ruggedize or fortify or think through the continuity of operations for water supply for AI data centers, etc., etc. Like is that stretching your brain too far or is that something you can anticipate and prime? It it is something that um just starting to think about in part. Um my friend and colleague Andy Bachmann is very focused on that topic and um he every day sends me stuff to

read and watch on it. I can't say that I've worked on it specifically outside of that example from about eight years ago where that utility did have a data center um run out of water and they had to take a lot of action to demonstrate to their customer that they that was not going to happen again. Right? So when I said earlier that it's rare that we have contractual obser uh obligations when it comes to data centers, I was totally wrong. I misspoke. Um there are significant contractual obligations in those situations. And you know, you might think also like why doesn't the data center just drill a well and suck water out of the ground, right? Well, I

was actually uh doing some work for the city of Carlsbad, California, right? One of just the most beautiful places on the planet. um they've tried to drill wells and I'll say they as you know private entities and and public entities in that area and there's just isn't groundwater. It's just solid rock. Uh I personally had to drill a well at my house. It was 350 ft deep just to get enough for a single household. I can't imagine how deep that and how wide that well would have to be in order to you know provide a data center with a reliable backup water source. So it's expensive. Also water quality can be a real issue. Please. And I almost wonder like we've

seen this before a little bit when Bitcoin was all the boom and everybody and their mother was like, "Oh my god, Bitcoin like let's string together 10 different video cards and then all of a sudden there were heating and cooling issues." Like the same type of thing from a draw. And so is there a coral I still can't say that word. I keep trying to to really see how that might in some way obviously the AI thing is going to be much bigger, but could we use like the Bitcoin mining thing as a anagram? Yeah, certainly a different word. I'm sure we could. Yes. >> Yeah. So, I think that this will become a much greater topic um just as you

know, source water stressors, you know, continue to pose problems for our industry. Josh, anything you want to add on that? Are we good? >> Yeah. Okay. Well, attend tomorrow.

I don't have a great answer for that. And and part of the reason is is that I think a lot of those um Bitcoin mining operations popped up in places where the energy rates were very very favorable to that sort of thing and I don't you know I do a lot of my work in California which was not um conducive to that. Um, not so much a question for you, just I've been listening to the feedback on and off. Um, and had some hallway feedback as well. Um, if you were here for the opening remarks with the the couple draft videos we did, there's a, you know, uh, Ginger brought up FUD, right? Fear, uncertainty, and doubt. And

one of the things we we said last year when we opened this track, but we maybe failed to remind people this year is this is a very tough set of topics because just because it's scary doesn't mean it isn't true. So there's there's equal and opposite sins that we could commit, right? Um I I feel like the more consequential something is, the more forthright we have to be, which means you never exaggerate it and you never coddle people or downplay it. And the problem we had, we we've leveraged FUD for so long in the vendor community that we've lost all our credibility. When you cry wolf all the time, when there are actually wolves at the

door, no one's going to believe us. So, this is part of our hazard for this particular project is several people this morning thought some of the videos were not scary enough and some of them thought they were way too scary. And we've had Congress people tell us, you need to make these scarier. So there probably isn't a single right answer for how scary they are other than uh we may have to make a lot of stakeholder specific videos. So the the water hammer video has been very effective with professional engineers who helped us understand that the hospital video has been very helpful with uh clinical staff. Neither one of them is very appropriate for our

neighbors. Right? So one of the open invitations it wasn't rhetorical. We actually want feedback on how to best strike a balance between truly scary things that also empower realistic action. So, we can talk about a really terrifying water hammer if the fix is $2,000 or $10,000, but we have to deliver it as a package deal. And even though this room is mostly technical talent and we like to break things and take them apart and put them back together, we're going to have to learn some storytelling and persuasive speech because if we just say, "Oh, it's fine. You'll be fine. Don't worry about it." Then we're not going to get the corrective action we need. If we sound too scary, we're going

to polarize people or paralyze people. So, uh, I'm not saying it's easy, but if you find yourself feeling that this is FUD, suffer, you know, tolerate that discomfort just for a day or two here and simmer with it because it might reveal a new fresh idea or angle. But I'd love your take maybe on how do we balance saying something scary without it sounding like FUD. >> Yeah. So, and to go back to that um water sector industry report that I showed, the utilities are already concerned about this and it it's what I found is that more people are looking for a solution than they are skeptical of the problem or they've shut down because the problem is too big. I've

gone into a lot of places where the engineers or former operators are really, you know, they're the executive leadership. So, when you start using terms around engineering and operations, they understand it, right? as Josh would say, it's their love language and we're just putting cyber on top of that. And so it's been a much more constructive set of conversations than it is if we talk about cyber security, right? And um I think that that's one of the reasons that CIE is really a big part of the solution to the know larger scale national security issues around cyber security. Please >> to piggyback on the idea about storytelling and the earlier point on how municipalities operate as someone

who isn't deep in the weeds on various uh critical infrastructure specific things because that's just not my day-to-day job. Um but I am a person who cares about the water that serves my house and I live in a municipality. To what extent do you see success or do you wish people would try this more to just show up at those meetings where the conversations are happening where the budget discussions are happening? >> That's a good question. So how much can a member of the public who's not a trusted entity go and really push for cyber security? I mean so most utilities either have a governing board if they're a special district or the city council is generally responsible.

So, I would actually just start by talking to your council person. And I would also, you know, most water wastewater utilities also understand that they're they service the public. They're taking the public's money and they're doing good with it. And so, they want to engage with you because they want to educate you. Uh I have a utility that um because of, you know, just making sure that reading standards are relatively low on any of their public facing information, they actually get a lot of phone calls from people are like, "How do you disinfect the water? like what is the treatment process? And sometimes they just say just come to the plant. Let's walk around. And they do

wonderful outreach like that. So, please engage with your utility. My guess is they're going to enjoy it. They love what they do day in and day out for the most part. And you know, if you're not getting what you want, call your council member. And it may actually be beneficial to them for you to call your council member.

We'll probably save this on uh Wednesday morning, but part of the reason for these 12 pilots as aggressively as we can is we're going to part of the output of that is we're going to try to give scripts to each of you that you can go ask at your town planning meeting, your city council meeting, your council person. So like you don't have to like write a bespoke one yourself. will give you proven questions that can drive the tenant outcome. >> All right, we have just a few minutes left here. So, I did want to just kind of make one one point. So, we did talk a little bit about Oldsmar earlier. Whatever happened, I've heard a

different story than what Bryson shared. Doesn't really matter. One of the things that happens in our sector is that um the public perception can be impacted very easily if there's a water issue in Florida and my parents are in Minnesota. So very specifically the hack in Oldmar was reported CNN picked it up right they ran with it. My parents my elderly parents in Minnesota called me and said we heard about the hack. Is the water safe to drink? And I said you live in Minnesota. the water systems are not connected. Yes, you're good. They're like, "Okay, thankfully." And, you know, frankly, the fact that they were like watching the news and asking those types of questions is really good. But, you

know, we we do struggle with perception in um in a land of plastic, you know, water bottles and all of those sorts of things that are relatively easy solutions. Municipal water is oftentimes higher quality and, you know, just better in general and certainly less expensive, like much less expensive. So, you know, there's um you know, that incident of course people are aware of. There's also Eloquipa, Pennsylvania, um Mules, Texas, um Abernathy, Texas. Are those do those names ring a bell? Yeah, maybe. Okay. So, when it comes to the incidents in Texas, there's a wonderful Wired article about it. I don't have the link, but if you just say, you know, wa