IATC - STUFF is on Fire - a Panel

really appreciate you joining us this afternoon what we're going to do here is I would argue going to be an extension of what was discussed this morning only we're gonna we're gonna widen the aperture a little bit we're not going to talk about health we spent a lot of a good time talking about health so we're going to talk about mostly not Health although health does does sneak in because we're talking about critical infrastructure and fundamentally our critical infrastructures there's many interdependencies in interaction amongst and between critical infrastructure which I believe you will see so stuff is on fire let us see who is with us today okay let's see uh let's see so we've got

Meg West who's a little bit off the screen and unfortunately Meg was not literally able to join us today but Meg sends a very special video message and she introduces herself um she's uh works as a incident response consultant for IBM's X-Force and she'll she'll talk a little bit about some of the things that she's done so look forward to that but I'm pleased to introduce uh Lisa young she currently serves as a metrics engineer at Netflix her Mantra is cyber risk management is a team sport team sport you heard it here uh Lisa has got a fabulous background including being an instructor please don't read that they can read it I'm going to read every word

no uh she's familiar she's been in a lot of different Industries including telecommunications Financial Services Airlines manufacturing Automotive and Marine

just the water Marines okay um and uh she she was a cyber security engineer uh for cert at Carnegie Mellon which has done a bunch of stuff in the space she was doing that for 11 years okay uh right next to her is Mr Dean Ford wave Dean uh Dean is a managing principal engineer at luminary automation cyber security and Engineering LLC uh Dean's entire 20 over 25 years has been involved with automation systems engineering and Consulting he sort of stumbled into this via a co-op opportunity at Anheuser-Busch in 1994. fell in love with Automation and the ability to see how things connect to each other um he's passionate about educating the public it's us and policy makers about

some of these issues regarding um the role of Automation in critical infrastructure he's also the chair of the American Water Works water utility technology and automation committee and I'm not going to use the acronym for that thank you Mr Bryson bort he is the uh before I got his bio I wrote a short sentence I wrote Bryson board is the CEO of Scythe an army veteran and an all-around good guy then then I got the longer version so yeah did I yeah then I met him good guy's been retracted yeah updated bio uh I did mention uh Scythe he also started uh Grim a cyber security consultancy the co-founder of the ICS Village which you should definitely

check out at Defcon in just a couple of days do check that out uh senior fellow with the Atlantic Council on Cyber uh statecraft initiative National Security Institute and an advisor to the Army cyber Institute and as an army officer an officer I tell you he was a battle captain in Brigade engineering officer in support of operation Iraqi Freedom before leaving the Army as Captain okay and we have also with us we're just tremendously honored Elizabeth or Liz leverages almost two DEC two decades of legal public policy and business experience to build and scale cyber security Focus companies she's currently the Vice President of Operations at Scythe prior experience includes serving as a senior assistant City attorney city

of Atlanta um and dealing with Aviation and she was also involved in incident response which we might hear about later hmm incident response um and I as your humble moderator my name is David Bots I'm the managing director of cyber infrastructure security at Edison Electric Institute um I have worked uh literally worked for an electric utility for about 20 years about a decade of Energy Regulatory Compliance physical cyber security working with lots of different federal agencies including Department of energy Department Homeland Security and others um yeah and I'll just mention one more thing I'd like to thank Mr Scott Alger from the I.T ISAC for his assistance in providing some background information on the food and egg sector risk so

here's how today is going to go we're gonna we're gonna we're gonna start it at a certain level we're going to start talking about examples of incident response we're going to then move into a discussion of risk and how to think about risk because risk is hard and generally as humans we don't ten we don't naturally do a super good job of thinking about our quantifying risk we're then going to go through a number of different critical infrastructure sectors and talk about how stuff is on fire okay so now here's Meg and I I put in a little tip here play the video now Meg's on a plane on her way here

okay [Music] we're not seeing it yet she's over there I mean I can see it great yeah oh let me do this

[Laughter] oh no all right so that would get anybody no computers wait where's Tom yeah computers are hard you know did you turn it back off and on again there we go all right um Meg who are you and what are you going to tell us [Music] hey there welcome to the b-sides Las Vegas I am the Calgary and stuff is on fire talk my name is Meg West and I currently work as a cyber security incident response consultant on IBM's X-Force team prior to that I worked as a global cyber security incident response manager at another Fortune 100 company need this to say I have quite a bit of experience working in cyber security

incident response and I'm here today to tell you about two real world incidents that I have personally observed of course given the nature of the conversation that I'll be talking about I can't tell you exactly who the clients were but what I can't tell you is that I've seen these incidents happen several times before so be careful because they could be coming to a network near you the first incident I want to talk about occurred in the cloud what actually happened in this case was the client had an admin account where MFA expired on it due to a licensing issue and that account actually happened to have an extremely basic password that password was also leaked onto GitHub if you're

not familiar with GitHub leaks they're happening all the time so be careful to keep your API keys and other sensitive passwords safe and left out of your GitHub code the attacker was able to Garner access into the admin account and once they got into the cloud they started up spinning up extremely powerful virtual machines so if you don't already see where this is going I'll give you a hint it has to do with crypto if you're not familiar with what's been happening in the cloud lately especially in the context of cyber security incidents a lot of attackers nowadays are garnering access to clouds and they're spinning up some really beefy virtual machines to host crypto mining operations the client at

the time probably had a spend of about a thousand dollars per month and after the attackers spun up these virtual machines their climb their spend climbed up to about fifty thousand dollars per month so you do the math that's quite an insane difference of forty nine thousand dollars per month especially when you're not expecting it to remediate this it was obviously quite simple changing out the past or taking down the virtual machines opening a ticket with the appropriate cloud provider and basically Becky Mercy saying hey I messed up can you help me out here there's some fraud occurring but this really could have been avoided how could we have avoided it the things I mentioned earlier being cautious what

comments you're making into your GitHub repositories making sure that your MFA is appropriately enabled keeping track of your licensing for accounts these are all really basic things but unfortunately they're very easily overlooked in the second incident I want to talk about this also occurred in the cloud however it wasn't about licensing and it wasn't about MFA nor was it about passwords it was actually about an NFP group being incorrectly configured if you're not familiar with the term NSG that stands for Network Security Group it's akin to an access control list on a firewall but it's more applicable in the cloud and unfortunately for this client they left their NFC wide open essentially allowing anyone to access

into their cloud and a researcher let's call it that was able to get access to the client's cloud and Garner a lot of sensitive data that researcher if we call it that reached out to the client said hey I access you know several hundred gigabytes of your sensitive data within the cloud and if you don't make XYZ remediation and containment steps by this time frame I'm going to go to the news and tell everyone about this vulnerability that you have how you're leaking all of this personal information so again this could have been really easily avoided it was actually found that the client had done Showdown scans earlier and known that the nfg group was

not properly configured however it can be assumed due to improper Staffing lack of resources and honestly just human error overlooking and correctly configured NSP that this incident was allowed to happen if we take a look at the oauth's top 10 reasons of incidents happening year over year we're seeing that misconfigurations are one of the largest reasons leading to cyber security incidents why do misconfigurations happen nine times out of ten is because of human error so those are two really interesting incidents that I've seen happen quite often in the cyber security field and they're also incidents that have we just taken some simple measures double checked our work not had employees that were burnt out overworked had someone

else to back them up and look over the work they were doing these incidents could have been easily avoided I hope you enjoyed this information I'm finding more and more incidents that are happening just because of simple human errors but the fact of the matter is the real life and real world impact is huge both of these incidents happen to Fortune top 500 clients so to be honest they could easily happen to you I hope you have a great rest of your conference and I'll see you next time bye Meg thank you thank you okay you ready for me hang on all right so uh I'm Lisa young I'm excited to be here and uh

correct I'm ready okay cool all right so so I have this talk that I give it's called all roads lead to risk so I just crib these notes from there so I didn't have to make a whole new presentation so every time you hear the word risk I want you to think uncertainty we use the term risk interchangeably with threat vulnerability control asset you name it so I'm here to challenge you all on your language but the most important thing is when people are concerned about something that doesn't make it a risk so I always say you should move on the Continuum from areas of concern to actually doing some analysis or something and figuring out if it what's

exactly uncertain about what what the situation or the condition or the threat or whatever is the area of concern so there's no part of an Enterprise regardless of mission regardless of type of organization that's not exposed to some kind of risk and the type or category of the risk might be different but the workflow to identify assess or analyze or take some action to respond to risk is largely the same and you know so for those of you who don't know if you weren't in the last presentation I serve on the board of directors for ISC squared so if there's any disgruntled people in the OR organization you can talk to me because we're making some

serious changes to the organization and I hope you see that um but in in that role we identified the top 10 skills that'll be needed in cyber over the next 10 years and obviously Cloud security data analysis malware analysis all those things are on there coding programming all of those but risk assessment risk management and the ability to talk across and work across silos right to actually translate the things that we do in in technical terms to business impact terms and I said this last time and I'll say it again but uh uh you know one of the reasons why the covid task force which was up here before if you missed it won an award is

not for all the good work we did which we did a lot of good work but it's because we made people who were in silos talk to one another right and I think that that's going to become really more important as we go go through this uh this everything that's going on in our world next slide please so uh when I I actually you know use the word digital I don't really use the word cyber much because I don't think it has a lot of meaning um so I actually have a master's in public policy and one of the um one of the exercises we had to do I went to the University of Maryland one

of the exercises we had to do was Define cyberspace cyber security and all those terms as related to public policy so I'm not really fond of the word cyber but it does get all the money so the but the problem is is that if you all know the OSI model it's all ones and zeros anyway I'm a telecom engineer so you know the whole point is that digital is pervasive and this whole notion of cyber physical what it's being called now cyber physical systems safety systems cyber physical so I like to think of the word digital and building digital trust right and so this just is a graphic I crib from the Masters program the University

of California at Irvine and they do a cyber physical systems but it's also referred to which you'll hear throughout this conversation I do a lot of work in critical infrastructure I just recently did an assessment for a water company in another country that was deploying a thousand iot sensors to to understand what was in their infrastructure right so you can imagine where this all goes so the point here is that you know these things are tightly coupled to all of the systems upon which we rely and you know use in our daily lives the other thing is I happen to be privileged enough to own a spa in my backyard and I fill it up every few months I mean you know you

drain it you fill it up it's not like a pool and I've noticed because you know the chlorine shortage right there's no more chlorine in my water um uh so when I fill up the pool I have to add additional chemicals now when I read their monthly water report it tells me that they've gone to bromine but we don't exactly know if it does the same thing as chlorine so I'm just telling you this because there's things that slip through the cracks that we don't always pay attention to um okay so next slide please so is there anything different about cyber risk right um you know cyber generally speaking in the context here is it's of or relating

to computers digital infrastructure but what um the difficulty is is that you know our risk which I use just the common dictionary definition of risk and when you put those two things together what it says is it typically involves unauthorized access right so it's not just the digital things the misconfiguration the things that we're concerned about but there's also this element of unauthorized hackers right unauthorized use of computer and computer technology but what I want to say here is that our risk increases cyber risk whether you call it cyber risk Digital Risk whatever you call it it increases as our dependence on these cyber physical systems actually accelerates uh next slide please so this is um I do a lot of content development

work for isaka so if you've seen the risk I.T framework the risk it practitioner's guide if you've seen the risk I.T starter kit or the scenarios I think we just released a few weeks ago so I'm a futurist and so I like to build scenarios for things that might happen in the future and I use and they're they're published now so they're freely available um uh eye socket just released a bunch of them but basically the constructs of how do you think about what could or couldn't happen because one scenario could have multiple outcomes in the future right and how do we start thinking about that as a way to get senior leaders and decision makers more

information or more option more optionality about what decisions they should be making and so this is just a high level risk management workflow people spend a lot of time on assessing controls assessing variety of things but what they seem to miss in my judgment is they miss the identification what are the things that are pertinent to us depending on the type of organization we are right because each of us has a different Mission but the scenarios can be used in every step of this workflow um to you know every step of the risk management process or workflow you can get a better understanding of what actions to take or what actions not to take based on you know your workflow

next slide please so I know this is a little complex side normally it's a build but we don't do builds here so the point here is that this is the organizational context to think about risk management so I'm going to start from the organizational Mission every single organization whether it's for-profit non-profit business government whatever it is has a reason for being okay I did a lot of work over many years ago helping the military you know set up cyber sort of thinking about cyber um as a as a discipline in a domain and when we think about the organizational Mission we use business activities whether you call them projects initiatives workflow business activities business processes productive activities

we use those in order to execute our mission but those uh business processes that we use are reliant upon assets right and you can think of assets as people Information Technology facilities Supply chains supplies raw materials you name it you need to Define what those are for you you and what the critical ones are to your business processes that allow you to deliver your mission and any of those assets people Information Technology facilities Supply supply chain we wrap our processes around them we call them risk management processes we may not call them that explicitly but all the things in this box over here are examples of things that we do to keep those assets operating in their intended

condition so that we can continue to carry out our initiatives whether those initiatives are automated or manual doesn't really matter the point is the more we rely on the Technologies which relies on the assets we need to wrap them in a way that allows them to continue to operate regardless of what happened whether they operate in a degraded state or anything so that we can keep going now the problem with many of these things in the Box over here on the right hand side is that they're all done individually they're discrete efforts they don't collaborate right and so one of the things that we can do and should do is to collaborate more across silos and

figure out how to collect data once use it many times right for multiple purposes so next slide please so this is just another way to say that you know risk identification and assessment uh they have to help the organization identify what is most important you know what conditions and call them by their proper language and then you know I think you mentioned quantification I'm not going to talk about it but you know I'm I'm not a probableist although I use probable Theory to figure out in my scenarios what could or could not materialize but also I'm a big impact person and um and you know it's important to understand and talk to Business Leaders in terms that they

understand about what the impact could be from a degraded state or an asset being taken out of service uh next time oh this is it oh well there you go okay so uh then never mind I'm all done that's all I wanted to say and we'll go on to the next person now so I'm here if you need to talk about any of this anymore after you're up next babe all right thank you thank you

oh I look pretty good there yeah you look great um you got to get close to the motor yeah so one thing um I just learned that I still had a toner account so don't send anything to that this is my this is my connection to the world now I I have given up on social media um so well risk is the first word risk identification no that one was mine okay I'm sorry there you go so stuff is on fire um so uh real quick uh what they uh what David asked us to do and and I tried to follow that and be a rule follower for once um so everybody's probably familiar with

a couple of the the most recent events that have really thrust the the water sector um and when I speak to water I mean Water and Wastewater to us it's the same thing to you it might be very different things but the plants are the you know if you're living Downstream of somebody else's City you're drinking their Wastewater sorry um so um so just so you know that so old smart everybody was very familiar with because it happened about the same time and the colonial stuff and all those things and and I'm not going to go into the details you know some old machine was sitting there and it was got left on and somebody else accessed it and you know

blah blah blah blah and they they got a hold of the the controls um the one thing about OT is that I'm sure most folks are aware you know just getting into access doesn't really get you anywhere you got to know what to do with that um so this particular person also knew what to do with that once they got access and they went over and started controlling some some chemical infeeds that were very bad um there's a whole host of problems with that that um that aren't cyber security related um and so I I struggle with this concept of you know everything's a cyber security problem to to an asset owner like water it's just one of thousands of

risks that we manage every day and unfortunately the public doesn't even understand what it costs to generate the water that they get and we are in a constant struggle you know you try and raise your rate by five cents and all of a sudden you know the world's on fire but most most water utilities are not um even covering their costs to produce the product that they're selling and another thing that most folks don't understand is most water utilities are the only Revenue generating um true Revenue generating uh entity within a city everything else is tax based water utilities Wastewater utilities are Revenue based they send you a bill you pay the bill so they're also the ones

that's that can get rated when the city's not doing very well um so anyway there's a lot of things working against the water utility in this case it was a simple issue you know they left the machine on from some startup a decade ago um so you know was this really a cyber security issue was it a people issue why did that machine get left there where is the process is in place to go around and ensure those things are taken care of um so we you know my my view of the world is a little bit different in that that that there were a lot of other things that failed and that's usually what happens when these things fail

right it's not just one event it's a Cascade of errors that occur in order to cause a problem fortunately this particular problem the operator happened to be paying attention and caught it in time um the other one the Baltimore city um ransomware you know it it didn't affect any water delivery at all um it was a billion issue billing systems issue which we're also going through some major upgrades um you know same kind of right that some person clicked on something they weren't supposed to launched a whole attack so a lot of issues out there that that are being faced you know utilities are in a unique spot where they have to be very transparent with their users with their

clients they're also a unique spot that that specifically in the water sector information is shared there's no competition so people share information all the time best practices the Water Works Association the water environment associations the Wastewater side ins and outs um you know there's a lot of information sharing and and um past what's on fire if you could go to the next slide there's some stuff that's smoldering out there um public awareness is is is a good thing and a bad thing so you know now we've had these incidents so now everybody wants the utility to take money that they don't have and go spend it to fix cyber issues so if the board's agendas don't have cyber on them

then you know uh turmoil occurs and things happen and people are upset well you know we also have to keep the plant running and we probably need to replace a generator that won't start to keep the plant running during a power outage which is a far more likely event from a risk standpoint than than a cyber incident so again it's a it's a management issue for us um and so public awareness also now has elevated a lot of the work that's going on here and why I'm excited to be a part of this is that it's helping us get some things pushed through which will then Force some you know when you're when you're in a situation it's it's one of

the few things that that we don't like one's a consent decree we don't like it but when it sure does open up the checkbooks and allows us to get some stuff done so my utility friends are gonna like that but it's true it's very true um along with that then comes from government regulations so um you know we we focus a lot on the quality of the drinking water um uh that's our primary primary objective is quality of your drinking water everybody feels clean today right you're welcome so um and again it's it's a lot like electricity you know you flip the switch on the lights come on you know there are there are an

increasingly number of fewer people that understand how that electricity gets to the house how that water gets out and and you know understanding how all this stuff works together so hopefully these camp we can get some campaigns going that's a big work of the awwa um there's some water as life campaigns water one uh one water um no water no beer um it has been um but essentially the you know the next thing that's coming is is that the Fert nerk ferc that is in the electric utility is coming to water you know water applauds that bring it but you can't apply it without also applying the funding we have a much bigger problem Beyond

cyber is that um you know the the Flint Michigan's out there you know there's a lot more of that than there are of of this cyber issue we've got a trillion dollar crisis and it's all buried nobody sees it so that was that report was 2016 so that's probably 2 trillion by now um most utilities you know pipe is only designed to be in the ground for 20 years 30 years 50 years most utilities aren't replacing pipe fast enough to ever catch up so we're still pulling pipe that's wooden out of Baltimore from the original piping it was in so just think about that as we start addressing this and then you've got the problems in California that

do I really care about cyber when I don't even have water and then when you start to think about it and say okay well I sell water now I'm regulating and telling people they have to use less water well but I have to spend more money to take care of that water and do cyber security but my Revenue just reduced because I'm requiring you to use less water costs the same whether you use a little bit or a lot that costs me delivering it to you is the same so there's a lot of those challenges that we're up against uh Workforce Development for us is um you know the the gray the gray hairs The

Brain Drain um what are the words there's a ton of words out there for it the problem with the utilities is it's as public sector jobs they're not able to keep Pace with the rapid changes in the market right so they'll they're a great training ground and people learn their their trade and they're gone offer double salary right now the good news is that with the Millennials and and the Next Generation there's a lot of social awareness and social conscience there we've got the game on that right we're making water good for society yeah um funding for risk mitigation we talked a little bit about that um the other big one for us right now is

is the fallacy of cyber insurance I I really wish um that that would be regulated out um and nobody would waste money on it um you know all it takes is you to file a claim their job is not to pay it you're expecting that they that they cut you covered um so it falls back a lot where you know the the boards and things are looking at this differently the bottom line is if if Europe had a known issue you know you're not covered that as you will find with most insurance um go talk to your insurance broker sometime and have a lovely conversation about that and we've got a huge issue with technology obsolescence we were talking

a little bit earlier um what's it you know that back in the day that when the river caught fire that initiated the Clean Water Act the Clean Water Act then caused us to go around and build a lot of concrete a lot of that concrete went along with that came a computer control system that computer control system is at the early age of Automation and nobody really understood what they did so they got a ton of automation had no idea what to do with it and so very few places actually did anything with it it was never in the water industry Automation and technology is not really a an investment it's a it's a tool and

it's a necessity Private Industry would never automate something that didn't have any return on investment the water industry has to automate things whether they have a return on investment or not because of the quality and the safety issues so we have to we have to change that thinking in the in the industry and change that culture I've had a conversation the other day with with um you know everything's going to the cloud Who the if if the people that are using the cloud data can't understand how to use a trend on my computer when it's local I haven't helped them any by moving it to the cloud I haven't gained and I've added in fact

I've added risk because now they might lose that the only source of data that they had so there's there's just a lot of that cultural change that that I see being a 20-year Private Industry guy making beer and drugs and cars and all kinds of other stuff and moving into the water sector a lot of challenges that that we've solved in gas and oil and electric haven't they're they're looked at as brand new problems in the water sector and so there's a lot of a lot of opportunity for sharing of information um so I'm gonna settle it up at that and uh thanks for having us out foreign

thanks very much Dean I appreciate that and when we talk about food uh and food and egg um I think you'll hear some common themes where people say this is a new problem we've never had this problem before we've no one has thought about this and that's what we're going to talk about we're going to talk about food and agricultural sector problems which are well frankly their Legion uh there's a lot of issues here so let me jump in according to the U.S government and sis specifically 2.1 million Farms over 900 000 restaurants over 200 000 registered manufacturing processing and storage facilities in these United States food is a big deal in the United States

okay show of hands who had some food today okay yeah so probably everybody or after you showered yes both and we should also ask who who had water today but um food is a big deal economically 20 approximately 20 of this nation's economy is structured around food growing it uh harvesting it manufacturing processing advertising delivery and the last mile at the store so 20 of the of the entire economies is a relatively big deal so what's going on are there any problems yes there are problems stuff in fact is on fire okay so let us talk let us first start with concentration risk now I would say on this picture we have two examples of concentration our first

example of concentration is a nice person who's who's thinking thinking very concentrated thoughts that's the good concentration there's a bad concentration as well in the bad concentration is uh the glass in this case a magnifying glass the Sun hits it starts a fire that is where we are at with vendors and vendors in the food space so just a couple of really interesting statistics um overall only 15 cents one five cents of every dollar that we spend in the supermarket goes to Farmers what 15 cents so that means 85 cents goes somewhere else uh processing uh processing storage transportation and marketing but consolidation runs really deep four firms or fewer controlled at least 50 percent of the market for if we take all

of the different elements of food Dairy meat what have you um uh 50 percent of uh four few four firms or fewer controlled over 50 percent of each each of those sub pieces of food for almost a third of shopping items the top firms controlled 75 of market share so there is very significant Market concentration well okay what's the worst thing that could happen oh there's actually a lot of worse things that could happen so in meat specifically right now in today's world there are four major companies that control depending on on the specific type of meat you want to talk about 55 to 85 percent of the market is controlled by these four firms

um the White House has accused these companies of contributing to inflation by Rising raising prices while generating record profits what's really interesting is that the U.S subsidiary of Brazil's JBS South America which is the world's largest meat supplier they Pro they they alone that one company is responsible for 20 of the meat production in these United States of America 20 one company no problem oh there is a problem the problem is that in late May of 2021 ransomware visited their firm so all processing of all meat in both United States and Australia stopped it just it just full-on stopped and a lot of a lot of Downstream uh impacts from that especially if you're a farmer who

can't deliver uh your livestock to the slaughterhouse there are no pictures of slaughterhouses here only relatively happy animals who are about to go upstate to New York um JVS however ended up paying 11 million dollars in ransomware to the Cyber criminals a week after discovering the incursion so other examples of concentration here's one infant formula according to data compiled by the Allied market research these four companies control 90 percent that's nine zero percent of the U.S infant formula industry you see the names but there's one name in particular that really gets my attention um and that is the good people from Abbott nutrition so uh there are some challenges here between December 1st 2021 and March 3rd

of this year 2022 the Food and Drug administ Administration received nine reports of infant deaths related to infant formula manufactured by Abbott nutrition in Sturgis Michigan so in February of this year February 22 2022 Abbott voluntarily recalls powder formulas that were manufactured at that one plant so that one plant just it it stopped um uh I'll have you just so you're aware uh Abbott alone is responsible for about 40 percent of the infant formula in these United States of America half of their production how much half of their production in the United States comes from one plant one plant so 20 so said another way 20 percent of all infant formula in these United States

comes from this location and there's some infant formula that 100 of it comes from this location because at this location it makes other formulations that are available nowhere else um so let's let's walk through what happens so in February uh the recall happened and Abbott stop production they said okay we're gonna do a better job we're gonna get after this bacteria that is causing problems no admission to guilt um so they did that four months they were offline producing nothing um you see there the picture of military assets being used to transport formula from Europe into the United States because parents were desperate desperate because infants could not there was no there's no formula so this

was this is a pretty terrible situation so early June Abbott resumes production cool um then what happens uh then the weather happens the weather happens and in June June 15th of this year a storm happens in beautiful Sturgis Michigan and Abbott gets water coming up through the floor and they got they got all kinds of problems in their Factory they stop production again and it takes them about three weeks to restore clean stuff out get it all going again but think about that think about four entities that have this control over infant formula which if you have an infant that cannot tolerate any other kind of food that is critical infrastructure to you no kidding I mean

this is the definition of a life health safety issue for short um what else is going on in agriculture oh we should talk about agriculture we talked a little bit about food let us get to agriculture so um uh as the the good people at the FBI did Issue a warning and they sent out uh they sent out April April 20th this year FBI warns of ransomware attacks targeting U.S AG sector um in the interest of time I'm not going to go through these but uh this came out this is a super public there's four specific instances of attacks against food producers grain storage elevators Etc this is like a no kidding warning no kidding warning what that happened on

April 20. nothing bad would ever happen after April 20th of this year oh but their bad things do happen oh how did this happen so um the nice people at fent um got ransomware on May 5th egg Co we can concentration concentration so AGCO that that's the name I'm not lying AGCO agriculture company uh they owned this big tractor company they own a lot of tractor companies they got ransomware um they've gone pretty quiet but it doesn't take it's not hard to understand the Practical implication of ransomware against a manufacturing facility that produces tractors can cause those tractors to not be available to the marketplace when the marketplace wants to do things like buy tractors fix

tractors use tractors so this is an attack against agriculture critical infrastructure and it's an issue but there are other issues related to tractors now we all laughed uh and we were super happy when we saw pictures uh picture on the left of tractors Towing uh disabled Russian tanks in Ukraine we said yay that's great um but then we got sad for a second when we learned that Russians were stealing looting pillaging Ukraine in an unprovoked war and were stealing tractors so he said boo but then we heard that John Deere remotely disabled the tractors and then we said yay what we said yay for a minute just a minute and then we thought about it and then we

said ooh yikes no I want to give a shout out he might be here in this room I don't know we did talk before this presentation and that is if there's one person that I would I would strongly recommend that um you follow you follow on the twitters and in in all the places is Mr sick codes um so big day big egg is Big Data and um I grabbed a lot of this information from him he's doing amazing work and I'm going to talk about uh his Defcon 30 presentation a little bit but um in this picture you might see a tractor but do you know what that tractor is that is a data collection

device it is collecting all of the data all of the time uh moisture uh uh it it collects everything about the production used in agriculture and um so you say okay well what's so bad about that yeah for all of the data from every John Deere tractor is being collected and stored in one place what could possibly go wrong oh right everything everything could go wrong because this data speaks to very precise operational impacts that are going on for a given farmer we talked earlier about I think it was 2.1 million farms in these United States of America if there's one company that say given their Market I I don't know how big the John Deere Market is compared to others

um my belief is that it's at least north of 40 percent probably higher than that but the good people at John Deere they have the data they have all of the data they got all the data now what happens if all that data is a sync is in a single portal would that represent a very nice Target for an adversary yes yes it would what could you do with that data you could you could do a bunch of things you could do things commercially like you could say tractor a does not belong to Bob it belongs to Robin and Robin's tractor belongs to Pat that'd be kind of a pain in the neck but you can issue a

multi-year denial of service attack against one farmer or more than one farmer maybe a bunch of farmers maybe through a single portal you reprogram the tractor there is remote access involved related to this portal you reprogram that tractor to dump so much fertilizer in a certain area of the field that that field cannot grow crops again for decades that would be a bad day but here's the thing this this is the world that we live in and again you want to follow Mr sick codes because and and uh he's he's trying to be as responsible as he can he's telling the good people at John Deere some of the issues that they have for some reason they were not super

interested in listening but he's he's working with um uh there's a lot of concentrated risk as it relates to um big egg and big data I don't want to I don't put all the blame on John Deere because basically every other uh tractor manufacturer is in the same game of collecting the data this relates to um uh carbon offsets uh production you think about in today's world uh corn is used for everything including food it's also used to create natural fuel biofuel a lot of issues related to Big agriculture um big eggs big risk this metallicy device is an example of a piece of communications Hardware that is in every or essentially every tractor which

fundamentally is a data collection device that sending all that information back home and so here's the big reveal Saturday evening or Saturday sick codes is on the main stage talking about hacking the farm breaking badly into agricultural devices if you're at Defcon you definitely want to check that out okay oh uh there is an announcement here

thank you so one of the one of the fabulous benefits of speaking at Defcon is you can make outrageous Pizza [Music] don't lose your cookies okay besides LV besides LV because besides LV is way cooler than Defcon because Defcon doesn't know how to do this uh is you can make outrageous requests and if you don't mispronounce the name of the conference that you're in they might accommodate you so I would like to welcome Mr bort to the floor to share with us your mic is right up there

all right well thank you for giving me a low bar to meet in comparison David [Laughter] just jump over who knows what industrial control systems are right what is an industrial control system it has now been zero days since stuxnet has been mentioned thank you so that is that is the common perspective we need to understand that all too often in our industry we talk to the echo chamber of ourselves right not everybody is as smart and as good looking and as nice as Jen Ellis foreign

right so who's got another answer for winning industrial control system is because that's not wrong that's actually literally what 95 of the world would probably say oh you got another one like the Tesla manufacturing floor Tesla manufacturing floor so manufacturing is a place where there are a lot of industrial control systems anything that controls life systems elevators cameras I like how you used the the name in it well it does controls right and that's part of it so the simplest way to think of an industrial control system is any computer that's at least 20 years old [Laughter] now unlike your random computers that you carry around with you or your pretty laptops these don't just focus on data

these affect the physical world that's what we're controlling it's not just data it's physical impact and when we lose energy we go to the Stone Age immediately that's not me worrying about cyber insurance that's not me worrying about data that is we are Stone Age like that all right so it's not often that I get to actually say I was a primary source so two and a half years ago at RSA that's another conference than b-sides hahaha then director Chris Krebs and I were up on stage he was the cyber security infrastructure Security Agency they care about security so much it's in the name twice and we called the coming rash of ransomware ransomware had been around

for a while but the difference was we painted it as a coming National crisis and as of 2021 just like when we think of industrial Control Systems from 2012 we think of stuxnet ransomware in 2021 became a kitchen word what I mean by that is it escapes the expert parlance of this is exactly what ransomware means it became something that was talked around the kitchen table because it actually affected your daily citizens life your average citizen does not think about cyber security in fact I would argue in your personal lives you rarely do as well you don't wake up and think about what am I cybering today except at my job right you think about shower

thank you for the water you think about breakfast you think about your boss as an you think about your car right you think about all of these things this is not a top priority and so when we get and it's always for a bad reason but when we get that opportunity as a community because something is bad enough that it gets the attention that is our opportunity for change right I am the Cavalry has been pushing these ideas for years and I mean unfortunately it takes those kinds of big moments to make change going back to where we're talking about risk we as humans are terrible at understanding risk and what it actually means for ourselves which is why we all

think well yeah that's bad but I got all these other things to worry about and until it happens I don't care so ransomware has now innovated right ransomware used to just be I just locked your computer right Saudi aramco all of their computers overnight paperweights now what's interesting about that is it did not directly affect any of the industrial control systems not a single one was affected but every single computer in the Enterprise Information Technology environment paperweight do you think that you can run your operational Technologies without your I.T nope so now they've expanded it not only am I going to lock your stuff but I'm going to exfiltrate it and then I'm going to

use that as double extortion so pay me the Bounty for well Bounty sorry if you've been following any of the cryptocurrency stuff that's been the latest thing is hey we're going to call you white hat afterward if you return the money true story that's a that's a bounty that was why I was in my head no so hey The Ransom to get the decrypter to turn your data back on so you can operate and we won't release your stuff then they thought wait we can do even better than that because what do we get with your data well I get all of your customers and all of your vendors and all of your supply chain

huh wonder if any of their data is in that I wonder if you not want them to know that and so that's the triple extortion now where I'm going to also Ransom that against all of the relationships that you have um so the question of this has always been almost all ransomware comes out of roughly Eastern Europe and Russia the great majority of it and the question has always been what is the alignment of clearly Russian intelligence military to that and for the most part there is no absolute Smoking Gun where somebody got caught with it being like in a direct handoff but there certainly was at a minimum a kind of wink wink will let you work here

and that changed when we had the Ukrainian War so one of the big ransomware groups there Conti within one day of the attack came out publicly and said anybody that does anything against Russian computers we're going to attack you back furthermore as we discuss this in a later slide we're going to talk about how very curiously a lot of the ransomware targets tied directly to Russian national strategy huh turns out we're not really talking about this computer nerd are we because this stuff doesn't just exist on the realm of what we have to do for computer security it's the question of motives and why um and then we have also example of activists using ransomware in the biela

Russian cyber partisans slide all right so why it's so important we don't call everything an attack have anybody ever seen those statistics where like we stopped 600 million attacks a second at the airport not a bulletin board when we landed in Vegas if you didn't see it you know which one which is the one we hate um dark Trace sorry I have opinions um and what they're doing is turns out that all of this interconnected stuff is constantly being swept by all sorts of different people who had written scripts to crawl it to touch and do that well when I touch a port and I touch that port and I'm enumerating the surface and touching that stuff they're

calling those attacks well now go back to our industry our community problem of we've come across as The Boy Who Cried Wolf and so it's hard for society to tell the difference well you said 600 million attacks and I don't see any real problem with that so we have to be very careful with our language we have to think what's the difference between reconnaissance what's the difference between initial access and then what's the difference where I'm actually causing impact right Dean mentioned this earlier the difference between axis impact those are the two key attributes when we look at industrial control systems the other part tying it to it right everybody's familiar with the CIA Triad

confidentiality integrity and availability availability is the biggest key in industrial control systems availability is the biggest key in industrial control systems I'm not doing it three times so we need to be careful about our language because it changes where people are going to be listening to us especially at b-sides where's your Bounty buddy all right so who's heard of the Purdue model I have okay the Purdue model is the de facto Enterprise architecture model that we use in industrial control systems and it's fooling you it was originally created to be able to establish the timing of these Control Systems so that I could get them to work and the problem is we looked at that

from an I.T background and from a design and a security background and well like one two three four five I mean that's defense in depth right foreign it intuitively gives you that feeling doesn't it but here's the reality Pixar is uh what gonna be recruiting me so we have it we all understand I.T beachhead so like in the Florida water hack what they did is they accessed a human machine interface these are higher level machines that are there in these environments so that I don't have to literally run around to every sensor or control system I have a single map of them I can see what the state of those are and I can make

modifications manually if I need to in the case of the Florida water hack the um the HMI they changed it so that the PLC stayed open the programmable logic controller stayed open and that increased the amount of lie that was going to be processed through the water at that point that PLC is at the bottom part that's the traditional operational technology these are relatively dumb devices they just do what they're told because all of the intelligence is being done up the beachhead that was a besides click please thank you oh forgot there's a theoretical air gap right has anybody ever actually seen an air gap yes you've seen a real air gap and maintained it and nothing ever

crossed it ever ever ever ever ever not once exactly so an air gap is not an air gap is not an air gap um and there are lots I mean there are better ones or Worse ones but the reality is you always have to have a reason for something to touch something at some point and the challenge here is that the primary reason that we have in industrial Control Systems these connections well just think about what's the most logical thing right it ties to a lot of what he was saying about in water well it's the same thing in electricity what do I need to get from my what's happening in the operational environment that ties to business administration

money Billy Money billing it's always going to be a need for billing so okay so this is the reality this is the threats perspective of all of that most of the access comes in through Information Technology because that's his job there is never an air gap so I move over into a beachhead and a beachhead is a higher level system like an HMI was an example I gave another one might be a distributed control system and here's the thing so when we talk about pure operational Technologies they're all these wacky really custom protocols and architectures and codes like you you take years of training to learn how to code at that level but here's the thing

that HMI and the DCS speaks all of that it's already built for it that's its entire purpose is to talk to them and tell them what to do and what operating system do you think those things run on the X works pick your version of Windows only two generations old

all right so that's what I looked like with my coveted beard you haven't seen cyber Gandalf I think that was an RSA uh interview so what we've seen in the last 10 years and this is public right I'm not sharing anything classified you can look this up and they've been publishing more of this since 2012 we have seen iterative intelligence campaigns we've attributed them to China to Russia to Iran North Korea shows up a little bit but their primary modus operandi is ransomware so that dear leader can pay for expensive whiskey the other three have much more interesting per you know purposes against us so why that's important is we talk about the Cyber Redline has anyone

heard that before right what's the point where an attack is something we're going to respond in a certain way because if you notice 10 years of hundreds of campaigns with this having an increasing Up Tempo and yet there's been no real loss of life as there why do you think that is what's that because that's a red light do you think the United States government's response is going to be we're going to cyber you back no the number one military in the world is not going to cyber you back if you cross the red line our adversaries know this and so what the intent has been is I want to map I want to do reconnaissance I want to

break in I want to understand I want to iterate and go through um a theory if I'm gonna can jump into the water example is that Florida water hack that change of that lie would have changed it to a dangerous level so does my hypothesis hold water at that point and here's why I think it does because there's a second manual safety in that process they don't immediately release that treated water directly into the water supply they manually test it in tanks that they hold this threat that actor who whomever it was knew that and what they were doing was measuring response

so last one Colonial pipeline Colonial pipeline Silver Lining is that that's that's the part that's the good that I have on there the one good part is ransomware became a kitchen work yeah speeding up got it so what's the bad the bad is a private company unilaterally made a decision that shut down oil and gas to the entire southeast of the United States they did that without this affecting operational technology directly it was I.T only that was affected and I believe this is my call and I'm being recorded so too bad Colonial you um no they panicked they responded out of fear they didn't know I mean they they shut it down with an hour of it being called like they saw

it and then they immediately shut everything down I second that what's that by second yeah and what is it that causes this fear the fear is when we don't know and we haven't trained it before they didn't train they did not practice the response tabletop tabletop tabletop if they had done just a couple of tabletops I promise you I promise I mean maybe some degree of confidence that that probably would have been handled differently thank you

all right lawyer Liz we need you to lay some True Stone on Municipal service sector problems are there any now not even a little bit and so typical lawyer disclaimer um while I am a lawyer unless you work for Scythe so Bryson excluded and I think we have one other colleague in here I am not your lawyer and even beyond that I am not your personal lawyer so if y'all get arrested in Las Vegas I am not licensed to practice here I will not bail you out of jail but I will laugh hysterically and make sure your mug shots get posted everywhere of course make sure I have full concerns et cetera et cetera et cetera but in a

prior life before jumping over into the startup world I was in private practice even before I joined the city of Atlanta so as anyone's familiar with state of Georgia because we don't do anything wrong or newsworthy et cetera et cetera et cetera but there are 159 counties in the great state of Georgia so if you're thinking of the sticky spider web that gets built when you have a 159 counties in a state that if you know Atlanta maybe you know Augusta maybe you know Savannah those are our big economic areas in addition to Gainesville Georgia in Dalton Georgia which are two capitals of Commerce for chicken poultry manufacturing and carpets so now you're thinking wow the great

state of Georgia and all these counties are just FL their coffers are overflowing with cash no so I got to serve as part of a team that were the county and School Board attorneys for about 30 of those different state or for 30 of those different counties and school boards so if you want to talk about getting in the weeds and talking about old school systems when you have some of the most economically depressed areas in school districts where schools are serving as you know everything in addition to counties counties are providing lots of different answers for all kinds of things so when you're talking about a ransomware attack on a County government that if they can't get

their systems up the Social Security checks other Community Support Services are immediately shut down so next slide so we're going to talk a little bit about and I kind of jammed a lot of things in here because some of us being lawyers and Rural followers held to the two slide requests [Laughter] thank you thank you very much some of us also read the emails and other Communications about the panel thank you so now that we're going to talk about Municipal governments as well because I wisely decided you know what I was having fun with technology so I'm going to get out of Private Practice and I'm going to go into public practice so I joined the great city of Atlanta as a

senior assistant City Attorney in charge of the world's busiest airport technology project so I in 2017. had joined the city of Atlanta in 2018. well before it became a national topic for everybody else around dinner tables in Atlanta we suddenly got to learn oh ransomware samsam and being one of the only attorneys uh in the Department of Law who had any background whatsoever in cyber security information security researchers hackers you can imagine kind of the oh moment when I'm on the department of aviation's network computers and my colleagues on the actual city of Atlanta Network computers suddenly we're going hey I can't access my files I got this thing up well by the grace of God we had also had

implemented Network segmentation so our Watershed Department was on its own network there was not crossover that we could tell we joke about tabletop exercises let me explain to you what happens when you realize there is still one uh computer that is probably still connected from the middle of the Airfield FAA requires that you have fire stations so that you can have fire trucks that can get to every any aircraft emergency with x number of seconds so we had one fire station that still perhaps maybe was connected to the city's Network that we are realizing in these so if you want to see someone run quickly tell them that you may have one computer that may be about to boot up that is

still connected to the city's Network that also is a crossover point to the world's busiest airport uh Network and if someone can get over to that and kindly unplug it from the wall and essentially smash it and do it right now that is what we're doing so what I wanted to highlight real quick in talking about municipal government so I could have painted this picture you've got these governments that are not full of experts everything literally is on fire when it comes to for example city of Atlanta and we had a bridge that actually caught on fire and collapsed so when you have all of this and you're thinking okay well what happened recently and Somerset County New Jersey

well they got hit recently and their county databases and everything got taken back to they got to party like it was 1977. so if you can imagine all the services um I'm not saying anyone Josh Corman recently uh became married um and you go file some paperwork well what happens if you can't access that paperwork because Somerset County New Jersey is partying like it's 1977 and they're going back to hand records so okay but not only that when you go a little bit deeper what happens when all right so our systems are down we're being we're on a reservoir attack we've table topped the out of this and oh wait how do we communicate with each

other or how do our citizens communicate with us oh I know what we're going to do we're going to set up Gmail accounts for the County Commissioners for the health department Emergency Operations County Clerk and Sheriff's departments we're operating off of Gmail accounts so fun when you think about it so it's taking it from this like high level oh no we're talking about it OT systems we're talking about risk mitigation now we're talking about Gmail because that's what the Playbook has so all right what's happening in the middle of the war room is all of this is going down well one of the great ways to highlight this is the great state of North Carolina in March the governor

um had an executive order says all right we've got the Statewide task force because as you're in the middle of the storm and you're thinking okay who do we call by the way half your playbooks are on the network you just shut down there are some weird people like me who are probably in your department of laws I think where I print everything so I'm racing to my office which was fun and heels uh to say oh wait I think I have hard copies of the following things so as you're pulling all this well the governor North Carolina says sorry we're gonna have this executive order everyone's going to work together we've got this task force yeah the task force

had been in place since 2017. do you know what no one knew it even was there for most of the time so great we've got all these resources who are you calling you don't know who to call because it's not obvious next slides please as we'll skip over that because we're going to jump right into the education sector because while education attacks are actually on the decrease decline what is happening is the cost so we're going to flash back real quick to save Atlanta because we talked about insurance earlier um yeah great Atlanta had just gone under our policy had just gone into effect we had checked all the boxes I believe we had made need one payment

towards our policy and 20 million dollars coverage but when you talk about schools and we're going to think of Alice Cooper school's out for summer well in New Jersey one of the largest school districts in New Jersey had a ransomware attack recently which meant for all of the high school students they had to cancel their exams so if you were waiting on that exam to lift up your grade that was going to help you graduate so sad too bad because when you talk about these school districts and everything being on fire think of what's Happening we're there still not back these students did not know what to do with overhead projectors because the school district had to

utilize overhead projectors they said well this is a great exercise in stem because what happens when you don't have access to it and you have to they were able to find them so not only that but they had no access to the online resources where you have now put all these great books in all of the educational teaching materials online in the cloud you're accessing them from your tablet no these students no longer had access to that because the county had to then pull it off but when you look at okay well who do you call when it's the school board the school district that's gone down well first of all you call the school board you call the superintendent

you call the school board you call the school boards attorneys you also call the State uh Bureau of Investigations you call sza you call the FBI the doj you also call and as I highlight on several of these slides about six different other agencies and if you want to have fun you sit with your popcorn in the back and watch all of them fight not over who is taking the lead not only that if anyone here has worked for a company that represented the city of Atlanta and our ransomware attack I applaud you a lot of y'all led me to drink because we had about five different IR teams sitting in the emergency response room all deciding that they were going

to leave because they had a contract I said great sweat you all have the same contract because that my predecessors didn't know but we didn't plan for this we didn't know what was going to happen we didn't know who was going to be to leave so everything is on fire it's even more so on fire when you have all these County governments municipalities School Board attorneys and no one who knows what they're doing nor do they have the resource to pay for it because they probably oh by the way accounting and city state municipalities I think a lot of them are self-insured because they get to opt out of insurance requirements so we have no pot of money

uh we have no idea how to solve the solutions and so I now work in the private sector once again for a startup [Music] so here we go thank you

thank you uh we have time for uh one question so if you've got one question if you can either walk over here or wave to Katie and she will bring you a mic quickly earlier this year I started to hear about an incident response framework called ics for ICS um do you have a take on that whether it's good or bad or well is beneficial or well yeah so yeah ics for ICS was championed by the Vice President of product security uh where she at now Schneider Electric she's at Schneider Megan Samford um and what her idea was so she had a background working Emergency Management at this in the state of Virginia and the

idea that she came up with is that if you look at how disast recovery and Emergency Management is done with like FEMA that it's standardized across vocabulary and process everywhere so when we're doing industrial control system response what if we could standardize that same way and so that culminated in um with a lot of joint agency work um that debuted actually at S4 in Miami in June um so in short it's a fantastic thing it's a great idea and there's a lot of steam coming behind it including training and certification Mr Corman do you have a question I'm just hoping that anybody that has ideas for how to collaborate with our water friend um or anybody up there

empathy starts with a drink it starts with a drink um no beer without water it's gonna start with empathy because the opposite of a profound truth is not a lie but another profound truth so these are both dangerous outdated systems they don't have time and money for and we can't allow them to be that vulnerable so if there's any coaching or advice you can give us besides giving you a drink uh as to how to best be truly helpful versus you know taxi drivers um I think true to the mission um you know we all live in a community that is being serviced by somebody um start participating in the board meetings and asking questions and

helping them understand what their risks are uh that's that's probably one of the best ways I could could offer up to to get involved and most of those who are televised now anyway so you know it's like any other industry though it's you know the the email server has a thousand percent more availability and uptime than the than the systems running the plant um so they they get computer racks and server rooms instead of the janitors closet so okay um please join me in thanking our fabulous panelists