ICS Security

scary would you walk you to the control of a power plant where you guys have been to Kennywood in the past couple years there's a ride as an old controlling minute and it's got like dials and gauges and it looks really archaic and each infants so when I moved into the control systems I was a very old industry very rigid industry but if you're doing automation for years and very very large groups as control systems move from analog to digital that's when the fun begin so go back to 2007 that my first engineering conference there's a group of nuclear IT folks it's of the nitsa organization nuclear information strategic leadership group I well came and there's a 500 people in there and it

was just about the time something that we were concerned about on the control-room side and I'm sitting in the room and they took in the tendencies and how many people are here from IT after room raised their hand they said how many people are here from engineering it was because they were split right down the middle and literally they didn't intermix it all all the IT guys were on one side of the shop all the control system guys were on the other side of shop and over the years I learned something interesting neither one understands what the other one does so engineers spend a lot of time focusing on getting things working they focus that particular layer

of the OSI model where they're focused on they forget all about the other sings IT folks don't necessarily understand engineering rigor so engineers design they build and they test with a nun testing they walk away and they go off at they design something else and they build it and they test it most IT folks find some technology rip it out of the box hook it up and start tinkering right and they get it kind of working then they think or some more and they tune it and then they stick it in and say it's implemented and wait for the patch and so a lot of times when we start mixing ite in with operational technology it becomes an exciting

adventure so does anybody in here have any control system experience good it's nice to hear so when we were building this presentation we weren't really sure the audience we're really sure how deep to go so this is a little bit of a higher level kind of presentation so we're gonna give a bit of an ICS overview because ICS can stem a lot of things right it can be something as simple as a PLC it could be something as complex as a full control system for an automated solution or something as highly secure as a nuclear power plant right it can be something that controls a robot it can be something that controls a television so it's something

that we have to scale based on whether we're talking about the HVAC system in this building that now is Wi-Fi enabled or something that's running an entire power plant so the briefcase is gonna kind of go along this way we're gonna talk about different use cases an adversary profile when I kind of turn it over today for for some of those deeper dives we're gonna talk about segmentation and isolation and and grading that based on the severity of a compromised we're going to talk about the security challenges that come with with industrial control systems and they are numerous then we'll dig into the stuff that you guys are probably most interested in some of the indicators of

compromise and then the approaches that we take to secure ICS so I named James Fiedler um I don't have the experience like Altman does that many years in the field I've been I mean security fueled by four and a half years um Westinghouse approaching two and a half years when I sort of Westinghouse I did the our cyber security upgrades are nuclear upgrades so hands-on engineering implementing and cyber security solutions architecture review network design learned the blog experience the time ago Westinghouse so everything I did nuclear has been there again recently good corporate went to the cloud so you know one extreme video so that further do we want looks like um no ICS so what's the short definition of

this technical definition you know it's a system that encompasses several types of control systems and instrumentation is used industrial process control what does that mean it's not very descriptive right so a bunch of actuators what's your bells just something that gives off an electrical signal to the PLC the suspect is supervisory control right it's all you know ICS networks are made up on so you know weirdly CICS networks so we have power plants you know nuclear gas whole solar nuclear is always air gets the most strict obviously manufacturing things we have automobile automobile fit food beverage textile are we ever finance we have petroleum coal chemicals robotics with health care research and development Skynet's in

there solar so I I see us it's it's everywhere in life we we depend on it every day even though you know you don't directly use we're not directly involved with it so you know who's gonna want to compromise and ICS network you know it's most likely not moving activism through so no eyebrows read it and I'm on Facebook you know a couple times something everybody gets their information off of memes they literally based factual news off of this so you know that there's always stuff going around like you know Russia's our friend multiplied this country is our friend no they're not and all the nice yes network you know that's what we're afraid you know

nation-states are gentle targets right so I'm gonna throw up some apt is just you know advanced persistent threat so you know this is all public knowledge it's wellness so you don't have to not if you're one of those people so apt 37 North Korea target ICS networks in South Korea in Japan they're noted for you know social engineering attacks using web compromised as their attacker 34 Iran they targeted energy chemical government they focus on cyber espionage in the Middle East they're noted for ms-office exploitation apt 33 so I ran again this group targeted aerospace they've targeted energy they focused on US companies they use use HTA extension as an HTML file exploitation so we can see

a pattern here on how they're trying to get into companies Vietnam it's kind of interesting so this group 32 they target foreign companies investing in Viet Nam's manufacturing was it they wanted to do it themselves is cheaper probably something with money on that 130 China this is China Creek so they targeted members of Southeast Asian Asian Nations on their noted for long term command and control dating back to 2005 these systems so they're also known for infecting air gap networks specialized software and us beautiful information they need their attack vectors you know large suite of exploit tools um just if you think about it command control from 2005 from a nation-state good luck getting them out of your system if

they're that deep in their they know it better than you do twenty-nine Russia not getting that one um work we're guilty of it Tim you know us does it Stutzman that's very popular that's always the one I think when I think of ICS so you know how successful until you know basically you f1 class results in Israel did so that's how do exactly technically got out so when it comes to ICS network and depending on the field you're in the sector urine you're gonna have Guardians wanting come after you because your critical infrastructure so that's your control system so what types are there out there so a lot of these terms you know these might be

familiar they might not a lot of these will be thrown around this presentation so I do want to clarify some things skate and Decius they're often interchangeable Artie use PLC's to very similar tables talking about PLC's earlier PLC's are basically a device that's connected to a sensor and actuator they're then connected to a supervisory system you know these devices feed the supervisory system process variables and set points those two terms are gonna be important later so you can dive down just a little bit and the difference between skate and DCs skater is the key ability to perform a supervisory control action or operation over variety proprietary devices so clumps of actuator is just something that sends a signal to escape device

it's what it's looking for now PS PLC's they're gonna connect back to these devices they're gonna send some sort of process variable it would be displayed on the agent lights the skater GUI basically you know an operator is gonna control and respond is needed so a Decius this just means to serve in a control system they're similar to SCADA and function what the key difference is you know in these systems there's going to be a larger number of network connections between the PLC's between the sensors between the controllers common attribute with DCs environments there's going to be more computers on milites to be more servers and controllers between these proprietary connections with PLC's dcs networks are also known for being

more controlling your process and controls with less human intervention SCADA more or less nice human intervention DCs you have software running on an OS basically that can help control it's less user intensive I guess you could say it's just general generalized summary but it's close enough for this presentation so next wall grid and talk about PLC controllers that's bebés devices so PLC stands for programmable logic controller they connect directly your timer sequencers valve relays they read output data the most basic function of these is just connect to an electromechanical relay er to some input/output device they're generally going to have smaller amount of devices going to them how they run it's going to vary on you know hardware

what system it's installed in some PLC's I often just referred these as controllers they're gonna have hundreds of devices connected to them so these kind of Peel caesars I'm just going to miss controllers they're actually gonna run an operating system they're gonna run something like the v8 or something like OS 9 so these operating systems you know they can have more advanced algorithms they can run software there there's gonna be more using them sending data back to the supervisory system so obviously with something that runs an operating system you're gonna have similar similar vulnerabilities similar vectors that you're gonna see on a regular oh s these things are run on 8-bit 16-bit 32 64 but a lot of all my

phones need so there's still like an operating system though if there's going to be separation between user mode between the kernel level they're gonna have unix-like processes they're gonna have secure Sheldon since this long bluetooth USB named you Europe full fledge OS all these things so on these PLC's they're gonna send process variables the setpoint back to the skater twice process variable it's basically a measured value there's something that's being monitored right so I think the process value the process variables are it can include something like pressure temperature love flow ups the setpoint is just the desired value that is supposed to be act you might want cold two hundred degrees so in order to

calculate and control these there's something known as a pit controller it just detects the differences between the setpoint process variable so it's just the s PPV air they control it through positive negative feedback so put in layman's terms you have one in your vehicle you know give a set point value of 70 miles an hour and cruise control the second P V at 68 take controller is gonna say hey do your engine your car's computer go back up to 70 that's just the basic way to describe these things so these are relayed that there are supervisory devices back to working back to the scale that's the DC s servers if you can modify these variables you can

disrupt the system you can spoof you can spoof across this variable like students did know you can start closing damaged all on these you can start doing whatever you want modifying these it's very difficult to do it's going to require a lot of knowledge but if you're going to do this you're gonna have deep pockets and a lot of experience you're gonna be able to pay some of your need for many people to do this so they take over so

so regardless of the type of ICS network for the most part we want to make sure that we isolated from general-purpose networks especially general-purpose networks that have an internet connection if it's something as simple as your your building HVAC system it may be a low-grade firewall maybe a next-gen firewall because the consequences of that that system being compromised are significantly lower than maybe the bad system or the security camera system in your building most industry user layer approach I'm gonna jump to that for a minute this is certainly not proprietary Google we're all familiar with DMZ and network security layers fortunately a lot of the process control industry does not have a very strong or very mature

understanding of information security so there's still very much on the strong shell gooey middle approach that's changing and as things go a little bit if things get more and more integrated and as regulations are starting to come up to speed but particularly the Ender energy industry there's there's a lower level of maturity when it comes to securing your control system we want to make sure that operations is DMZ off from the corporate networks so the folks that are actually being able to make changes and get to the human interfaces are going through a DMZ or they're sitting local further down your basic control this a little more closely before we put it in here there should be

a firewall down in here because your engineering workstations are possibly suitable for compromise between engineering workstation and the actual HMIS human machine interface which of the folks that are actually sitting doing the control and the instrumentation in control systems and safety systems are at a much lower save your security level so the nuclear does he uses a four layer approach so when we look here nuclear industry basically has level 1 it's the corporate network level 2 is the plant network level 3 is data acquisition where things like historians and data transfer out can come from and then layer 4 encompasses control systems and safety systems so if you control the reaction in a plant or it's something

that needs to be actuated to shut down safely it has to sit on the inside of a deterministic Universal very unidirectional gateway so some people call the data diodes all technology makes a day-to-day oh there's waterfall there's a few breads out there they do them now but if you're in a nuclear power plant and a lot of this I think is more perception risk than it is actual cyber risk but we can honestly say there is no path in to any control system or safety system in a nuclear power plant from a corporate network or from the internet and I think a lot of that was the NRC just wanted to be able to make

that statement as of 2012 the NRC heads I believe eight milestones that had to be completed by December 31st 2012 and one of the milestones was to completely air-gap every controlled safety system and so there's no there's no network that pushes them back yes that's correct so there's literally there's they they cut their receive wires on them on the connection and most data diodes are there's a transmit unit a receive unit and a single fiber that goes between them and and some of them depending on the data diode technology that you select have specialized transmit and receive or they remove the auto gain so you literally when you buy the device you have to tell them the distance

you're traveling and they'll they'll set the gain on the transmit and receive you know when you when you have a standard you know fiber media converter or fiber channel even though it's transmit and receive there's bi-directional communication that goes on to set the auto gain so the de laser doesn't burn out the camera on the other end so some of the data diodes were put in place with that functionality to disabled so they'll set the the gains or you say you know we're gonna have a 300 meter cable between just the transmit and receive and they'll dulcet the game for that distance there's some other neat things that we found with data diodes like it's

not we but the industry found is that they there were some people that were actually figure out if you could pre compromise a device on the internal network you could look at power consumption on the day to die of and actually get data back into the network and get your data out by by pulsing the signal you could get data back in seeing the power consumption on the transmitter unit since I read that it was imagery compromise that David that would bypass to do that so this fields always evolving

so the power plant would have yes no the NRC's is a regulatory body they're not going to be fine so the NRC wrote the rule right so the rule is 10 CFR 73 54 is this just cybersecurity law for nuclear power plants and they put a red guide together and then the nuclear energy industry their Institute also put some guidance together and most of the power plants follow that it was loosely based on on NIST 882 and 853 but they took a series of controls and prescribed them as opposed to just saying meet 853 they said here's a list of controls you will do all these controls on every device so if you've done assessments in a nuclear

power plant in the early days that was that was kind of how it was done that's not helping with interpretation to control this yeah no no no problem you know I can I can digress for hours on some of these things and so one of the first things we want to make sure that we're doing because the traffic is different and most industrial control systems are very static in nature the more you can isolate them the easier they are to control and we when we start talking about some of the the challenges I'll be back up for that you understand a little bit more when we start talking about the weaknesses and industrial control systems and some approaches to

make up for those weaknesses leveraging some of the other pieces of this system

oh it's gonna do a couple touch points on challenges of ICS security so there's very few tools available for monitoring these systems everything is proprietary you know how'd you buy something for security if it doesn't know what's monitoring you know everything's coated mostly just to give the system functioning there's nothing more to it they just want to get it working don't touch it it's Gator shows process variables it shows set points that's great but how do I verify the data is actually concrete how's it how I know it's actually great limited operating systems it's an issue vxworks oh it's not other systems how many people in the ICS network actually know how these things work probably not

if you go to the engineering firm that it installed the system they probably don't know even know hm ice they're scary nobody knows how that scared device is actually getting its data no is it some magical fairy who knows right so integrating some sort of monitoring in between these you're asking for a lot there because nobody understands how the processes work along infrastructure learning curve that's another challenge a lot of people are brought in with an idea how this equipment works you know what it does they're eventually going to learn the system they're gonna know it more than anybody else in the company it will be the SMEs on eventually but you know when you ask person a to integrate

with person B's Network don't they know how to get these devices you know working frankly how do you get them to communicate there's gonna be a barrier there nine times out of ten they're probably just going to get communication working and not really worry about security integrating new software and hardware that's difficult some systems are fine-tuned you know there's challenges in adding any software that might break proprietary software system you might not be able to add monitoring to network you might not be able to change a path this increases latency in the network no padding software increased overhead by increased delay no at this issue with that latency process variable set points to rely on

something to change latency you're throwing other things out of whack you're you start approaching a safety issue at one point so it's security or safety you know safety always wins so to get the goal of more on you know a few tools available a few tools available non-standard Hardware everywhere nothing in these networks is commercially off-the-shelf can't go to Walmart Aero CDW's don't go sell any of the stuff private area software and protocols they're almost always used and no one knows anything about it they need if you actually need help on these UNITA the developer people implementing there and have some support but not as much the proprietary software it's often poorly written it's created with data

programming practices law to stuff your users from 1998-2000 you know you're using encrypted communication secure protocols nobody worried about this stuff a lot of places think you know proprietary means security because nobody understands it how am I supposed to you know hack your proprietary protocol if you know nobody knows how it works will you send base 64 encoded over the systems not very secure right outdated software it's usually intentionally looked outside of update programs or upgrade programs you know this might be due to budget patching it might break the system it's working don't touch it there's a ton of excuses when it comes to trying to update outdated software

some documentation doesn't match are there is there anybody that problem almost trying to translate these messy protocols it is something that could in some cases yes so there there are some standardized industrial protocols like Modbus PROFIBUS steel bus that control system vendors are starting to leverage more so it's it's trying to get away from I'll pick on Siemens because they've been in the news you know up Siemens and I wrote my protocol for my DCs and that's how it's gonna communicate but there are a number of them and they wrote it the way they did for a reason right there's a there's a vendor that that I worked with in the past that wrote the deterministic

protocol for their DCs and you know there's two schools of thought you could serve network bandwidth and only transmit the points to change you know we only transmit the deltas every second or every tenth of a second depending on how fast it goes or do you broadcast every point every second drawback to to broadcasting every point every second is it's it's continual Network overhead right and all that data is out there every second the advantage of it is if you have a big transient you don't get a network spike whereas if you're only broadcasting that point data whenever there's a delta and you know in a nuclear power plant that could be significant if you have a loss of

coolant event and all of a sudden pumps are kicking on the pressurizer pressure so it is increasing heat to make sure that we're making up for the lack of pressure do gen-i leak all of a sudden all the state is changing is the network sized radar currently capable of handling that giant spike so there was a couple of schools of thought there so there are always going to be proprietary protocols yes

you know not not that I've seen for the most part in a lot of cases it's layer 2 multicast unencrypted layer 2 multicast we're not even at the TCP stack no that's starting to evolve and some of the vendors are starting to say you know now that now that they can leverage QoS and some other some other features that when they were writing some of these things years ago weren't necessarily available during the move from FDDI to Ethernet we're starting to see more of it

yessuh sometimes if you have the rated they're very so that that's gonna vary vendor to vendor if you think back to how the InfoSec industry was maybe ten years ago before information sharing became a staple of our industry where somebody had something they kept it fresh but if we're not going to release this because it's gonna make me look bad microsoft certainly not gonna publish a vulnerability we're gonna go hide and fix it and we're still evolving there but the industrial control system industry their bread and butter is their protocol so they're it's very difficult to get your hands there there's there's like but they're using either big data type stuff or that or signature based

things so they're they keep necessarily tell me what's going on in the protocol but if peeping out enough identifiers to say you know it's kind of a comedienne

this one which so we spent a lot of time looking at that and for the most part we end up having to duplicate the ICS and for reasons because it's difficult to simulate how something is going to respond in an adverse condition I can simulate the look and feel of a control system I can simulate an operating system but unless I'm writing that code when I break that code I don't know if the simulation is going to adequately represent what's gonna happen in the code so some of the things we developed early on for testing results of a compromise is literally simulating Stuxnet for lack of a better term if you think about what Stuxnet did is it sat

between when you break down a controller there's the actual controller logic and then there's the IO right so the IO is what translates the control logic into a 4 to 20 mili-amp signal whether it's a digital or analog signal that goes out to your pump or it gets data back from here or meter right that says move this thing 20 degrees and in it literally they wrote a hardware abstraction layer between the IO and the control it was it was feeding the control everything's fine and telling the IO to do something different so we kind of built to a simulator with two data sets the data set was what's presented to the plant simulator and another data set was was

presented to the plant simulator one was to go back to the user and one was to go out to the control and that way we could say okay if you have a situation where your control system has been compromised and the data that's coming back to you is different than what's actually happening in the free end how do we identify it can we identify it what the tools do we have and that's something you had a little bit in don't go to some of the things that I usually when I design it a security system with the limited tools that I have right now we're kind of talking about vulnerabilities and a lot of times the

vendors at least historically didn't have vulnerability information so when I'm buying a PLC from nuclear instrumentation for instruments I'm getting a little box right and it's running an embedded operating system that may be proprietary or it may be a scaled-down version of a VxWorks or a qnx or sometimes even Windows Embedded but they didn't bother to enumerate the vulnerabilities right they're getting better so the NRC requires all the power plants that have their current current vulnerability with

yeah yes yes so to build a representative sample of the control system is somewhere around 60 grand and that's just that's just the hardware that's not there yeah for the most part they're starting that's starting to happen every is any this is if I'm sticking right where my base and ologies in the power industry every starting to put some things together and they're they're starting to look at building vendor sheets that have their their common vulnerability and security database so they're they're building a new assessment model from the ground up I don't know if it's going to take off or not but they're looking at if you think if you think in engineering terms of the material material data sheet all

right if you're gonna build a device so they're looking at I've been doing this for almost a year victim they're building literally a component data sheet and then you can stack those component data sheets into a system and layer the defense of architecture from the ground up okay

right so you know Stuxnet maybe is a bad example because we all know that was an inside job right I mean but the way it worked is a good reference to understand what's happening but it's almost all code developed by the vendor that were purchased at least at Westinghouse we don't really so so one of the things that that we have to be careful of if you're gonna have an engagement in an industrial control system network and you're gonna do a security audit be careful scanning it what you'll find is a lot of the network stacks or maybe incomplete or developed to the bare minimum of the standard and they may not respond appropriately to non throttled

scan we I've ripped a couple of devices I've seen some of my customers come in and want to run a necess can Dave leave ricktum we had a couple that were unrecoverable you would think you could power it off and bring it back but for some reason we had to replace the units you all also find out that some of the operating systems have not evolved or in some cases they're using an old operating system because that's what works and that's what always worked and they don't necessarily prioritize processes appropriately so if the network stack gets a higher priority than the operating system it'll freeze you know we're not gonna see that in Linux or Windows but in some of these

older operating systems we'll see that and in other cases you have a deterministic network and you overload the network it's very susceptible to dos I don't mean DDoS it's acceptable to dos because the network bandwidth is very tightly calculated to make sure that we're getting that deterministic data we need it

it's a regular lead on levels I'm not sure which means great so each levels gonna be gonna be isolated at layer two right you're not gonna see communication going between those security layer OSI layer 2

I guess this is so while we're still on a topic so again a lot of the a lot of the tools that we have are very good at looking at things that we know about right we can we can look at DCP IP we can break it down into ten ways from Tuesday we can look at what's happening at even that layer we can look at application layer we have next-gen firewalls all of those things take time and when we're looking at even voice traffic right the latency that we can accept voice is something that we maybe not be able to tolerate in industrial control so working on a particular power plant and they had a data link that was

coming in from the other side of the planet and they wanted to secure this data I kind of like that's fine I'll stick an industrial firewall in between a link and they said well we're kind of with the limit of our latency I'm like woods with latency and they're like six hundred nanoseconds mm so I you know I pull up the specs and it's two milliseconds on the to get through that firewall you know I had six hundred nanoseconds to work but so so what do we do we don't put the firewall in there the best you can do is put a passive tap in and steal a little bit light and look at it in parallel so there are there are

certain things that that we're up against when we're dealing with control system because that data has to get there when it has to get there nuclear reaction is maybe a little more difficult to control than some of the other things that we're dealing with but you know if you've if you've got robotics that have to put a screw down at a certain time in a process and you know if your if your tolerances are very tight and you know that piece of material is going through and it's got a hit when it's got a hit and you're off 100 hundredths of a millimeter maybe 10 thousands of an inch you may blow up the whole process how much does that cost so

those are those are things that you know a lot of times when we're putting RIT hats we don't get the engineering side of it it's just like well just just stick that barrel in there you know I need to inspect every one of those packets as it goes through it yeah well you just broke it so we'll look us a little bit of approaches that I I've used in the past things to deal with that

so a lot of issues with ICS networks it's there's also beer with them if you don't understand them so not completely understanding am i supporting infrastructure it's going to be a challenge for keeping these networks secure there's limited understanding between you know an HMI users they're not gonna know what's going on they're gonna be trained look at the information on the screen they're gonna know what the values are supposed to be they're gonna know how to do their job they're gonna great at it right but but they can understand something on the ordinary occurs so trying to get more towards I have C's than previous bar abilities that we're discussing to expand this a little bit um the PLC mi7 alarm

it's bad right they might send some sort of debug some sort of you know the variables down all the way no somebody might ignore this alarm somebody might just say you know what we're gonna look at this in the future like we don't have downtime we don't have an Allen to our production we can't test anything right so they're just gonna they're gonna be at the HMI there's scary stuff the value where they needed at some point you know it's just gonna keep getting recent reset and everybody's hearing or nobody's ever fought an issue nobody can figure it out nobody nobody has time to look at it so at this point is training a user to expect it's normal

on the network you know this is odd the Ordinaries of malicious might not be you know this bad solder it's possible but we need get this investigated and moving into indicators vault compromised you know a lot of people and things like this come up they're not gonna investigate so we have to train users to start looking for these indicators of compromise in you know this stems from vulnerabilities that might be on the system so we need to investigate some that nobody understands you're gonna get a lot of pushback you know it's not broken don't fix it no one understands to the breaks when I can get it running again now we can't afford out I can't afford budget

all these things they can be indicator to compromise there's tons of things on ICS that work and if you'll understand it you not gonna know what to look for for anything I owe C's so things we look for for indicators compromise um it's not it's not a whole lot different between a regular Network no we're gonna be able for deviations in network traffic dcs systems but there's a lot of computers firewalls don't you know you get domains on there normal normal practices apply to this you know it's possible monitor traffic to these PLC's and what's going on we can do that whether or not you're going to implement it it that depends on the sector urine

so you know changes in setpoint or process value data it's another very important IOC it's probably one of the most difficult things to detect on network as we were discussing before and there's not a whole lot to do to you know check for changes in this and how to actually figure out something some licious data historians process historians they come and use and often I was grading custom software to rate these things changes and user behavior you know that's an obvious one bond ICS network everything's predictable you know I should be able to know what's gonna go on in the future nothing changes these networks were very static almost workstation I took that's an obvious way no changes in dcs servers

deep these are all standard operating systems sometimes they run Windows they run Linux you can look at no regular indicators compromise you doing your homework station another great thing for finding IFC's is using thread intelligence so sir the IHS the off for investigations you guys know this then they release the roles and techniques that are currently going on in the wild it's a very very valuable resource so moving forward we're I'm gonna just go into a little high-level overview will we actually look for for OC no deviation Network very static predictable so we're looking for a future new IP address' its new hosting its new our banter exchanges in other countries cancellous that's easily brain poor connections we all know that

all ball connections so stand like that pivot there should never be a vulnerability scanner on the network there is you know it's gonna be documentary me in downtime it's gonna be from a well-known IP we should be able to notice this really quick so how do we actually detect this stuff how do we control it so again stance our security techniques IP addresses host names ARP ARP entries we're gonna use some sort of integrity monitor ring use road system detection can't flooding random port connection scans anything we're gonna use a simin that same thing we use on a regular corporate network knits on for us for nuclear it's always as miss as long this we want everything to pass by react

earlier false positive breaking something yeah that'll bury on the field so you know since mandated your net send syslog Pierce and grab whatever logs you can know all this it's well talked to me should be able to create a list of IP addresses and pipe it into your sim you should be able to write roles around everything I just have to be for everyone comes an ICS network so the next thing to discuss is changes in set point data this is incredibly difficult to do it's gonna require a lot of knowledge of the system this is a point where they beware you know IP was on one side engineering was on the other side when it comes from bond tree you know

set point data and looking for vulnerabilities we need a work with the IC engineers all the text they know what's going on we don't we don't have the skillset to actually understand the software so we have to work with the people to understand it so no simple data it's predictable should follow a pattern by patterns like summer versus winter on certain temperatures of no product output changes in functionality with peak times and down times so another indicator of compromise to look for it's you know you're gonna take your set point that your your data historians you're gonna want to compare the present compare it to the past are these values are you similar last year and you know what they

are now is there a pattern moving in the direction I take all my data and put it in a spreadsheet you know it's my heat index going straight up you know am I gonna start blowing something up what's going on you have to just compare on a historical look we're gonna want

my for us were trying to get there but right now it so you know our historians you know they look at that but to the level of you know basically gets threatened longing for this stuff not long

in distributed control system it's going to distribute the status of everyone so typically you have a series of controllers those series will work stations and they're all in network they're all broadcasting whatever is they need to broadcast so if you're if you're an operator and you want to make a change you're literally changing a set point value the next time that gets broadcast the controller that needs it picks it up that's my value my current value is 4 it needs to be 7 it makes the appropriate change in the control logic to effect change until that set down that goes back in 7 teams some of that control I just will have ramps in it or we've instant change or

it'll be a change or you know needs to be set in seven minutes or 87 in my experience I've found to us it's the beautiful initially built in for a cyber-terrorist long engineering and control systems will happen where it say something if this PLC breaks and worried about that having something really bad is going to happen we'll start putting in redundancy that was working that redundancies usually just I had to st. PLC's or cyber security it's so I guess some things it's inherently engineers design you might see that and say I don't trust just one PLC I won't have to verifying this very interesting

that you're listening to

there's no other pretty sweet of ones in broadcasting on that right so the broadcasting

so how do we detect control you know there's there's multiple ways generally the first thing is user process historian so this monitor is aldea this receives it literally reports every single set point

we have that amazing tool

but we've got got visibility into parts of it and you've got some dark so the historian and the control system are very good at looking at data they're very weak security controls network controls

so your data is orient I mean do you silo that off [Music] [Music] there's a lot business system here

[Music]

so what do you think that like like you saw a little bit set point changes throughout so with a nuclear power plant you look at the historical trend 820 bills from offline you want to see nice solid grant zero even hundred percent power 18 months and one straight line right months

it'll be a detection not user analytics not history and not you know smart algorithms two pieces together Sims can do that these days nothing you know we're technically using its gonna do it you know there's gonna be stuff in the works but from what I've seen I don't know you know I've been technically out of ICS for a few months so we could we might be using a 12 but for me we haven't done anything that specific for state hate user a B always comes on at this time it's just we're gonna have signals we have some sort of you know these are the windows they shouldn't be

so if you're an operator station you're doing that you're doing that control room that you didn't get into without going through radiation scanner metal detector x-ray I passed the whole bunch of armed guards or sitting in a control room with three people to do with everyday behavior observation so we're looking for not necessarily an insider threat so we're looking for you know we typically see five set point changes of the day we just saw changes

literally the way it works you read the step operator which opens to step this is I I am going to move the mouse over top of this button on the superior actuator operators as I confirm you move the mouse thanks dad I'm going to left-click the mouse on this buttons when you're out here so if I confirm click the left

I was I was just listening to a presentation right chief nuclear officer of water for our plan right after Hurricane Katrina he was talking about was length to weather a hurricane and at about day three the National Guard came to them and said there's a chemical plant manufacturing plant just up the coast when the hurricane came they just shut that light off in left they did shut anything down they didn't know what this days of his plan was going had this thing gone up they were having discussions as to a nuclear power plant they were determining when they bring everybody from the plant into the control room they had eight hours of survival time or did they keep the

operators in their plants table for the 24 hours it was very very interesting scenario but to keep in mind that there are process controls some of them talk about mass destruction much more dangers so you know like I said we start talking about industrial control systems like safe to anywhere from the HVAC system in this building all the way up so think he Carter Davis Doran spring to answer your question you can either have role in their story or you have another thing often that work to do it but we do keep this pretty secure so got just the beta historian the setpoint data we can actually create some rules around a so there's talk

about you know how much are we actually because proprietary for the call so a lot of times we also rely on the software to generate its own along so you know some this software it can generate one those long schemes every system so you can also type that in the sim first it is needed and create alarms around those you know some some proprietary software it'll actually break all of its endpoints all of its process variables to work at file we can parse that we can actually make some rules around that so we can take modern security software and integrate it with this another interesting one they have PLC firewall so these devices there's a lot of NDA's so can't really

go in much detail but now I can put a PLC firewall between the actual PLC a sensor between the PLC mesquita device no I don't know wow I can set these firewalls only a while certain tradition so I can only allow one-way communication to Modbus device for example so if I just one you know communication going out of the Modbus device bet the SCADA device I can tell this PLC firewall you know what registers is watts and what range you know 800 to 1200 one way of anything out that I don't care about exactly two way it's not damage anything so these are very great I keep integrity you know it stops an actor from modifying

I don't see change in behavior in all this workstation activity you know this this is straightforward for us so everything comes down shift work and mins working up to you you know we shouldn't be seeing any weird codes on system there shouldn't be windows logon code ten RDP from a random server you know code for for a scheduled task for a regular user we shouldn't be seeing this stuff this no sim should be able to tech this pretty easy odd filenames this can be difficult to tech but using threat intelligence you can also find some some files on server excuse by process should be read-only no you should monitor these files closely for changes new software

system no you open the door for ransomware so this section you know it's straightforward but I also see it talk about more in theory than actually being implemented a lot of places I've worked on you know it's just when I see its other corporations like a lot of people don't like this their applications they hold whitelist or software they don't use application control so you know the Sims gonna alert it's gonna alert on evil activity but they only go so far threat intelligence helps voluntary monitoring might be able to walk a couple miles down but no you actually need application control you need white listing of some sort you plug ransomware and all the ESS and just kiss goodbye

you know when using application trained on these systems you need a lot of things done to follow the dog directory level because you can high stuff anywhere it should be pretty easy if you guess you know wearing a store files you just whitelist based on directory a lot of things proprietors sticks out like a sore thumb a lot of times you're gonna have to work with the vendor for application for one ICS network when I've worked with whitelisting an application control I've hurt the system multiple times I freak multiples one sitting for no it happens so threat intelligence I kind of want to this is actually probably one more important sides I have you know it doesn't

actually list the single indicator threat Intel to subscribe just going on the wild so no wonderful thing wonderful resource you know the FBI DHS cert you know other three letter and organizations they perform in investigations based on your Intel they receive something fallen a compromised then they release this information to us so for an ICS Network you know we need to know this information it gives us the campaign's going on you know what nation-states are actually occurring right so what's going on in the wild right you know a nice yes we're kind of in our in public we don't know what's going on using threat intelligence it's a great resource for us come these articles

they're incredibly detailed they go point eight no a-to-z everything they cover everything like how the actors started how they got in where they got what data they got out they just everything they can find they list in these articles and if you actually follow it you know the techniques that you know these bad actors use the very clever that very witty that when I read they personally they just blow online so no control all this text right here is from a single threat intelligence article top ten clients you know mr. snort rolls bottom stuff is your best practices this is actually pulled from a threat intelligence article of a campaign with Russia this campaign specifically targeted when the nuclear

power plant I was working out I was actually on site and then try so there's a lot of information actually cut out half of this and there is this isn't just you know do do something simple this is actually detailed relating to the campaign so the best thing I can recommend is look at these articles be proactive for the linear program

slowly between maybe rewind a little bit we all know the cih I am very confidentiality integrity availability in most control systems number one piece when I first say that everyone pushes back it says what about availability well you're just you know I would say earlier a lot of times that availability every one of our systems in our clan are either dual or quad redundant so if the system drops they'll feel in as a statement a notice Dave almost all so that pushes availability down integrity is way more important from a security standpoint that robot arm swings an extra six inches that things can happen the temperature goes up a grateful combustion or chemical reaction that

things can happen if it's off just a few degrees or the mixture of something it might just be enough that you don't notice it which is why being able to know good helps you find bad industrial control system is so important taking the time to analyze every little piece we care of looking at all of the processes all of the protocols the communication and what's normal what talks to what everyday so if something is out of normal it's easily identifiable and it just takes time other piece is those companies don't want to spend anything they don't see the dollars it's easy to it's easy to talk to a business owner about protecting their money it's harder for

that or sometimes to be able to translate that process that makes control systems are not cheap compared to a standard IT system factor commercial details considerably more expensive

and they had some groundbreaking technology it was fantastic they did exactly what we're talking about it so it's all of process control data all of the network data integrated and fed it back to the civil look it been correlated it and they allowed you to make rules or set point changes per second compare that to the batch data literally pull all that out integrated and feed it into the same to do an additional parsing process the alerts

and that's where needed adjustable controllers I already put in this control system for us it's at a nuclear power plant by the time you've gotten in there if you pull a screwdriver out of this note on the procedure there's a guy with a weapon so I have to talk to you only if works she was she was going into No

soup on site she was putting actually doing a similar to going through all our site clearance with the security guards had an updated basis in fact it didn't work that was with them less than 30 seconds from the time who are you what are you doing here these commands in this room we went to south so you know in many cases they don't want to hear so you encounter that too there are technologies there are companies that are doing some of these things

didn't they think yeah for them at this point securing their plan outside of the regulatory compliance they don't see this you know they're not they're not making an extra dollar okay to go digital control

the plan right it depends on a lot of things the how power soul isn't regulated or unregulated market leave a slice yeah the average I think the average calculation that we use 200 grand an hour

[Music]

there's only a few though aside buys the big one and they're pretty open each vendor has their own semi-serious question I think so in my experience around you have a standardization yeah well even then they still want to integrate extend what was I

so all the devices does matter Peter made by they're all going into one story apparently so you can analyze that data relations they already have in PowerPoint that's true if you needed to summer manufacturing the less regularly won't seek consent thought I was like there he's grating things because he has to do everything but if you look at that chlorine playing per se I guarantee samples or this itself is this decision and you know the CCC understands that we need enter creation you mentioned for reordering materials that's associated with that so business drive they will get that useful but it's like I just need to know this random temperature in this room there's no yeah

all of all the energy is oil gases as well

[Applause]