Mechanics of an ICS/SCADA MITM

um my talk is about a uh man- in-the-middle attack uh against this industrial control system here very simple industrial control system just got a controller and a PLC or a PLC controller programmable logic controller and an IO block uh but I actually got a chance to get down into the details of what what went on during the man in the-middle attack itself um to the actual control signals so I try and go into that a little bit more um about me of course uh I'm a senior investigator at Conexus Consulting uh I do industrial control system and skaus system uh assessments and designs for a variety of customers uh anywhere from energy companies to oil

and gas to process control to um uh manufacturing and everything uh pretty much between uh I've been doing this for 20 plus years uh I'm also a developer on our analysis software internally uh and um I previously worked at nist in the engineering lab where I did Control Systems automated vehicles uh Wireless sensors and systems uh I'm now basically uh um I've been a consultant now for three years uh that was my midlife crisis so um this also shows I'm also the uh uh co-chair of the ISA 99 committee and if you know anything about industrial control system security that's the basically the committee that's developing all of the industrial control system cyber security standards

internationally there's nist is doing some and some other people are doing it but we're trying to coordinate everything across the board um so when this shows up Main and the middle attacks are nothing new I'm not going to run the coolest man in the- Middle attack here uh because honestly industrial control systems are the kind of things that you can look at wrong and they go down the in general they're very fragile they're very stable so they work really well as long as you don't go near them in general um some of the newer systems and and newer Protocols are doing better uh but um generally the systems are to be protected uh like your most secure data

they've got to be uh protected with layers of protection so that you don't get the standard kind of uh things coming down in there and a lot of times even protected to where you don't want your it Department coming down too so the IT department a lot of times is uh restricted from coming down into the system as well there's a ton of videos and tutorials and all that kind of stuff oh thank you out there in the world for doing um man thee middle tax so I'm not going to go into those uh one of the best ones I actually found was from Iron geek uh he has a really good tutorial on um doing man the midle attacks and

writing filters uh and everything in the mean uh in between so I would definitely highly recommend taking a look at his stuff uh if you get a chance iron geek uh Adrien krenshaw he does a lot of the the videos and things like that but he also really knows a hell of a lot about Eder cap and and things like that so um this is not a talk about how to run a really cool man in-the-middle attack it's how to do a man of the middle attack and see what happens to the control system when you do that so I IC and ska systems they really rely on deterministic Communications and when you run into Mana middle attack

the determinism of those systems can really get affected and we actually wanted to see what would happen to the real-time Communications when you get down into that point uh into the levels of actually having this man- inthe middle attack uh and see what what effect it had on a different deterministic Communications can you detect the man the midal attack using simple tools or do you really need a big monster IDS system or something that's homebuilt or something like that we wanted to see what what you could do and how easily you could find some of this stuff so what do I have in this box I have a controller so this is the this is

the controller the PLC here programmable logic controller I've got an IO block right here that uh controls these buttons so you hit the button the light BL uh the light turns on the light on these is independently controlled from the button push so the button push sends a signal to the PLC the PLC says turn on the light so it sends a signal back to turn on the light uh so you can actually do the man- inthe middle attack to actually adjust what happens with the lights in this box um I'm running basically I ran uh Eder cap in complete script Kitty mode I said plug in turn on I'm not an Eder cap expert I got a chance to play around

with it because it was fun uh but I really don't know enough and talk to other people if you want to know how to do Ed cap better um total script Kitty with this I captured traffic um this is one of the net gear switches has a mirror uh has a mirror Port uh and so we spanned the tapped or span the ports and basically captured off the mirror Port there so we had at least a clean capture of all the traffic on the network um the PLC to IO Communications uses what's called ethernet industrial protocol this is not ethernet IP as in TCP IP terrible name but it was what it is uh look it up um I

go I think I've got a little bit on that uh later uh we ran a 10 millisecond frequency which is a fairly standard for uh industrial Control Systems not too fast to where you're doing like really high-s speeed control uh and you really get down to the point of actually looking at like how you can synchronize multiple devices clocks and everything like that this is uh general purpose manufacturing uh autom manufacturers typically use 10 milliseconds uh air uh uh aircraft manufacturers like Boeing and and Euro uh uh Airbus they typically run all their controllers at 10 milliseconds so it's a fairly common uh frequency to run it's fast enough that we can get some good high quality

uh High Fidelity data too uh and basically we were running a Mana midle against the PLC itself so Target one in Eder cap was the PLC uh and not we weren't running it against the io block itself um so a little bit about ethern IP uh it was originally developed by a company called Rockwell Automation Alan Bradley um and it's now managed by a group called odva it's generally used at the lower levels so you won't see ether IP traffic up past sort of uh the controller level generally the controllers talk to something to some sort of historian or HMI system using another protocol uh and internet IP is typically between the PLC and the lower

level devices down below the uh um hmis the motors the iol blocks sensors all that kind of stuff uh odva or Ethan IP uses it's basically a common layer for and up protocol it uses the tcpip stack unmodified so they' figured out how to actually do determinance Communications over standard tcpip uh some of the industrial protocols have actually gone and removed portions of the TCP stack as they've gone along uh for different reasons um some of them get down into doing synchronizations in the Nan seconds when you're actually like running newspaper Mills and stuff like that where they're actually running a mile long 100 motors have to be synchronized at the few microseconds so they they've found that

they have to do it a different way uh using specialized protocols and they've actually removed moved even psses down to uh removing like the 802 link layer stuff to where they run just literally over the 802 uh fi level so uh but Ethan IP has done all they've made a point of stating from the very beginning that they wanted to run everything over um unmodified tcpip uh it uses both a command response type protocol so kind of like your uh a server client kind of stuff so give me your data here's my data give me your data here's my data uh and they also do a publish And subscribe where uh the devices kind of agree during their hand

shaking of how fast the data is going to come out and then it uh the io Block in this case just says here's my data here's my data here's my data here's my data here's my data and the other guys say okay that's cool I got it and they just sort of uh they use a handshake to basically make sure that the communication is still going on but they don't go into anything of actually having to ask every single time for that piece of data um if you look in wi shark you'll find the protocol in there it's uh in wire shark it's enp is the uh the four digigit or the four-letter kind of code

for it uh the command response uses TCP 44818 uh and this does what's it does two different types of messages one's called unconnected so this is the I want to start talking to you I don't know who you are yet so here's here's a message to get that started this is sort of the initial handshake process uh then they have what's called connected messaging that will actually use long-term TCP connections uh and sending commands and responses over that uh long-term long duration TCP connection uh with some sort of periodic rates but again most of the time it's it's uh that would be a uh like some sort of HMI system uh or um uh command response where someone is

actually having to actually a human is having to be in the operation typic uh it wouldn't be the actual low-level control stuff the low-l control is typically done with publ And subscribe and that uses UDP 2222 um it's considered real-time messaging because it's it's an implicit connection they agree during their handshaking process of what the message is going to look like how often they're going to send and then basically that initial communication handshaking goes away so the initial TCP command stuff goes away and then it's just a full UDP back and forth if they don't hear a UDP message from those devices within a certain amount of time they will actually close down the connection uh and then if it's

programmed that way it'll reestablish connection otherwise it'll go into some error state so um and this is how pretty much most uh of the real-time communication Works uh it can use either multi for the um publisher so in this case the io block itself that's publishing data uh it can use a multi protocol or it can do udic cast uh they did they did multicast originally because they wanted to have the idea of having multiple devices be able to subscribe to that um and this guy would only have to produce one packet to that everyone would sort of listen into so um what do I actually have here I kind of went through this a little bit

already um PLC iio block uh Netgear um my had the GS 108e so it's got a it's got the ability to do mirror port on it um and it's gigabit um I ran uh man ofe middle man of the middle machine was C Linux 2.0 VM uh and I just used the default C version of Eder cap that was in there so um and then the capture machine was Cali Linux native uh and whatever their native version of wire shark typically I actually recompile wire shark myself just because it's a lot better better that way but uh in this case I just did a dumb completely a new wipe of the system and and just reloaded it by

itself just to have a clean system that I didn't have to mess with or basically prove out so uh I did erer cap uh with Target one being the PLC itself that was the main target of the P of the man of the middle attack uh and Target Two was the io block um and basically I just ran uh AR poisoning with this uh with the sniff remote connections just cuz it was a little bit easier to do that uh since the networks are extremely small things like the icmp uh uh I forget what the some of the other things but basically it's like icmp there was no Gateway I could go to like spoof out the Gateway for these

connections so since this is so small uh and this switch really wasn't doing a whole lot of layer three routing or anything like that art poisoning was the only one that really made sense since it was all physically on the same switch it wasn't have to trans transmit over uh routes or anything um and just to let you know I actually did run all of this uh stuff through uh through virus total Network Miner and bro like all the actual stuff that I actually ran against this uh system I ran those through and none of them detected the attacks or at least alerted like there was some really bad thing going on it noticed some new traffic it

noticed some icmp traffic that came through for the for the art poisoning itself but that was it there was nothing extra special that it really uh alerted on so these are uh pretty stealthy if you actually can go through them it uh they are working on it I know I've actually worked with and he actually wanted to try and play around with one of these boxes um but he left DC before he could get a chance to do it um so but yeah he he wants to play around and actually learn more about ethany I know that they do have something in there for it but I don't know to what extent they've got stuff

so um I did actually run a filtered attack just to see if I could actually adjust the data that was going through here and and turn lights on and off uh without um without uh the system really realizing what was going on so inside the because it uses UDP um it's a connection it's connectionless so what they do is they handle the connection oriented stuff at the upper levels at the actual layer 7even application layer so they have in embedded in the real-time communication they use a connection ID a sequence number uh and then the data is a separate field um so in order to actually spoof out the signals what I did is I Advanced the sequence number by

five uh to basically make my my signal more relevant and it actually basically said okay the PLC signal that was that the sequence number minus5 is old data so that doesn't matter anymore so it just said screw it you're you're done um and it took my man- in the-middle data as preferred data that Point yep exactly um and and there's supposed to be some protocol stuff to handle that where it's only allowed to handle one sequence number change at a time um but I chose five in this case to make it obvious but you can actually I I did it two where I just Advanced at one sequence number and that worked just as well so I only did it for five here just

to make sure that it was actually noticeable in the data itself uh and then what I did is I actually wanted to see what would happen to the data itself uh if I adjusted things so I adjusted the data signals for these buttons uh coming the commanded data what I did is I the command from the PLC to the io block I actually adjusted I added four to that which uh you'll see at the end basically just shifts over which button it thinks is pushed so um it actually it when it thought it was open and not pushed it actually looked like it was pushed from the PLC said go ahead and light that light so and I put this up here on the

screen I'm actually checking to find out I have to check with my employer to find out if I'm allowed to release these since it is actually an attack against an industrial protocol I don't know if I'm going to be legally allowed to do that but or if our lawyers will just throw a fit so um and I've got to basically I I'll I'll find out and let people know on Twitter if that's going to happen so um so this is actually the series of tests that I ran I ran one series with the multicast which this is actually the more common way that ethernet IP is run uh with multicast uh publisher data and I ran a baseline I actually rent through

and and did the button pushes just to see what the different data comes out as ran the middle attack uh here I did man the middle attack with the button pushes ran the filter and then reran with the the button pushes and then I ran the whole series again with uh unicast data as well so uh and this is another one of these things I got to check with my employer whether uh I'm allowed to post these capture files I think the the wire shark captures I might be able to post I'm not sure about the filter itself but it it I mean this was this is the filter here it's like four lines five lines of

code so but I just got to check whether I'm allowed to post that up on the Internet um so here's the actual like detailed information uh I had the MAC address uh for the PLC the IP address Mac address IP address um the PLC to IO block was running 10 milliseconds uh cyclic frequency uh and this is a unicast connection here so this is always saying I'm talking to you directly over unicast and then the iock publisher of the data was using um did both multicast uh so the multicast was going out to this IP address at that 10 frequency um unicast of course goes directly to it and then that's the MAC address of my VMware man in the middle

station so when you actually get into looking at stuff um here is just again so for the Baseline test uh this connection here was unicast uh and then this was either multicast or unicast depending on the test that I was running um and so let me explain a little bit about what this is uh what you're actually looking at is the cyclic frequency of these signals so this is actually 10 milliseconds right here uh and what you see is between about 10 this is 10.4 here and this is 9.8 So within uh about 400 micr seconds distribution you get these signals all going um basically at approximately 10 milliseconds every 10 milliseconds you've got your spacing on on what the

data is uh and then this is the this is the PLC communication to the io block itself a little bit wider distribution but again uh this is 10 this is 10.4 10.6 so again you've got about maybe 500 microsc Distribution on here um and so these are fairly deterministic uh this is well within 10% of the mean uh of the signal so this is this is pretty well deterministic on your signals so now what happens when you actually run the man the middle attack um and so for the multicast test because this signal was multicast coming back the man in the- Middle attack did not actually do it because it was going after IP address to IP address and since

this was actually going out over multicast it never actually was affected by the uh Mana midle attack itself the only one that was was the signal between the PLC and the io block so when you actually look at it you'll see two packets from the plc effectively from the PLC to the io block but when you actually start looking at the ethernet packet here um like in this particular packet this one went from the PLC to the VMware station the next one of course went from the V the VMware station to the io block itself um and then as I show here down here there's the sequence number and the data um from this I was not running the

filter at this time I've got another one later that actually shows the filter but you can actually see in the sequence numbers uh and then again in in the uh ethernet IP like version here you've got a connection ID and the sequence number and so you can actually watch it moving up as you go along so with the multicast test basically there's no difference between what the PLC is sending out and what the io block is sending out they don't even notice that a man on the middle attacks going on at all they're pretty dumb with this they don't even know anything's going on but then what happens when you actually look at so this is looking at just the IP address

of the um the PLC to the IP address of the io block and then so obviously this is where I was running the man in the- Middle attack here um this is actually combined it's seeing both those sets of packets so looking back at this it's actually doing uh it's actually just analyzing both these sets of packets in here so this isn't really what you want to look at this is what you want to look at so this is what the actual um IO block is seeing that's directed at its Mac so here you've got the PLC signal and then you actually launch the man in the middle attack and you notice the man in the middle attack

itself doesn't okay so it doesn't affect the mean of the signal at all but there is a noticeable difference in the distribution of the attack from the PLC to the io block where this actual man- in-the-middle attack was going on uh but again most of these devices they take averages so as long as the average number of packets Remains the Same they don't actually record anything about the um they don't record anything about the uh the packet distributions or anything like that internally this is the kind of stuff you can measure yourself though you can go in and just run wire shark captures take wire shark um and Export to whatever you want uh the I actually

did a this is using my specialized analysis software but you can take and throw this into a CSV file and plot it in Excel if you want this is actually not that hard to do in Excel uh basically you isolate the you isolate the stream you want and then it comes out as a nice straight long number you can post the data and actually figure out what the communications is on this all this is is just literally the time during the test over the time between subsequent packets so it's a Delta time versus time so it's not that hard to do I've got actually a little bit more information on that from a talk I did at

bsides DC last month I think it was so if you want that and it's it's not not that hard to do so now running unicast the man of the middle attack actually did do the reverse Communications and now you get something very similar so in here you've got now two packets going for each one uh that you initially had uh from the from the devices and again the PLC itself and the io block itself didn't care one way or the other that the Man hle attack was going on they actually didn't adjust their timing at all but then like the first time this is actually what it looks like just doing the IP address and in this particular

case the Herring bone patterns from what I've seen a lot of times has to do with clock skew between the two different devices the man of the middle the man of the middle PC was just a slightly clock skewed so it it drifted differently than the uh device so that's where you get these weird patterns in here but again this isn't really what you want to look at this is what you want to look at and so this is actually uh looking at the man ofthe middle attack here uh this is the the standard PLC to IO block Communications or this is actually sorry this is the io block coming out and again the mean Remains the Same so in

general you actually wouldn't no statistics would happen in this system that would tell you anything was going on this is the kind of thing that you can capture yourself on a monitor Port uh and do some form of uh of analysis NSM basically with very cheap tools a laptop wire shark and Excel you can do this stuff

yourself yes yes exactly and that the kind of thing that uh this is what you would do periodically testing so um this is a huge amount of data to try and go through in real time um but it's the kind of thing that if you had to do a periodic yearly annual assessment you go through you go to each you go to different places around your network you capture the traffic and you spend a week or two analyzing it coming up with a signature of what your system has in it to begin with and during your factory acceptance testing your site acceptance testing your commissioning basically all during that time where you can get clean

gold standard versions of What should be on your systems at any one time and then you can come back and start comparing them and look for these different signatures of oh something something is different than what it used to be H maybe I should go and investigate and then then you go in and do second tier third tier analysis of what's going on but just looking at what happened now versus what happened before you can really get a lot of information out of your systems so um now I did actually run the filter just to see what would happen uh and I can show you this downstairs too a little bit more um but uh so we we

really weren't looking to see what effect filter really wasn't very difficult as you saw it was a few lines of code it really didn't do very much but it was something to see if the filter actually had any effect on the performance data did did running a filter here on this actually move these lines at all because of the additional processing that had to go on in the man inthe middle attack itself well basically we did the we did everything here and it actually didn't affect the performance at all it was pretty much the same thing uh what I did um just to lay it out here so I actually went through and and ran a whole series of

tests to one went through the whole process I only put in a few of the buttons here but I've got the full set of like two of them and then three of them and did a whole series of that kind of stuff that I've got recorded and basically what what you can see here is that this is the signal that was originally here and all it did is just add four to the data effectively what that did is that shifted the bit pattern over so one of the buttons acted opposite of the way it should so um I think in this case it was this second button that was uh that was uh basically bit number four it would turn off when

it was supposed to turn on and it was on when it was supposed to be off so pretty simple instead of it was basically on now and then it would shut off when I would hit hit the button but effectively it showed that you could actually do this with relatively simple kinds of things and as I said I I learned how to use Eder cap in about like half an hour uh and found iron Geeks tutorial and wrote a filter in about another half an hour and ran a man in the middle attack so it's not that hard to do especially against industrial control systems that I mean the stuff exists in wire shark you can go and figure it out and then

adjust what you need to do um and then here you can actually see so uh where is it so this this packet right here these two uh this was 2542 uh it became 2547 and then also the data in this particular case was 40 and it became 44 that's all I did uh by advancing the sequence number I basically took control by being newer than the than the previous data so it disregarded what was going on um the captures as I said I'm going to try and probably uh post them up on our GitHub but I got to check with my employer first uh I will post stuff out to Twitter after I find out yes or no

whether I can do that or not uh and this is my contact information so uh Jim Gilson my Twitter and uh so that's it there any questions yeah

extremely so yeah the guys that actually did stuck net not that people know who did that or anything they had very detailed information about exactly the systems exactly the data that was being contained in there uh and and they didn't actually run a man- ofe metal attack against disc commmunications they ran it against the engineering workstation which programmed this and so they they adjusted the engineering workstation to actually cause even worse things I just did one simple thing but they actually added a lot of extra math to go and actually like vibrate it more uh and and do some really nasty stuff to it so but they had very detailed information about the very specific

protocols and uh data sets and everything that go along with

that Yes actually ethernet IP is in the process of of actually building in security to their system since they use a default tcpip stack they're actually putting in SSL TLS uh and certificates and the whole deal uh some of the other protocols uh dmp3 has uh has actual uh security built into its connection um but there are very few that have security these systems were never designed with security in mind they were designed to get a job done ethernet IP is a protocol was written in it was published by odva in 2001 it was uh built by Rockwell before that uh in the late 90s but again it was actually based on a protocol that was

written for thick n pipe uh inside a plant that you couldn't physically get on to because you didn't have the uh the actual connection to tap into it so the protocol was based on really old stuff uh field bus technology that was never designed to have security built in very little closed stuff where literally if you if you didn't have your Terminators screwed in tight enough you would throw the whole system off so this is a massive sh it's a absolutely some Protocols are um other Protocols are not so things like uh profinet which is a European Standard they've basically decided that um they don't need security because they're now literally getting rid of parts of the tcpip stack and when you

get down into the real time what they call uh industrial realtime uh where they're literally synchronizing Motors at microseconds if things aren't plugged in just right you're going to throw the whole system off so again they're getting down into where they basically modeled a field bus over uh STP cables

um yeah well okay so this test uh this particular test was around 70 seconds of data and this [Music] was 7,000 points so I mean not not a huge amount but the capture files are already just for that 60 seconds and this is one device talking to one other device that's not when you actually consider a full plant when you start considering a full plant then you're getting into gigabytes of data per day that you have to go through an analyze and analyze really down deep not really because most of the time the real-time traffic does not leave the local switch so the real-time traffic uh in in a lot of these cases they're very

um cell oriented so so this PLC will talk to this IO block and this screen this HMI all plugged into the same switch and unless there's some specific amount of information that needs to go out of that switch nothing ever leaves that switch you need to tap into it and that's where things like industrial IDs systems they're they've tried to like really push them down lower into the architecture but the problem is you're now talking having to throw a 1500 to $3,000 box at every single work sell just to do analysis and then you've got to build out the infrastructure to talk from those guys to some sort of supervisory IDs and you really get into

problems yes and if you're dealing with uh Medical Systems Food and Drug uh any change to the system at all requires recertification of the entire line so not just recertification of this one cell you've now got to recertify an entire 50 to 100 work cell line for simply doing AV updates so companies don't want to do that how do they do security today huge levels of isolation and segregation generally they basically disconnect it as much as possible and don't touch it if they're into that point of where they're doing nuclear regulatory certification or um uh Food and Drug certification things like that most of the times those systems are install it and forklift upgrade it in 25

years so that's those systems are me to go yes yeah literally like their downtimes their their uptime downtime is in 7 n's reliability they they just don't shut down

so again CIS log will give you averages it won't give you the detailed fingerprinting kind of stuff that you that you really need to determine what's going on yeah averages will give you some things and you can tell oh yeah all of a sudden we're get a Dos from something over here but I can tell you most of the time when you're getting dos stuff in the industrial world it's because the Nick card on the HMI went down and not because you're getting a real actual external dos attack or the guy with the forklift ran over the power wire or network cable or the fiber that was hanging across here all of a sudden started to go down uh got hit by the guy

running the stuff across the way or you turned on a motor on the other side of the plant and the Emi translated over the wire and now you've got uh now you've got a problem with that

so yeah 99 whenever we've run into things I've only run into one active malware uh inst when I already did an assessment and even that malware was questionable it was it showed up as an attack in virus total and a couple other places but when you really looked at it it it looked like it was actually an an old industrial device that was using a proprietary protocol and doing something it wasn't supposed to do but we couldn't verify that so we had to at least flag it as malware so

of course there is stuff out there um the companies that I have gone into and assessed that's not the problems that they have so um and and if you look at hav havex was not and that was nothing to do with the industrial control system itself that was a rat on the vendor site so they went and basically did a a rogue version of the firmware for devices on the on the actual uh vendor site and so when people downloaded a new version they downloaded The Firm the the V virus from there it wasn't actually an attack on the industrial control system

itself yeah again it's one of these that it exists it's out there people found it um but you would yeah the likelihood of actually seeing that on a system you're assessing is extremely rare because of the simply the fact that these systems don't change and if they do somebody did something they're not supposed to do

so um it's actually a lot easier because you you can go through and fingerprint your systems you can actually load whitelisting on a system in an industrial control system and get really good results at it because those systems even the HMI systems and the controllers and all this kind of stuff that are PC based they don't change but once a year typically so you only have to redo that white list once every year instead of every time Microsoft decides to ship out an update or every time there's an adobe failure or things like that so so white listing actually works really well for PC based systems these systems here um typically they don't even bother to try

and fix it and determine what really went on forensically on here they will just say I need to get this system back up and running now I have this copy of my program that was running on here and this firmware I'm going to load it because it'll take me 10 minutes to do that whereas it would take me days to rip this out and put a new one in and forensically analyze what's going on in here and all this kind of stuff so it's mostly about dollars of getting production back online so so as I said um I will be I have three of these systems with me I actually put them on the table

downstairs just before lunch if you want to uh feel free to play around with it uh I've got Port one set up on each one of these switches as a mirror Port so you can go and and see what's on there um I would ask you uh don't be a dick and reprogram my devices um I can I have the programs with me to fix them but I would basically just ask you not to yeah exactly so thank you [Applause]