GT - Improving Security by Avoiding Traffic and Still Get What You Want in Data Transfers - Art Conk

ons festivals testes testes 123 ok welcome in my day job I'm a college professor and my night job I'm still a college professor you can ask my wife and back that's all I seem to do anymore I don't get to play the cool things anymore used to play a long time ago that is her Lotus for those who are looking so now it's important everybody has to put up these sorts of slides these disclaimers these days and the bottom lines only thing is yes I am a doctor I've got two Doctorate degrees but I'm not your doctor and so I don't want to see your rash however if you want to show it off to after the Q&A

session over that part of the room so I'm going to whip through this because this is written for slightly what I'll say less informed audience I wasn't certain how informed but i'm watching the questions all day and the faces i know some of the people in the room so i'm going to talk about a problem that i'm trying to solve and that is all bad stuff comes in over the network over your network traffic and so if we're talking industrial control systems yet input skate in the title sorry you don't want to let bad stuff so now what do we do we isolate it and so we're going to look at how we got there how we want you

know what do we want to do today because the world is not like it was 30 years ago and so the problem i'm addressing is we have a lot of really important infrastructure stuff things we paid literally billions of dollars for and it's all over our cities everything runs on this infrastructure and now we want to connect it to god forbid the corporate enterprise network and then somebody came up with this idea let's use the internet because the Internet is free and okay technically well okay maybe it comes with costs and now we seem to be having problems and so the question is how do we address these problems one of the trends ants created

this problem is high connectivity somebody decides you got it we can connect it let's connect them all together see what happens and the Internet's magical it really can connect and can carry virtually any traffic you want now if we didn't vent a flag in the IP protocol it says this is an evil packet and then make a rule everybody had to if you have evil package you flip the flag life would be better we don't have that rule hence what happens is bad packets cause problems or control systems which means movies like die hard for can be made because can you do that stuff probably not as sexy as in the movies but yes you can seriously mess up

control systems as I was mentioning the beginning talking with people nerve I get my students PLC's to play with and about one a semester comes back with no more blinky lights it's dead dead and it's not because it connected 110 up to a 5 volt input it's just these things don't like certain things that happen to them and then they go brain dead so let's look in reality world well engineers want reliability that's all they care about if it worked yesterday I want it to work today sort of work tomorrow and you say well what about they don't care we're not changing anything patch it like it's working the business wants more integration I come

from Houston Texas it's the land of oil and gas a lot of those companies are built by people that were once called wildcatters and believe it or not CEOs sit in their office and monitor individual drilling rigs why that's what they grew up doing and so they want to have a live feed from a drill rig as its drilling you know really but that's the way this stuff were in today's technology we can do that should we do that different question but he's a CEO isolation is how we've always relied upon this okay isolation though unfortunately fail because air gaps are not isolation air gaps are merely temporal gaps means no words for a certain amount of time it's not

connected but sooner or later you're going to connect or information is going to come or go and so in today's world where I want data coming out and want logs I want to take updates in all these sorts of things how do i isolate it so putting on the comp sci thinking cap although I'm not a computer scientists I'm a technologist there are ways we can isolate it because we don't need to isolate it from legitimate commands just all that other traffic and so I always get this question when I present this commercially or aisaka I so meetings things like that well VPNs are great they solve all the problems you should just put a VPN around it and I have to

remind people that when you connect a machine via the VPN you get everything from that machine not just what you wanted everything and I always just remind people and I hear that words why not just let the engineer then from the outside sales company walk right up to your rig and plug their machine into your protected Network segment we couldn't do that we have use of VPN does that mean excuse me sorry so just like we have this rule of no us bees out in the plant floor things like that because USBS carry evil things no VPNs either really we'll use VPN but for very selected things because we will I mean at least people in the engineering world

it get this understand that VPNs are not security devices in this respect to the machine you're adding into the VPN little language here because every field has its own language and we're leaving IT land and moving into this process control land they use two terms zones and conduits zone is just a group of elements be it machines operators it's defined by a border that says this is what's in the zone this is what's out of the zone and typically a zone is what you would think of as one security profile so everything inside of it is going to have the same basic security rules so this way i can manage the security rules to everything in one

group of things called a zone by enforcing it at a condo because the conduit is the only way you're in or out of his own so we do these same things in networking we just don't label them this way but the labeling is kind of important in the ICS space because they really do as engineers trying not to connect everything to everything because again they want it to work so the conduits where you place your security controls nothing comes or goes out of the zone without going through the conduit therefore we'll just check it there classy IT solution firewall let's make it modern next-generation firewall how useful those have been in the IT world we even have ICS firewalls their

companies that that's what they specialize in they specialize only in firewalls that really understand these special protocols guess what same problem occurs the problem isn't in the special protocols the problem is in all that other traffic and people find ways hackers and evil actors find ways around things like firewalls so for high impact or high-value zones like think nuclear reactor we don't allow these devices just not allowed let's look at why and how we want to change this firewalls work at the network layer because they manage your traffic and they try to define a set of good traffic versus evil traffic allowed versus not allowed and so they look at traffic and say should this go across but then we think about

problems like heart bleed that came along and will hardly cross it a firewall and the answer is yes there's a lot of types of traffic that will cross firewalls because you've allowed that protocol that it's carried on and so really bottom line the underlying problem is this thing called traffic we allow way too much stuff in and out but we don't need so bottom line we can go around all the tools of IT security if somebody here thinks they've got a bulletproof IT security solution that nobody gets around trust me you wouldn't be sitting here right now be people carrying your money downstairs so I started thinking about this along with some friends and looked at some

solutions that are out there they're called unidirectional gateways and that's what we use in the nuclear space they only allow traffic to go one direction there's no feedback channel and I started thinking okay this is how they work and I said well what we really need to pass is just the information i don't need to pass traffic because if you think about a database replication solution it only passes specialized database traffic between one database server or another it doesnt sit there make a new packet and send the same sequel commands it just sends update this block to this that's all it needs to know it doesn't even have to know a table what user any of that it just

replicates the data changes so how can we do the same thing in this place okay how can we move information on internet protocol there are lots of ways we did it long before a matter of fact it was talked earlier about serial buses so we still use serial buses in this world and so your typical system right now looks somewhat like this you have a control system on one side and it's got all these fancy control system terms you know the zone hm I human machine interface historian plc engineering work station etc not that important to us it's this it's the really important stuff that I don't anybody to touch and then goods across network to my business

enterprise oh and so this I don't care how many appliances you put in the middle it doesn't work an IT and it allows as I put it network carries data plus one yeah we always want that coveted plus one at the party you don't want plus one in your systems when it's talking about information coming in you only want what you asked for not that extra guests and so when I think about database replication as i mentioned sequel requests come into a database server but then it just tells its backup server its replicant hey this block of data on the drive changes that's it it doesn't have to go through all the other stuff the primary database did all that

resolution hence you can't just jump in the middle of it and really make a difference because you not one that meaningful because you don't have a clue as to what the different blocks are and how they move so what I want to do is I want to move state what is state in a control system like let's take this room if the room was wired with a control system we have these lights the lights that have two positions on and off okay there's AC in the room it might be a temperature which would be a number or just a simple on or off and so state is all these different parameters and the cool part about plcs is they're really

simple in the end they only do two things turn something on or turn something off and they also measure something they have a way of taking it analog measurement turning into digital give me the answer all your logic that's magic but you know what very simple devices you'll turn the lights halfway on switches are on or off so if I want to move all the information on my plant do I really need to packet eyes it put it in all these different protocols and make it so go across a whole internet well the answer is if I want to get into the CEOs desktop yeah probably i do because that's how we move the traffic

but now how do i scrub all the other traffic out and i'm going to do it by putting something in the middle called the state transfer device two devices one on each side does not have to be far apart where in fact one of the arguments I just have one of the vendors of the international gateway is why can't I put these both in the same box and just have to network connections one going in one coming out on one side is a gateway interface what it is it's your standard computer interface it can speak all the different protocols it's fully networked aware but it's also aware because you programmed it with exactly what states

you're interested in transferring how many switches how many measurements all those things and so it translates from your protocols coming in it looks at the skate of protocols as Oh skater protocol this I can get a state value out of I can transfer a state and the same thing on the other side is a duplicate box on the other side hence the only thing that's transferring between these two boxes is the state of a given switch a given measurement a given coil whatever so what I'm trying to do now is basically block it's going to block all over there traffic it just drops it a bit bucket there's no place to send it because the protocol between

the two is very very simple it's just the data that needs to move now if I want to actually do this it does require a little bit of planning but the interesting part is it doesn't require an architectural change because it's just a bump on the wire however it does require that you have a little more knowledge let's take a look at that you have to define what information you want to transfer in and out because if you forget and leave off a crucial switch it's not coming in or out okay and so you have to go back and look and see how am I going to architect my zones what pieces of information need to be moved

and then i can just slop it in so let's define our needs what really needs to move in and out of a SCADA system or control system well I have data that's kind of like the number one thing that's why these things exist and they come from a source called the historian you gather your day to you stick it in is called a database for lack of many of the terms although people make the program's give them special names called historians you're storing your data into database well I want to replicate that out no problem how about log data from all the stuff you know log files from this that sure syslog we always want

that how about coming in do I need to update things yep how about updates to my plcs yep how about admin updates to things like my active dry three replicate that's in the control room dns servers things like that yep now before you get too excited those are really stripped down versions they only replicate over the part that works in that tiny little part of the world because in a control system at any given time you have to assume that like my plant just blew up and the rest of the world does zombified but I still want to make whatever my refineries making and so I need my own little replicant of everything in the control room and so

the control system network that tâches to all the devices is highly redundant highly secured and lives in a bomb-proof room such a way that as long as you quit was still working it'll work but then this is a question I always get when I'm talking about this to people in the industrial control section well what about our HTTP you know we play World of Warcraft at night they won't say that out loud but you'll find it on their machines you know and it's like I know you shouldn't have that on your machines what about male we always like checking our mail in the middle the night because we catch up on the sup know all those

other useful things I'll give you a solution it's a simple solution it's cheap and it'll solve all sorts of problems for you so what I have to do when I operationalize this is I don't have just one interface I end up with a bunch of interfaces because for instance for a given set of controllers they're going to have states and maybe the data is going out so my historian maybe one set of traffic going out okay a different set of traffic going out or say the logs I may need to move other files out that are specific types of files okay and so in essence the data that goes out I have to be able to

define what it looks like so I can do a state transfer same thing for data coming in I have to define those channels so notice all the channels are unidirectional and it looks like a bi-directional gateway but really it looks like bi-directional network traffic but it really isn't okay data goes out data doesn't in etc so you have to sit down and define what all those mean because they're programmed into both sides of the gateways now remember all that stuff that's really important like roller Warcraft the middle of the night checking email actually believe it or not I mean I spend a lot of time control rooms and refinery sector the number one thing they have one machine dedicated to

is the weather go figure whether it's kind of important to those guys so it just they subscribe to a weather service they get better weather forecasting we do from the stuff on the TV news they pay for it but it's an HTTP connection stick a standalone network computer outside the zone of the important stuff therefore if you decide to go fix your banking or you decide to go play World of Warcraft or you're looking at I don't care where you go there's no wiring hole attack that's going to affect your critical stuff of your good zone and in today's networking how hard is it to have two machines sitting side-by-side on completely different networks really

and the other part about it is the machines that I want to run all the special control system stuff on are usually things like XP and lock down to a very old version because the vendors are like oh well we have validated anything any of these newer things hence it really gives you a crappy experience on some of this other stuff IE six is great for some old control systems not so much something you want to put on the web today hence you isolate you just separate the two by the way that engineering work station sits in a separate zone what are those costs today couple hundred bucks and they remove how many headaches you know just one problem

a year pays for that workstation so in the end this bump on a wire and I'm using the same term unidirectional gateway that the standards used what it does is it provides a separation of everything inside this control system zone from the outside world it is a complete physical separation the only traffic it that it allows are predefined states that you've sat down and decided this switch gets to transmit in or out you may not even want to move all your data out you may decide only pieces of your data need to really go out hence we stop passing traffic now so what will get to the so wet in a minute it actually does and we've done some of

this in the lab and it's got some really useful interesting effects it's not a data diode solution this is one thing I want to bring up you'll hear about this a lot of times all we use data diodes all dated diodes really are are unidirectional network traffic devices yeah well let the traffic come in from these these places that's a trusted server will let that one communicate in so what's the first thing a good red team er does own that trusted server because then it's so much easier to get in and so dated diodes don't check traffic per se and even the ones that now there starts it will put a next-generation firewall on it that'll

solve it you know I'm not a buyer of this technology anymore because once you're past it too many things can go wrong so the bottom line is I want a gateway that understands the state being trapped being transferred not led trafficking because I can't really look to see if that's a good packet or a bad packet that system has failed us now why is this secure it breaks the attack surface pathway to your systems from the outside so if you go look at all the current attacks take the heartbleed attack it could clobber one of the clients and knock down one of their gateways all your attacks can do is hit that client and since that machine is a

separate physical machine from the other side there's not one processor running two programs two processors two separate program you know computers in essence built in there can't cross you can break the outside Raspberry Pi but the one doesn't care because it just says oh I lost state no more states coming in so you take heart bleed you can't cross the latest TLS hack same problem buffer overflows I don't care buffer overflow it to death it doesn't matter you're not going anywhere we hear that machine ooh let's get really exciting a race condition doesn't matter injections droppers drop something on the box so you can connect and do something evil to that raspberry pi and I built one of

these using raspberry PI's for instance Oh guess what it doesn't matter it only affects the outside piece can't get to the other side and why do we want to do this talk to the engineers reliability reliability reliability we're not letting anything in anything that wants to come in has to be previously defined across the states okay and the reason being is any failure to come with comes across reduces their reliability and constant problems now I've had numerous people say my ICS systems aren't really targets know there are other things their targets and my answer that is guess what just consider this the future that you know you haven't figured it out yet because there are two classes of

people running systems those that know they've been hacked and those that don't know they've been hacked so we'll put you in that second group right now so consider for the future when you know you've already been hacked these systems are target's sometimes by malicious actors sometimes by accidental actors call it what you want attribute what you want we're seeing the traffic all the time when you go inside and look at the different systems still little science because I am ultimately I have to lecture i'm a college professor why does this improve your security it reduces your attack surface okay it reduces the system control surface I'm not trying to control all these other pieces of

traffic that I don't need okay why because it tax are carried by the traffic the data is carried by traffic therefore i get a problem if i want the data but not the other traffic so what did i do i separated those two state can carry an attack but it's much much harder to do matter of fact by and large most these systems don't allow outside commands in they only pull data out for a variety of call it simplistic technical reasons but the guys seeing in the control room has to have the ability of control system now science not a lot well actually there is a scientific law that works on this and it's called the law of

requisite variety and it's violated by every major security vendor all the time and this is a long-standing figured out back in the 60s from systems engineering that says a system under control has a regulator that controls how it works and if that regulator is going to control it that regulators complexity has to be at least as complex as the states that are being controlled so simply put if I want to put a box on the edge of my network that can control that can tell me good or bad for everything coming in understands everything I could ever want inside my network the complexity of that box is the same complexity as my network hence the idea that I can go buy a box

that could screen the whole internet for me the complexity has to be of the internet hmm yeah it's not really an infinite number of states but it's pretty close it's just not controllable Matt for what did I just do in my system though I took all that stuff and threw it away my system is now defined by a very limited number of control states coming in and going out my control space is much smaller hence my ability to control it goes way up so for instance we always talk about doing things simple this is as simple as it gets because the traffic in the sensitive area by definition will be clean because it will be traffic

generated between these devices without regard to the outside world so technically everything is going to work the same way it always worked on the inside it's just traffic gapped from the outside so let's look what happens with 90s we just had a previous session here about IDS's things like that think how really cool your IDs would be if suddenly it says oh look HTTP traffic HTTPS traffic alert any non allowed traffic alert why it can't be a false alarm that traffic doesn't belong on the inside there's no mechanism to get it there unless somebody cross connects the network see an address its new my definition has to be wrong because we don't allow traffic from outside this

similar to where I live out in the country if you don't belong on my neighborhood you get stopped they know who lives out in the country local sheriff is really good about you but you're not from around here it works the problem is try the same thing on the edges I work at a university you can imagine what the network traffic looks like coming in and out of the university all those students so where does this belong if I was a Salesman representing company i tell you everywhere put it everywhere but I used to be an IT guy and I'm going to tell you this is going to be a pain and about to put in some

places and doesn't work at all in others but if I have a high value high risk system this will allow me to isolate if I have XP machines that I am dependent upon and I can't move them because the vendor you put this around them and only allow in or out what needs to go to those machines guess what happens people can't exploit your XP machine because it can't get to it but yet you can still get what you need in and out the states so it's not really limited to ICS what if targeted actually isolated their point of sale system for all the rest of their network you know they're wondering about that question right now will this work for

target probably not for a variety of other reasons we're not to that level sophistication yet could it should work great for point of sale system I mean how much information they really need to be transferring in or out of all those nice little boxes there are vendors out there in this space they build unidirectional gateways for nuclear power plants and other places but again I'm not sitting here say hey let's go to these vendors by a tool and or buy a product put in our system it solves the problem now I'm saying it's time to start thinking differently about your traffic and saying do I need to allow traffic through this area do I need to

allow traffic into this neighborhood and I need to start thinking about how do I design these systems putting security in mind of what needs to come or go because then this is something more reliable

[Music]

[Music]

[Music]

okay target is a good example because their point of sale system can be consider very similar to any modern control system I have registers they're running some sort of operating system it's probably a forum windows PE they've got an app they bought from XYZ corp that you know scans gets the prices goes against the database well guess what these things work the same way that thing called a historian is a database it's a local database of all things have been happening my control system and so in there can be my prices and there can be everything loaded that makes my point of sale work and so I now have I'm thinking this like ATMs because I'm

formerly you store it for financial services firm and believe it or not there's still a lot of ATMs running OS too why because they worked yesterday they're paid for and they're actually are on an isolated network for other reasons because it's too expensive to move their network but so all the things you want to move in or out really become very very small subset there really is data that comes in and goes out sales data things like that but that's a replicating of database server which is actually pretty easy to do in this model it's actually probably the easiest part of it the harder part is actually so I need to update my point of sale

terminals with a new update I have to whitelist things in white list things out and that takes a little bit of planning on exactly what data crosses but literally all you have to do is decide here's the update and you walk to the you have to manually go across the air gap to the inside and program hey this program is coming across and here is its signature hence it allows that piece of data in and it matches a hash the hash is what controls what goes across the state so you can move programs in things like that the problem is is thinking of these things as general purpose computers and other things and you want to remove all that

thinking and make an isolated environment and so when you look at it it's actually a lot simpler than a lot of people believe on making those things run number one rule if you're a small business put out by ABA probably three or four years ago if you're a small business and you do your electronic banking use a separate machine dedicated for electronic banking that you don't use for anything else why it's necessary because if you are I go to our bank and we're hacked what does the bank do well we're insured we're covered if your small business they say we're sorry for your loss all those nice insurance things that we have for credit cards

things like that do not apply to businesses okay so yes small business get hit all the time they went to the isolation route years ago banking but you can you can have stuff leave the inside of these networks the problem with the target hit is the malware SAT where on the inside can't get in under these situations that's the basic premise here whitelisting becomes very easy under these circumstances you can whitelist every box in here completely and easily because you know exactly the only things are supposed to be running

okay um I get hits it real easily by looking at first all your what we'll call control system protocols are very easy when you start looking at a look at my Wireshark your little binary strings and it's like the device code the switch code the value and so hence you can pre-compute all those and say these only things go across that one's easy moving things like data out of a database server again because the way David Lee replication works that's pretty easy last one r2 but last one that one I thought was going to be toughest but actually turn out to be pretty easy green updates two things in that requires a sneakernet solution you have

to have another channel to tell the other side this is the legal hash coming in and it cannot come on channel because that would allow a path for someone to break it okay so if you have another method so in other words Stuxnet was successful because of humans okay it passed to air gaps because of humans so you can still beat this with the air gap part with you know that operator can bring in and say yes put it in this piece of malware because here's a signature and it would go across then legally the hard one or log files not really crack that nut yet because log files are different every time and so

you're letting information come out that you have no you know I can't define it state really so that one is one we're still trying to figure out a convenient way other than just allowing alpha text out there is no protocol layering in the middle per se ultimately if you ever put this into production you should probably use two different FPGAs and you have a series of 64 connections across for a 64-bit and you'd have a state model on each side send it across if it matches the state model you accept it if it doesn't you reject it very similar to how actually the incoming part of a cisco router or switch works it just brings it into a long register and then

process it all at once it doesn't read it into a computer memory piece by piece by piece by piece and then build it I mean it literally has a little register in there when it brings in the bite so in just right you don't need it all those stacks yes all those stacks exist for the convenience of being able to pass anything to anywhere and that's a good thing works great don't need that

yes yep no this is not near ready to go primetime exactly I mean until you if you want to put this into the real world you're going to have to have a real company that has real backing that somebody's going to say I'm going to trust that company to be around for umpteen years but the i'm not trying to sell the product i'm trying to sell the idea of we get enamored with traffic we don't need to be enamored with traffic we need to be enamored with communicate only what we need to communicate and that's it harris county r harris county is a huge county however each little place the local deputies drive around our neighborhoods pay attention to the

kids they know who's supposed to be there her lizard I was driving to work one day and it was a guy that was slumped in his car didn't belong in a neighborhood and I took one look went around the corner called and said hey you need to come look at this guy went to the gas station turned around came back home because I didn't you know this guy didn't belong in her neighborhood so I'm in the house a couple minutes and the door knocks guy in dreadlocks beat up beaters out front my drive when you stand there and there's US Marshal badge right here it's like can I help you hindi it can we talk and sit you got

creds 6 of creds in through the door so I opened the door and he says so you noticed me huh yeah what do you mean he says sheriff Cole you're the only one in the neighborhood only person I saw all day you called me it you call the apt ninja I said do you belong here he goes nope but neither does this guy have you seen him and I couldn't help him but yay we get response you can ask my wife from back about two minutes for ya so yeah there's good news to live in the county alrighty and six o'clock shows up next

you