HuxleyBarbee

[Music] uh hi everybody welcome so my name is Huxley Barbie I'm that is actually my real name it's not a handle uh and I'm the only Huxley Barbie you're ever going to meet uh let's go to the next slide please next slide there you go okay cool all right so this is a little bit about me I am the organizer for Biz New York City and I love going to other Biz to check them out this is not my first time in Calgary I've transited through Calgary in the past I actually had a project in a town called Medicine Hat which was my first introduction to Alberta uh I'm also the security evangelist at runzo uh if

you're afraid of QR codes that's the link if you want to uh connect with me uh but more importantly I've had a an exended career as a security consultant and many of my customers have had uh OT environments that they had to protect uh so we're talking about for example uh all the signs that you would find in a rail station or or a bus station or a manufacturing plan and all the manufacturing devices that they may have and even some cases higher education they have they have OT devices as well and I don't have my slides so always know what's coming next all right um so in in this talk I hope that one

you'll learn more about OT and IC than you did um an hour before and you'll learn about some of the challenges around protecting OT and also challenges around figuring out what you have in your OT environment and some of the ways that you can deal with those challenges so before we go to the next slide can anybody tell me of all the the chips that are manufactured what percentage of them go into embedded devices or non-it devices I should say wer 98 98 7 99 no 91 91 let's do prices right so I want you to go as close as possible without going over you went over 99 went over you're right on you are right on

yes next slide please approxim only 90% of chips that are manufactured are uh meant for the devices that are on the right hand side over here so if you can imagine the attack surface for the adversary is actually much larger on the right hand side of this diagram as opposed to the left hand side in this talk I'm going to be focusing on this group over here the OT devices right and and what do I mean by that because often times there's a lot this conflation of iot and OT so I want to take a moment to explain the difference with iot devices we're talking about headless devices that tend to be in uh your office or your homes

they tend to not be considered critical infrastructure or key resources whereas the devices on the right hand side here what I'm calling OT uh are sometimes there's a subgroup of them called IC industrial Control Systems these are critical infrastructure these are dams these are transportation manufacturing all sorts of all sorts of environments where they're doing things that have some sort of ramification for our society and our lives another way I think about OT versus iot is if I walk into that environment do I need to wear a hard hat right which is not 100% true but you know it just gives you some sense of okay these are the types of devices that matter to our lives our society that

make our cities and our states function our provinces function as well uh one thing I want to note here is often times medical devices are considered internet of medical things however because they are critical infrastructure I include them in OT as well okay yes out

okay sorry I'm all right so if you are like me you probably grew up on the it side right CU in your dorm you could set up a server in your laptop or what have you and you can go play around with stuff very few of us have had a chance to have a robotic arm or an intelligent conveyor belt in our dorm room so OT is a little bit foreign for us right so I want to take a moment to go through and explain what an OT environment looks like so you have a better Baseline understanding all right so let's go to the next slide actually let's go to one more okay so a big big caveat here this

there's a lot of variety in OT environments right they have a lot of because they have a lot of specific I know either

they can be used for many many purposes right Financial modeling on the OT side devices are very very specific for their one task and because there's so many different tasks there's a lot of variety because they're all built specifically to do that one thing and so I'm showing you an here but just realize in the real world there's a lot of Divergence from this particular example okay one example lots of variety so what we have here is a water treatment facility right so there's this tank here there's dirty water that comes up from the left uh pipe here goes into the treatment tank right so this um this valve opens up and this pump has to pump

the dirty water in and it sits in there for about an hour uh before it comes out all right the way this operates is when the water is lower than the lower sensor that's a s lower than the lower sensor uh the the pump starts pumping water in and then after an hour after the water reaches the higher sensor then the pump stops and then sits there for an hour and then gets drained out let's go to the next slide please so these two things here are sensors okay let's go to the next slide whereas the pumps and the valves are known as actuators right and collectively these are called field devices often times and in the real

world you actually might see the sensors and these actuators combined into a single device and an integrated uh field device for example next slide

please smarts usually there are no smarts on those field devices all the smarts is in that PLC next slide please typically a technician or an operator doesn't touch the PLC directly rather uh they operate through an HMI when I say HMI you should think of like let's

say pushing some buttons or flipping some switches next slide please that particular PLC is done through an engineer engineer's workstation and typically this actually is an IT device and typically it's running some really old version of Windows XP is not unheard of uh I have actually met somebody who said he once found Windows 3.1 on an engineer's workstation all right let's go to the next slide okay so this this what you see in this one Circle here is what is known as a system and when you have a site you have you might have multiple of these system collaborating together and what is known as a distributed control system DCS but there are cases where you would

have these each of these little systems distributed out over a large geographic area and organized into what is known as a supervisory control and data access a skate up and those cases you would have a remote an rtu over here that broker the conversation between the PLC and some sort of central location Mission Control if you will all right so that is OT this is an OT en again it's one example in reality there might be a lot of variety but hopefully it gets your sort your your mind thinking uh in in the right way about what type of devices we're talking about here next I'm going to move on to what are the challenges in

securing OT that's different from it let's go to the next slide please all right so in it we tend to be focused on data making sure only certain people can get to that data is that data encrypted at rest or in motion uh and so on and so forth is that data backed up and so on so forth right on the OT side we are very much focused on the movement of Machinery widgets and gears and so on and so forth next slide please typically it devices don't live very long how many phone

that's for on the OT side these devices tend to be in operation day after day second after second for decades I even heard of some locations where the time Horizon for these devices goes on for about 50 years right so really really old things let's go to the next slide please all right so one thing that's different about this talk is I'm giving away money so who can answer this question for $5 that that hand went up first inte avability that is correct the CIA Triad next slide please on the it side I I think all of you know this right we're very much focused on the three different pillars of or three different concerns I should say uh of

the CIA Triad on the OT side it is not the same availability is relatively way way way way more important than confidentiality and integrity and I want to dig into a little bit why that is so when an IT system goes down people can't log in or or whatever like you know the transaction doesn't happen right away when an OT system goes down that means the M the factory is not building things that means the oil is not flowing through the pipeline that means the water is not flowing out to the municipality right depending on type of organization you can sort of understand why availability is so important right if you have uh a commercial organization

every second that that OT device is not operating like it's supposed to one that's Revenue lost but two these environments are considered critical infrastructure and so they are heavily regulated and when they go down the government is going to come in and start inspecting and potentially find that Organization for that outage so in the United States Colonial pipeline went down and finra which was the regulatory Organization for for uh oil and gas uh came in and finded a colonial pipeline for $1 million so it's not just a revenue loss but additional fines now if we're talking about not a commercial organization but a quasi governmental organization you can imagine there's some politician out there who really

does not want the bad press of the recycling plant not operating properly or you know not having clean water for the city so for a variety of reasons there's this really really strong focus on availability far far far far less on confidentiality and integrity all right next slide please on the it side we have these operating systems that we all know but on the OT side we don't have time sharing operating systems we have real-time operating systems and further there are way way way more of them and so think about what that ramification would be let's say you are an a vendor that's building an EDR selling an EDR uh to to the world what would you do well

you would build an EDR that's compatible with Linux and then OSX and then windows and probably not bother with BSD unfortunately um but what does that mean on the OT side are you going to build out 65 different versions of your EDR especially when those operating systems are proprietary so you can't even get a a copy of that operating system to try build against it this will be important later so so keep that in mind all right next slide oh uh programming languages you all know the programming languages on the it side on the OT side they have completely different programming languages and so what this means is the last many decades of innovation in

software engineering release engineering quality insurance testing sdlc and so on and so forth they do not carry over to the OT side this by the way is sort of like an IDE for one of these programming languages all right next slide all right so on the it side we have Microsoft Patch Tuesday right we're all familiar with that uh on the OT side it's either patch never or or patch September sometimes yeah um why is that well this goes back to this goes back to what I was saying earlier uh about about availability these organizations do not want to take any downtime whatsoever to run a patch to run an update and they absolutely want to avoid

any sort of extended downtime that might occur from a bad update and so for that reason most organizations do not patch or they rarely patch all right next slide please right you might you might you might think I'm funny for saying that it is secure by by Design um and arguably you're right but on on the OT side it is it is way more true it is insecure by Design I'm talking about devices that don't require authentication devices that do not encrypt their traffic and so on and so forth next slide please on the it side typically there's some sort of governance right everybody needs to have EDR on their device and so on so forth

on the OT side depending on the industry there might be nothing there may be no security controls and in fact typically there's also no governance meaning devices are left online with default usernames default passwords with default configurations like listening on default ports and so on and so forth all right all right next slide on the it side you know devices are are directly or indirectly connected to the internet usually right but on the OT side this is where things are different back in the day if you wanted to compromise an OT device you actually had to walk up to it and then starting around 2005 or so that started to change so it changed ched because of the business on the

operational side in order to gain efficiencies they wanted to connect all these OT networks onto the to the it networks think about it this way let's say I have this like pipe out in the middle nowhere and somebody has to go flip a switch if we're air gapped that means I got to fly this person this technician out to the middle of nowhere just so they could do this one thing one that takes time and two that costs extra money wouldn't be nice wouldn't be so much better for operational reasons if you could just remotely make that adjustment on that particular device and yes absolutely operationally that would be way more efficient and way more

costeffective but what this also means is that that curtain of being air gapped came down and frankly in my opinion it is because these these environments were air gaap that people were okay with not having patches that people were okay with not having security controls that people are okay with having default usernames and passwords and so on and so forth but once that air gness came down all of a sudden all these insecurities of our o environments were laid bare to the adversary over the internet all right next slide who can tell me what's the name of this model oh there you go he was fast sorry all right yeah so some of you might say you know based on my

characterization of of OT like what are you talking about right there's this thing called the Purdue model all right so let me explain what Purdue is right so oh next slide please what we have here is a stratification of an OT environment where you are separating your levels of risk uh into these layers and what's supposed to happen here or I just before I say what's supposed to happen so down here you can see the field devices right the sensors the actuators you got your plc's over here the hmis over here and so on so forth and then the way up there uh there's like more it stuff so what's supposed to happen here is devices in each layer is not supposed

to communicate beyond the adjacent layer so if I'm in layer one I can talk to zero and two I cannot talk to three four five the other thing that's supposed to happen is in between these layers you're supposed to have some sort of security control maybe an IP s or a firewall that's ad adjudicating the the communication among those layers so a couple of things here two things that I want to mention for for the people who are like you know Pur Purdue fans right first and foremost you can see at the top there that's more iish right those are like it devices whereas down here it's like super OT type of device based on what I've told

you so far about the lack of security controls and lack of governance what's going to happen here is the moment somebody can infiltrate the top each layer down gets easier and easier each layer gets easier and easier so if you can get it through the top it's a it's a foregone conclusion that they're going to get to the bottom it's just a matter of time and number two very very few if any organizations fully expressed Purdue it just doesn't happen it's a very nice model but that's all it is it's not actually done can we go two slides please so remember right one more remember what we said earlier the plc's they're supposed to be down there like

layer one right so you should be going through like five other layers to before you can get to it fact of the matter is you can use Showdown or or even just Google to find a PLC that's directly connected to the internet what happened to layers 2 through five they don't exist here's a PLC with a web interface directly connected to the internet I did not try this but default usernames and passwords might might have worked I don't know so this whole thing with Purdue is you know it's nice but it's just a model all right let's go to the next slide please uh oh sorry one more okay remember how I mentioned that that

many of these devices because of a lack of security governance they have default usernames and default passwords these are all easily searchable on Google as well you don't have to go very far you don't have to work very hard to to get to them all right next slide now if in the case for example you find a device that actually does have a non-standard username or password remember what I said earlier about patching these devices are often never patched and you can go right to the cesa website and get a catalog of vulnerabilities that you can try out uh for for exploitation and the next slide and for those of you who think well I don't know much about OT devices

like what tools do I even use to to to to compromise them well you know metas has modules already there so all right next slide I I hope at this point I have impressed upon everybody here to go and rethink about moving off grade um no no not really but but I I hope I impress upon everybody to that you know this is something that we should all be thinking about if your organization has OT environments you should really think about what is going on there and what you what you can be doing to protect it earlier I was a little I think a little fous or cynical when I said oh you know uh politician don't want the bad press

or these organizations don't want to get fined but that in no way discounts the importance of availability of OT environments right these OT environments are the ones that produce the Pharmaceuticals that we need the electricity that we need the gas the oil that we need and so on and so forth and so it behoves all of us to think more about securing our critical infrastructure and key resources and arguably one of the first things that you want to do as a security team with your OT environment is figure out what you have historically when security teams have tried to go figure out what they have with active scanning it's resulted in catastrophe often unreported often

unreported but specifically what happened is when people have scanned their OT environments they would crash their their OT devices causing the very outage that you're hoping to avoid um and so for that reason conventional wisdom in OT security says you should be doing passive Discovery well let's take a look at that next slide please so when you do passive Discovery typically what you do is you will span from your switches onto a network collector so you're Gathering that information and then you are uh taking inventory of the devices that are talking on the network you can also do Taps and if you have a super flat Network you can do broadcast as well but by and large you're you're going to be

spanning and spanning is actually very easy right just three iOS commands and already you have a a Spam Port set up or I should say it's very easy in a demo by a vendor and very easy in a PC by a vendor how many of you work at a place where you only have one switch just just one switch nobody nobody works anywhere where there's one switch right so let's go to the next slide and see what reality is what reality looks like fact of the matter is you're probably going to have interconnected switches and it's not going to be just simple three iOS commands and then you're done you're probably going to be using one of these

protocols to make it all work and you better pray that all of your switches support that protocol and that version of that protocol and so on and so

forth

um Australia and and the EU but it's going to take some time just like just like everything else yeah

yes yes was an i system right

but so is is there some that we see this you're in a different world where I is bridging come up with an overall solution for envir yeah uh so he's he's referring to that the gap between it and OT um these are different operational groups that managing those environments so it and it just makes it worse for security right because security already has this Gap with it and so now you have this Gap with OT as well so like it's a Triad of of non non- collaborativeness um it I I don't know if it's getting any better um I definitely had this one project where our solution was to make sure that um there was a streamlined workflow between it

and OT so we had the situation where it and it security was coming in they said uh they're installing a network access control right a neck and so they wanted to make sure these devices um couldn't just you know get with nearly get on the network right that anybody could just plug in and uh the OT folks are like no we're not we're not you're not doing that because we need to be able to plug in our devices ASAP like let's say this robotic arm goes down I need to replace it with another one I'm plugging that in and it better work right um and so you know our solution was to to build this

workflow for them that allowed ot to be able to update the neck without giving OT full access to the neck right because it didn't want them to have control over the neck and so I was just like it's um you know in my case it was a a software based solution that that created that workflow that allowed them to work together but I think there has to be more than that right there also has to be like you know culture shift as you mentioned like people coming together to understand each other and so on and so forth I don't know if we necessarily solve that problem between it and security so and I think like it and OT

is even further out um unfortunately so know yes not my switch not my router not my problem go away yeah all right uh another one come on oh yeah well I'll go to him do you foresee a time like it's quite obvious that there needs to be kind of an elimination of these silos it would you simplify to just it and OT you foresee a time where OT is part of it OT is part of it so yeah I don't know how to what's the best way to put this like on the OT side you have a lot of technicians like you know the person that came and installed your FiOS right they come in with the device they plug

this in and they you know they wired up and then they they make sure like the the the light is on and then they leave right there's a lot more of that on the OT side whereas on the it side I think there's a um larger breath of knowledge that you need in order to be successful yeah but I'm just thinking kind of eradicating soci mindset you know so that in two entities they are one the same serving greater good based on what I see today I don't see that that uh distinction completely going away but organizationally it is possible um but no I don't I don't see that right now not to say it can't

happen all right he was he was next

yes

yes

yep data diodes yeah

um i' I've not seen as much thought as you've given it so so far in your question yeah

okay only but installing another set of devices which your data quickly to the CL allow you all right so let's go to the next slide because I'm gonna run out of time all right so there was a time where you follow these rules but these days you all get into Ubers you if you have a Tesla you you might have allowed it to try and drive you I think most of you have probably bought Bitcoin at some point at least at least you know a little bit all of you have worked from home by now and so maybe it's time to consider active active Discovery in your OT environment alongside of passive Discovery all right

next slide please this is me if you want to connect happy to connect reach out to me like making new friends all right thank you

right yeah um I've seen a few i' I've seen a few environments that have set up like a um a simulation or like a lab yeah like it looks like the thing real thing so far it's mostly been with electrical yeah no like like a full like mock utility electrial plant I've seen that in the other Industries not not as much I I don't know about being Advanced I think it's just like the [Music] wherewithal