Using documentation to build better OT security knowledge graphs

excellent all right thank you very much um yeah as Ryan said I'm going to be talking about uh OT IC skada uh whatever you want to call it uh this is some work that um that my team at work has been doing and I I thought it was interesting so even if it's not directly applicable I think to to what you do I think there will probably be maybe some some ideas that might inspire you uh as you said I'm from uh from Canada and moved to Norway apparently I'm I'm fond of cold places uh working for an OT security startup here called Omni and uh so you've heard me throw this word or This Acronym around a lot

at this point what is OT uh so if you're not familiar with it uh OT or operational technology operational technology is basically anytime that you have uh computers and digital systems interacting with the physical world so maybe that's in a factory setting like this maybe that's in with some medical devices um maybe you've got like a big uh robot trying to catch your enemies um and it's it has a lot of similarities to it security but also a few things that are different uh similar to with the medical stuff that we just saw a lot of it's very old hard to patch uh not necessarily designed with security in mind uh and that can lead to

some interesting challenges uh particularly what we were looking at was uh context and visibility for risk assessment and for instant response things um so in a lot of these networks they were built like 20 30 40 years ago uh so maybe you have uh some Network sensors that you can see you know your standard like it things in the network but there's going to be gaps there and if you're an incident responder and you see an alert come in on an IP address maybe you see that it's critical but you you don't really know why uh you you see that but it's fairly opaque and so we were trying to figure out is there a

good way that we can uh provide some more in insight into this and maybe show someone who's responding well is is this something that I really need to care about is this controlling you know like the the uh main drill on like an oil rig or something like that or is this controlling like the AC unit in the break room uh it's if you're an instant responder that that can kind of Handy to do um or the the other uh sort of thing that we were trying to do here was help with some risk assessment because if patching is really hard and patching is expensive you want to really focus your efforts where it's going to have a a

meaningful impact um so what we found was that these Network sensors were really good for the it stuff they can tell you there's uh we we see the services running on this IP address it's it's got this vulnerability not so good for showing you what that thing was actually doing and uh that's where we started looking to some alternative sources uh so the the example here maybe we can see that you know there's this controller thing and we might have some guesses as to what those wires are going to but we're we're not quite sure so where where can we else or where can we look to try to fill in these gaps uh and

as you might have guessed from the title uh the answer turns out to be a lot of docs there are some other tools that we have as well you can go to the site and actually walk around and see uh maybe you can do like some end map scans or something like that uh though in OT you have to be kind of careful with that because again a lot of these systems are very old very brittle uh something as as simple as running a scan uh too aggressively could knock the whole thing over uh so you want to be careful with that you also might interview some site experts but again these guys are very

busy takes a lot of money uh a lot of time and so we wanted to try to build up uh as good a at the site as we could before going into these sort of more expensive methods uh and that for that we turn to docs so our plan is to convert these docs into a machine readable format uh build a graph from them which I'll talk a bit about what we use that for and then use that to answer some questions either about risk assessments and and trying to pinpoint where we want to spend effort or on uh helping give instant responders more an idea what they're dealing with there are a few types of docs that we'll

go over in specific uh some of them you're probably familiar with things like Network diagrams or risk assessments or functional specifications which are basically just like a word dock describing how the system is supposed to function uh others are a little bit more specific to the the sort of Physical Realm uh things like P and IDs are piping in instrumentation diagrams and they look like this uh each one of those uh icons there represents a an actual physical either like a a valve or a tank or or something like that and then lines are going to be frequently like pipes between them and so this is laying out how the whole process the physical process is is going

to work for that particular part of the plant uh and there's a lot of good information that we can pull out of these uh in particular as as mentioned those symbols are generally kind of standardized so that's might be a uh a valve is the little thing in the middle there uh a lot of them have tags which are labels associated with them and as we'll see in a second there's some semantic data that we can pull out of those uh and the relations between which valve is attached to which tank which tank is controlled by which uh which computer is is of Interest as well uh tagging is the first thing that we decided to look at because uh pulling

text out of these documents is is much easier than trying to understand the uh the actual picture information at least as a starting point um a lot of these were actually just scans of uh of PDFs but even those you can at least do OCR and there's some pretty good tools for that and these tags uh once you uh once you have them in text format you can actually pull some some information out of them and it'll probably look different from site to site but for for sites there is generally a a pattern to it so for this particular uh standard that they use for this site uh it's laid out into which system which subsystem uh

which part of that subsystem and then even has some information about uh if there's any parallelism and which uh instance of that item it is so when we're looking back at uh something like this maybe this is like from we can see it's a valve and maybe it's valve one versus two and you can see sort of where it is on the uh the thing there uh in this case uh 84 is maybe it's the electrical system and elj is the code for a junction box which uh has something going on uh so the first stuff that we did was we took these docs we put them through just some standard like offthe shelf OCR

tools uh and then you can start doing text search on them as as you noted the uh or as you might have noted the tags are very very standardized which leaves itself well to regular expressions and so you can write a pretty easy script to say uh given a tag look through all the documents that we have and find Which documents are related to this piece of equipment because that the tag identifies like a a piece of equipment uh or given uh given a document give me a list of all the equipment that it talks about and we're dealing with a lot of docs usually so this is um something that already can start to narrow down your uh your search

when you're trying to figure out what's going on with that particular system uh and of course we've got to use llms because those are the new buzzword uh and we can use those with especially the more sort of text based uh functional documents uh where it might describe what uh what operating system a computer controlling a valve is using in just like plain text and so we can start using some uh again offthe shelf tools like Lang chain and uh like llama index just to to start building up uh the graph which we which we'll see in a second template matching with just like some standard uh open CV stuff can also be used to start matching the the

symbols in those diagrams uh so here's an example of the the obligatory llm uh can the system that it's associated with this tag run with with one of the meters offline which is maybe important for your your risk assessment and it says yeah we found this document uh and it the the glowing green chemical thing can run uh even if one of them is uh is off because it's got one built-in spare and this is really what we are trying to build uh and one of the things that I think is is most interesting and and more broader than uh just OT sense um is that graphs can can do a lot for you there is a lot of really good

tooling uh available around uh doing queries and doing sorts of reasoning and algorithms on on graphs uh and so we wanted to to try to take advantage of that so the graph that we built uh had nodes for uh more traditional it things like network devices that are running instances of software which is affected by cves and are connected to networks uh and then from some of the diagrams that we had like those PN IDs and uh and other similar ones we can see which physical things they're connected to uh which functional assets flow to which if there's a a tank that's flowing to a through a pipe to a different tank then if you can affect the Upstream one then

that that'll have an effect on the downstream one that's good to know uh and from some risk assessments that we found uh we also found that we could tie these functional assets to consequences if the uh the tank hold the fuel uh is ruptured then people from uh from the companies have probably done a risk assessment and put like a a severity on that of that you know there's this a high environmental consequence or something like that uh there was still a lot of uh of manual work here but using some uh some python we were able to reduce some of that boiler plate and then we basically just put it in a uh a commercial like

off the shelf graph database uh so a lot of this I think is uh not too difficult to um to reproduce and I'll have some uh Links at the end to some of the the software that's handy if you want to try doing something like this yourself so a very small segment of the graph ends up looking like this uh you can see that we've got um the sort of turquoisey nodes are uh our computers uh PLC in this case stands for programm programmable logic controller uh which is controlling some robotic arms a conveyor belt uh a self-destruct thing because if we're looking at the Acme TNT Factory it's it's got to have one of those by law uh

and HMI is a human machine interface but again basically just a a computer that's connected to that Network there and so we can start asking these graphs uh some simple queries like if I'm on the HMI which uh which physical devices can I control and and uh there's a like the the graph database does that pretty quickly and it can find us from uh it translates that from the the query down the bottom there and so we can see all right if I'm on this HMI I can control the robotic arms the conveyor belt and the self-destruct thing or if I'm trying to do a risk assessment on a particular piece of equipment maybe uh I'm looking

at the self-destruct thing I can ask the database which uh which uh it devices have a path to be able to control this uh so you can see how that would be uh useful for some of these higher uh more destructive physical things we can go even further and tie this to things like uh consequences from those risk assessments so if I'm an attacker and I'm on that HMI I can say show me the the fastest way to um to cause some kind of consequence that would be

bad um but more exciting than that would be to actually set some agents loose on this graph and see what they can find so uh with again just another simple python script you can have it start at one of your um when your computer nodes uh and have it look for other any networks that are connected to that uh device uh other devices on that Network and then find cves on those devices uh which are remotely exploitable through the network and sort of build up your your uh graph of things that you have control over and if they have a a cve which gives you code execution you can keep pivoting from there uh if it's a

cve which can cause uh Nal of service then you can potentially disrupt whatever physical assets are being controlled and so you can sort of build up this blast radius from uh each of the devices that that you can start from and if you do this to enough of these you can uh start to build up a notion of which uh which nodes come up again and again and sort of maybe try to pinpoint uh which specific cves are going to be the most impactful to to patch if you want if you can only patch like three things maybe we can find some hot spots in the graph and uh try to get the most bang for our buck

when when uh patching because if you if you run the the whole thing and you you can see what the most valuable assets are uh take a few of the uh the cves out and run the whole thing again you can compare uh the cves in context and actually see how how much uh more uh more secure your network is afterwards rather than just looking at the uh like the CVSs score uh and there are tools that do this as well uh so we we built one but there's also a couple of Open Source ones from uh miter which again will be uh linked at the the end of this and they they do basically the same

thing uh so here's an example of that um we can uh I told that we're looking for uh roots from the uh engineering station in the chemical process plant to anything that will have a high environmental cost and you can see that it finds some devices uh it looks for the networks on those devices goes and discovers a bunch of stuff and eventually it finds the the self-destruct flamethrower that's reachable through some steps gives you the attack path there uh exploit the cve pivot to this thing and then you can control it and it has the environmental consequence of high uh this was the I think I already sort of went through this verbally but

um again you uh basically if you run this uh differentially if you run the whole thing uh with different uh different aspects in the graph maybe if you put in like a firewall or if you take out some of the cve then you can uh see how specifically that'll that'll affect how difficult it is to find these paths and how easy it is to get from a likely entry point like the internet to some sort of unwanted event like a high environmental consequence from something in a risk assessment um but this is still pretty pretty early days uh there was a lot of manual work here which we're trying to get uh get down um the diagram parsing in

particular is signed that still requires a lot of manual labor and so we're we're looking into some more computer vision things to to try to deal with that of course I'm sure there's more we could be doing with uh with llms going into the future uh and a a really big one is connecting to more data sources to deal with drift because uh these documents are are generally there are some from when the the equipment gets updated but of course when you're dealing with systems this big and having contractors come in and and whatnot things are always going to change and that's not going to necessarily be reflected in the docs uh so something that's still to be tested

is um how uh how close these models actually get to uh to real life and that's another place where I think it's a good starting point but you still do want to use those other tools like the site visits and interviews to uh sort of refine it afterwards uh these are basically all theol tools that we used uh in the process and I'm happy to if you're interested go into sort of more specifically how we changed them all together uh but I'll just leave that up for a second um uh cat the one at the bottom there was the the miter one that was doing the attack paths and that's kind of fun to uh to play around

with but the uh the two things that I really want to stress and that I thought were was was most interesting about this was one uh there's a lot of untapped information in uh docs especially when it comes to some of this information that you're not going to find just by doing like a network scan or something like that uh and I think that's probably true of some other areas as well where they're if if you think of of all the information that you have available to you when you're solving a problem uh frequently there will be some that's a little bit harder to get to but but might give you some good Insight uh

and two uh graphs are great for modeling complex systems there's been uh a whole bunch of really good uh open source software made for these and uh like Theory done in them with algorithms and things and I think it's uh a tool that we should not be forgetting in this world of you know llms and and there all lot some things we we've got some other tools that are I think better suited to some problems um so I want to say thanks to the people that were were doing this work to me or with me um and that that's uh all I have for now so I don't know if I have time for questions but I'll be uh sticking around

for the rest of the day as well no let's move on but come see Ian thank you Ian