GT - F Your ML Model

good morning welcome to uh bides Las Vegas obviously if this is not your final destination please deboard the airplane and find somewhere else cuz you definitely broken the time loot continum uh this is f your machine learning model by Colt Blackmore a few announcements before we begin we'd like to thank our sponsors especially our Diamond sponsor Adobe our gold sponsors uh bluecat prre Toyota it's your support that makes this conference possible please silence your cell phones and as a courtesy for your speakers if you're going to ask a question please move to the microphone raise your hand when we're ready we'll call on you and to alleviate some of the time crunch we had getting things set up hand it

over thank you all right how's that volume can't hear me can't hear me that's just mean all right we still got people wandering in but that's all right I'm going to meander a bit at the beginning here so I have a theory about why I ended up going first so I I'll attempt to describe it uh I was looking at the schedule this morning and there are by my count uh Baker's Dozen talks here in ground truth over the next couple days they aren't all about machine learning but a bunch of them are uh about a third actually exactly a third and I can only assume that uh maybe some impish organizer made that decision with

the implicit understanding that we would start things off with a bang by uh let's say crapping on machine learning from a great height for the life of me I can't figure out what could possibly lead somebody to uh such a belief certainly not the title of the talk uh or or the description uh but as a matter of fact I have nothing but love in my heart for machine learning so rather than do a typical sort of speaker intro I'm going to do an origin story and I'm actually really curious to see if my experience here is unique or if it's actually pretty common with all of you who do data sciency stuff so as show of hands how many of

you remember the exact moment that you first encountered machine learning that's way fewer than I thought uh interesting all right let let's Whittle it down still a little bit I'm curious if anybody's going to be left um so those of you who just raised your hand uh if you found your way to machine learning on your own uh like it wasn't a school assignment or a task you were given at work or something like that put your hand back up that feels like more hands than there were before you guys are are are ter terrible audience ridiculous um but that's good all right so we we have something in common uh I also remember the exact moment I first encountered

machine learning uh there used to be a website called gamma Sutra it was a video game industry site so not like for fans of video games but for people who worked or at least aspired to work uh in the industry and sometime back around 2009 is I don't remember the year the date obviously exactly but I I remember the moment uh they published an article on this new thing that people were starting to use in video games called machine learning and the only thing I remember about that article is the example that they LED off with because it was so damn cool so there was a hospital in Canada I'm uh almost positive it was the Toronto Hospital for

sick children but the article is long since gone from the internet so I can't verify that but I'm pretty sure that's what it was they're attached to the University of Toronto and they were using machine learning to detect when kids would get sick before it actually happened and again this is 2009 right so the state-of-the-art at that point uh compared to today not so good uh it was like a basic time series model the feature space was quite small I want to say it was around two dozen features if I put you guys on the spot right now and asked you to name features we could use for some kind of model like this they

were using the exact kinds of things that you would think of right it was heart rate it was temperature oxygen level skin conductance uh blood pressure those kinds of things right so about two dozen of those and with that in place they were able to determine with a reasonable degree of accuracy right 70 to 80 % about 24 hours in advance when one of these kids would become symptomatic right it's not like you're not figuring out that a kid's going to get sick before they're sick you're figuring out that they're already sick they're just not showing it yet and of course by knowing that 24 hours in advance you can apply early care you can

reduce the impact of the illness and like the long and short of this is literally saving babies right that's uh I think we could all agree not a bad thing so machine learning uh actually pretty cool um that was to that point in my life as a technical person probably the coolest thing I'd heard of I I didn't have uh any kind of background in statistical Methods at that point I don't think I'd even heard of linear regression for example um so I I didn't know anything but it was an interesting enough example to Dive Right In and start working on this stuff and so uh a year and and change maybe later uh I

made my first malware detection model and and it worked uh quite well so this was 2010 and uh well enough uh in fact that about 5 years later when I was working at pal to networks uh we took that thing that ID built 5 years before and uh kind of a stripped down version of it that wasn't quite as good and we shipped that in a couple of different products so again machine learning literally saving babies uh more or less built my whole career on it uh I can't say too many mean things really like nothing for nothing in my heart but love for for machine learning uh it it's pretty great but there's got to be a but right

so but machine learning is not the best solution to every problem in fact there are whole classes of problems where machine learning isn't even a good solution and and actually there are cases where you can prove this mathematically so you can look at things like inapproximability results and uh in certain instances you can prove that machine learning is just going to be a terrible approach to a problem because the answer it gives you can't be guaranteed to be more than like 50% of the optimal answer or 60% of the optimal answer so that's just kind of how things are meanwhile there is this big old wide world of AI out there Beyond machine learning often very different from

machine learning but sometimes similar that in a lot of these cases where machine learning is not effective can be used to tackle the same problems and can do it better than machine learning can right and so what I've been wondering over the last five or six years uh as I've become more familiar with these other areas of AI is what the hell is going on in cyber security where we don't hear people talking about these other methods we don't see them using these other methods why is everybody so fixated on machine learning and we could speculate a lot of different reasons why that might be the case but the long and short of it is like this is this is

where we are um I think a good microcosm of the problem is actually uh self-driving and since I started with a clown slly it I figured we might as well have another clown and every clown deserves a nose so there you go Elon uh self-driving if you ask you know Joe on the street or even probably the average technical person they're going to just immediately associate that with uh machine learning right and and we know that that's not entirely unreasonable machine learning is a big part of what goes on in self-driving but is very far from the only part so if there are sort of three foundational systems that exist in self-driving machine learning is is

really responsible for one of them sort of foundationally right so the perception systems the the car's ability or whatever you're driving I guess it doesn't have to be a car but its ability to to see to sense its surroundings to know there's a sign and it's a stop sign or a yield sign or a stop light to see lanes and Lane markers to see other cars all these kinds of things right machine learning drives all of that so it's totally fair to associate ml with self-driving sure but it's only one of these three core systems and the others are equally interesting and we can find ways to apply them like meaningful ways to apply them uh to cyber security so

for example planning systems are quite important um if you're not familiar with automated planning or AI planning which has fewer syllables um planning systems create a logical representation of the world and our capabilities within it to allow us to reason about how to achieve things within that world so really basic example I wish I had like an attached mic so I can move around the room to try to illustrate this better but um we we do automated planning or human planning I guess in our heads all day long every day um if I have a goal which is say to advance to the next slide right I have multiple ways I can do that I brought a

clicker thinking I might be able to walk around and so if I was over there I could use the clicker to do it uh the other option of course is to be at the laptop and then I can use the keys like that works too all of those actions I could take have their own dependencies I can't use the clicker if the battery's dead I can't press the key if I'm on the other side of the room so my and the location of the laptop come into play but this is what planning is right it's a big logical representation of the world and a system for navigating that and being able to achieve things within

that world so we're going to talk a bunch more about that the third pillar of self-driving is Control Systems uh control systems are really where the rubber meets the road right so if planning tells you when to change lanes and when to turn right and left it's sort of like the Google maps of this whole thing uh the control system is the thing that hits the gas hits the brakes turns the steering wheel and these are usually formulate formulated as mathematical optimization problems usually and they have some kind of physics based constraints right so like gas brakes turning the steering wheel sure but if you hit the gas too hard you might fish tail and run into a wall if

you break too hard you have problems if you steer too hard you have problems so physical constraints come into play there and you get some some really interesting problems my uh favorite example actually of uh Control Systems from let's say the last decade actually has nothing to do with cars it comes from uh SpaceX another musk company uh and the vertical Landings of rockets which are just a acious uh control system optimization problem so of course Elon Musk wants everybody to think that he's Tony Stark and he solves all these problems themselves we know that that is not the case in fact we know exactly who at SpaceX is responsible for solving this problem making things happen it is another

Blackmore no relation to this Blackmore that I know of so Lars Blackmore formerly of NASA JPL uh left he worked on a team there that explored this kind of stuff now he's at SpaceX leading the team there that explores this kind of stuff and he is the uh the main guy who's been responsible for making uh the vertical Landings of the space uh SpaceX Rockets real and the way he went about that and the people he worked with at NASA went about it and the other people at SpaceX way they all as a team went about it is really really interesting so if you're familiar with mathematical optimization you probably already know uh there are these two sort of broad

categories of functions that you generally have to deal with one of those categories is convex functions uh convexity is a really nice property for a function to have uh it means that when you look at the uh solution surface for the function get a nice Bowl shape like this so like if you drop the marble in at any point on the function it's going to fall down to the bottom and rest there it's really easy to find whatever Optimum of the function that you care about right so it's it's nice and easy to deal with then you have the sort of hormonal teenager function where it's non convex it is all over the damn place

uh you really just have to watch yourself around it because it gets angry for no reason all that kind of stuff in this case if you were to drop a marble in from an arbitrary point in the function you have no idea where it's going to come to rest right it could be a local minimum it could be a global minimum it could be all over the place when it's a an important problem like Landing Rockets wherever it lands like it might be good enough and you land your rocket safely but it also might not be good enough and your rocket explodes um and there aren't people on the rocket so that's not the end of the world but

it's also not exactly the goal that you're hoping to achieve so what Lars and the NASA folks and the SpaceX folks figured out is a way to relax the non-convex function of the hellacious rocket Landing problem into a convex version right we call that a relaxation of the function and this isn't uh particularly interesting in itself because the way that you do mathematical optimizations often to find relaxations and solve those and use those to bound the other function just sort of zero in on the ultimate answer but what they figured out how to do was find a relaxation where when you find the solution for the relaxed version it's guaranteed to also be a global solution for uh the original

problem which is pretty damn cool so instead of trying to tackle something like this with neural networks where you have no guarantees around the results you have to figure out how how do I even run this in a rocket uh doing things that Rockets do which are maybe not amenable to you know holding Nvidia gpus or whatever uh they found a precise ma mathematical way to uh approach it um and now they land Rockets like you know three times a week or or whatever like it's kind of routine for them so we're not going to talk about uh Control Systems per se today but we are going to talk more about mathematical optimization because it is an important

tool in our toolbox for dealing with uh security problems but we are going to start with automated planning which uh is a lot of fun because it's linked to video games so um autom planning is not new it's kind of an ancient inv venerable field people are still doing like Cutting Edge research in it most of that deals with real-time systems so things like self-driving robotics um that's where all the hard problems are because in real time to be navigating a world reasoning logically about it right that's not a trivial thing to do so cool cool work is being done there but the area where I was first introduced uh to AI planning and where I've spent the

most time with it uh is is video games because you can do really cool stuff with this uh in video games so the example I want to call your attention to is the game fear this is not a new game if you're curious it's 20 years old uh I think originally published in 2003 the AI lead on fear was a dude named Jeff Orin he went on to do his PhD at MIT and is uh turned out to be quite the kind of AI and computer science guy but back at this point in his life uh he was building AI for for video games and so what he did is he looked at the way that

people did AI in games up to that point which is really really basic it's things like uh Behavior trees or finite State machines which have to be manually painstakingly explicitly encoded by human beings it's a terrible terrible approach uh and he didn't like it and the results that it it delivered like didn't like those nobody likes those so he started by taking a system from Stanford called strips if you're familiar with it it's the uh the Stanford Research Institute planning system strips and he uh so to speak stripped a bunch of stuff out of it and then enhanced it with some other stuff to make it work in video games and from there he was able to build a system that

basically blew everybody's hair back uh people even today they go back and they play the original fear game and they feel like when they're playing against the computer they're actually they're actually playing against other human beings like it has a a real sort of liike quality to it um it's very Dynamic uh and and interesting right it just feels like there's somebody else on the other side of this thing uh to the point that it even weirds some people out a little bit so the reason we know so much about fear actually is because uh Jeff did a talk like this at the game developer conference he published a paper on it that paper was called three

states and a plan the AI of fear I encourage anybody who's interested to go read it because it's it's very approachable but uh the long and short of it is pretty simple um when you boil AI planning down to its core there really just a few things right you have States States can be as simple as basic proposition logic right so you can have variables X Y or meaningful names they can be true or false or you can give them Turner values give unknown put unnown or null in there um you can also be much more specific right you can make planning as complex as you want it to be so um a state could be coordinates in a

coordinate system it could be temperature in a room it could be a color it could be really anything you want to uh to reason about right you could you can build it however you want so you've got your States and then you have actions so actions are things that you can do within the world that generally are going to transform one state into another state right so if I want if I want to advance the slide whether I use the clicker or the keyboard uh I take the action to advance it and now I've changed the state from the previous slide to the new slide right so it's just a transformation for uh for States and then you combine these

two things together using logic to get these really complex interesting emergent uh behaviors so the way that works is you have your initial state which would be like the state of this room as it is right now you have a goal state which is whatever changes I would like to make to the room then I look at all the actions that are available to me to make those changes and I reason about how to execute from those actions to make the changes real and now I've transformed the state of the room to whatever I want it to be right it's it's pretty straightforward the uh the implementation that they did for fear they gave it an awesome name it's called

GOP I like it so much that there's a dedicated slide for it there's no reason for there to be I just spent an hour with mid Journey tring trying to get it to make text and it was absolutely worth it it's like goop and soap put together really clean slime I don't know but I love it so uh GOP GOP is really cool and I I wanted to have a video to show you guys so you can kind of get a a feel for how Dynamic these really simple implementations of AI planing are the problem is when you take like the first person shooter version unless you're the one playing the game it's just a it's a

lot of visual data to process right it's not easy to make sense of uh what I found instead which is actually kind of awesome is a some random person on Reddit he like a hobbyist game developer had been struggling to get AI to work uh in the hobby project that he was working on uh had done finite State machines had done Behavior treates had done all these classic things none of them were working really well and so uh this person discovered GOP and did a quick implementation and was just like holy crap this works really really well so they made a video and then they wrote it up on posted it to Reddit it's like you

guys you don't understand everybody should be using this was more or less the tone of it it's easy to Google you guys can find it but so I'm just going to play a quick 20- second clip from that video so you can kind of see what was going on um I'm not sure how legible that text is for you guys but what we got here is we have the AI agent in the bottom right we've got the human player in the top left you can see some of the state variables mostly Boolean that the AI agent is interested in in playing with um and you'll see those change over time as things happen once I

start playing and uh you can see in the top the goal of the agent and the plan that it's going to implement to achieve that goal so in the beginning it's just chilling out because it doesn't know there's an enemy once it becomes aware of an enemy it has to go through a process it's like it's okay I got to make sure that I can see the enemy so that I can take aim at the enemy so that I can shoot the enemy then when the human player disarms the AI agent it's like well crap I can't shoot you without a gun right so now I have to go get a gun nobody told it that it needs to go

get a gun it's figuring that out based on the fact that it wants to shoot and it needs a gun to do that and this is all happening you know every tick of the video um this planning process is being carried out so so I'm just going to play it 20 seconds and you'll you'll get a sense of how it

goes see takes the gun

away so the point here isn't that this is like the most amazing game AI you've ever seen like obviously it's just somebody's little hobby project right the point is this is Trivial to do in an afternoon and the behavioral complexity that emerges out of it is just completely disproportionate to the difficulty of building it like it's a really powerful kind of system all right so let's move on why the hell am I talking about video game AI at a cyber security conference uh the answer is uh because uh a few years ago eight years ago something like that early 2010s this new product category emerged in security called sore security orchestration automation and response and sore sold

itself as kind of uh it was going to be I don't know the Messiah of security it was going to Overlay all of your existing security products it was going to help them talk to each other via apis so you could take data from one place and use it to execute actions in another place and it was just going to make security amazing um that's how it positioned itself uh at least the reality turned out to be a little bit less impressive um the automation that Sor provides uh has to be manually constructed right so it's the video game equivalent of the finite State machines if you use sore you have to go make these playbooks yourself that's what

they call them playbooks their workflows H it's a painstaking thing to have to do uh to ask understaffed underpowered security teams to go do this uh realistically like they're not going to do it and so what happened with SAR I mean there were some really big companies and some big exits right the top two Phantom and demisto they exited to a Splunk and pel to networks respectively for almost a billion dollars combined but they don't have that many customers like there aren't that many companies out there using this stuff CU it's just too damn hard right too damn hard so this is what we ended up with so I thought a fun thing to do

because as far as I know this still has not been done right in my opinion fun thing to do would be uh to build a little AI planning system around sore today so we can kind of see how it would work there are any number of Open Source libraries we could use for that I'm actually not going to use any of them uh because there are some unique characteristics to Security in composing apis where we want like a high degree of parallelism and and uh stuff like that uh it's easier to roll our own in this particular case but there's good stuff if you like lisp there's shop 3 out there if you like python there's a library called Fast

downward there's lots of good stuff uh one though I did explicitly want to call your attention to is NASA because again space and Rockets and robots are are cool stuff um on GitHub NASA Europa you can see the planner that NASA has been using in a variety of different uh space missions for 24 25ish years now uh it's still around it's still kicking they use it for lots of interesting stuff bunch of Mars missions uh even today it's what they use to uh control the solar arrays on the International Space Station uh because that's how it Powers up right so uh they use the Europa engine for that it's pretty cool the code is there so

you can actually go and play with the planner that Nasa uses to drive robots on Mars it's just there we're not going to do that though we're going to do something else all right so let's get into into it now a warning here I have something like 120 slides it's a lot of code and when we get to the next part a lot of equations we don't have time to like Linger on every little line every little equation so don't worry too much about catching all the details the important thing is the high level Concepts right so I just focus maybe more on on what I'm saying and not trying to make sense of everything that's going on here and

uh I think I think we'll be okay so we're going to build a little planning system well I've already built a little planning system but we're going to Define some things that it can do and then see what happens um so the way that we create states in the system is to Define uh enumeration values so we're going to start with just one ATL risk if you never heard atto before it's account takeover right so there's some risk that we're aware of of account takeover there was a fishing attack or something like that and uh what I'll often do is so that we don't we don't have to write conditions and effects out long hand

every time is just declare a variable in this case account at risk uh so we have some shorthand for how we're going to reason and talk about these things Okay so we've got our one State uh now we're going to give the planner some actions that it can execute on so we're going to start with two we're going to say you have the action uh where you can force a user to reset their password that's a useful thing to be able to do to uh reduce risk and you have the action to force a log out so to terminate any active sessions for a given account and that can also help you reduce the risk of account

takeover we don't want our planner to only be able to act on the risk that we tell it is there so we're ALS Al going to give it an action that allows it to go and find risk on its own so in this case we're going to let it talk to a firewall where it can see hey did any of the users behind my firewall click on URL that we know to be a fishing site and if we see that then we know that there's risk and and we can act on it right straightforward so we can plug all of this into the planner we give it our three actions the start state has nothing in it right so no no conditions

to start with and our goal is to uh mitigate risk like Li the risk so note that there is no risk in the start condition but the goal is to eliminate it so now the planner is forced to go find risk to eliminate if it wants to achieve its goal which is a useful thing to have it have it doing all right so so what does this end up looking like well we have our start and we have our goal and uh depending on the size of what you're dealing with you might want to have ristic searches there a lot of different things you can do here right but this is a fairly small plan so we're

just going to combinator explode the plan space so we can see all of it sitting there we can see The Logical relationships between steps of the plan and then we can run something like a shortest path algorithm like a star and it will find a way to get from our start to our goal if there is a way right if a way is available so in this case uh it's going to go to the firewall find the bad thing Force the password reset reach the goal pretty straightforward the thing here is this is like a standard linear planning type of thing to do but we don't really want that insecurity if I have two or 50 different ways of

mitigating risk I probably want to run them all I don't care if if just one gets the job done like I just want to execute everything and I want to do it in parallel I don't want to have serialization of actions I don't want to be limited in terms of of what I can execute so at this point I go into to the the baby planning system that I'm building and I just make it do everything everywhere all at once right we're going to win an Oscar with this planner that's that's the goal so at that point now it's just going to do all the things right so it gets to the firewall it finds the bad uh the bad URL

now it's simultaneously going to force the password reset log the user app out and now we've reached our goal now this is a little bit of of nonsense and I say that because uh firewalls are a little bit of nonsense right most traffic these days is encrypted most people are not decrypting with their firewalls so the firewalls aren't actually seeing anything a firewall as a source of data is not super interesting with apologies to my friends at palel to networks so we want to integrate uh some additional sources of data here so what we're going to do is we're going to go to endpoints so some not near nearly enough but some EDR products uh will actually log every

single URL a user visits on their devices and just throw it all up into the cloud we don't actually know if these URLs are good or bad or something in the middle U but we know that the user visited them so we're going to start by defining some new states that let us keep track of whether or not a URL is known to be fishing uh whether or not a user has visited it and whether or not we sandbox it in cuckoo or something like that uh and then we're going to start adding new actions so here's our action to uh go to the endpoint product the EDR and uh start pulling URLs pretty straightforward then we need to enrich

them right we need to figure out is this URL a bad URL is it good is it in the middle like what what's actually going on one way we can do that is to ask virus total and if virus total has seen the URL before well isn't that nice now we know it's bad maybe and uh we can take action on it but maybe virus total doesn't know anything right maybe it's never seen it before that would be pretty standard for virus total in my experience so maybe we also have a local solution we can use a cucko instance whatever right our sandbox some other analysis system doesn't really matter in that case we can ask that system hey do

you know if the URL is bad and if it knows it'll give us an answer this time though if it doesn't know we also have the ability to submit the URL to the uh the sandbox and then it will determine on its own whether it's good or bad and uh get back to us with the news and again we can take action so now we're going to define a new problem new planner throw all our actions in there but the goal has changed instead of trying to mitigate risk now we are trying to figure out is this URL that we have seen bad or do we not need to worry about it that's the new goal so

how does this play out all right we've got our start action and our end action and if we hit the firewall again nothing's going to change right it's going to tell us it's bad now we know easy peasy we're done when we go to the endpoint though we don't know automatically if it's good or bad so the planner is going to say okay what can I do to enrich what I know about this URL well I can go to virus total and if virus total knows we've reached our goal and again everything is good but if virus total doesn't know we need to look somewhere else so now we've got our sandbox system hey sandbox have you seen

this URL do you know if it's bad and if it knows then again we've reached our goal but if it doesn't know now we have this extra step the planner says okay well now we can submit it right we can figure out if it's good or bad dynamically feed that back into the system and that takes us uh to where we want to go now we've made a determination is it good it's a bad URL everything is going exactly as we would like it to go okay so now we're we're starting to cook we're going to do one last round of enhancements here we're going to add a couple things first of all when we're getting into the business

of detecting bad stuff it's not enough to just resolve it on the back end and be done we want to alert somebody that hey we found a bad thing and we're going to take some actions to resolve it right so we're going to add a new action alert visited fishing URL and that's going to take care of that for us it's also going to take care of registering the fact that there was account takeover risk so that we can then take actions on it and we're going to add a new mitigation option so this Action Auto Purge similar messages means if we find a bad thing we could reach into our email server like exchange for example find any messages

that might contain that bad thing and yank those back out okay yank those back out uh and if we do that well we've protected a whole bunch of users who weren't even impacted yet right so it's it's kind of Handy uh so now we to find a new problem through all of the ingredients in the pot here we're going to make a nice stew and again no starting conditions we're going to go out and find the risk ourselves and our goal is back to mitigating the risk right so now we're not just looking for is the Euro good or bad we're back to Seek and Destroy mode for the planner all right so what does it come up with

here it's going to start with the plan that we just had basically the only difference is instead of ending with figuring out good or bad it's going to do something a little extra it's going to issue that alert and that is the first step of the second part of the process when we issue the alert from there we're trying to proceed toward our actual goal which is mitigating the risk and now we have three different ways we can do that and again we can execute all of them in parallel so we can force the password reset boom reach the goal yank the bad stuff out of exchange boom reach the goal force uh the account log out

terminate all the active sessions boom reach the goal and that's how you can use automated planning to make Sor great again right we're really just scratching this surface here but hopefully you can kind of see you can imagine maybe what it would be like to have a security operations team with something like this running on the back end overlaying however many dozens of products they they're using all their data sources the ability to take actions for them whether it's in real time or human in the loop through jro tickets whatever it might be you can imagine how powerful uh that might be and how it would enable a whole bunch of security teams that don't use sorted a cuz it's

too damn hard to actually benefit from it so this I think is something that has to be built like somebody should build it my hope is that maybe somebody in this room will go build it because I don't have time right now uh so if anybody's in the market for startup ideas please uh email me I will be happy to explain more I will send you some of this demo code I will do whatever it takes to help drive you toward the goal because whether it's an open source project or or a company right this needs to exist cool cool all right now we're moving on mathematical optimization part two my opinion on security problems kind

of boils down to having the ability to translate a problem into different representations is the ultimate superpower so every form you might give a particular problem is going to lend itself to different kinds of solutions so by being able to translate the problem you get access to a bunch of different kind of tools for solving the problem and one of the most compelling sets of tools is uh math you can translate problems into mathematical structures and when you do that all of the ways that human beings have developed over the last 2 or 3 thousand years for wrangling mathematical structures present themselves to you as tools for solving your problem and that is a very powerful thing to be able to

do so I'm going to start this off by doing something incredibly stupid and as your attorney I I recommend that you absolutely don't do it but it's still fun we're going to take that planning problem and turn it into a function and optimize it and uh it's ridiculous but it it kind of shows how this works and then we'll we'll do a more interesting problem after okay so how do we turn a planning problem into a function well we need some helper variables so we're going to create uh you can think of these as vectors or ordered sets it doesn't really matter but they're just uh sequences of integers they're going to represent indexes into other

structures and we'll have one to represent our actions our conditions and our effects demonically a c and e pretty pretty easy to follow uh for Simplicity sake ease of reference we're going to say that our starting action which is always the first one is Sigma and the goal action which is always the last one we're just going to call that gamma all right we need uh some matrices to look some things up in so we're going to have one for Action conditions and this will be a zero if an action doesn't have the condition and one if it does straightforward we're going do the same same thing for whether or not a given action satisfies a condition so you have

these two matrices filled with zeros and ones that represent actions that have conditions and actions that satisfy conditions the last thing we need is our decision variable now the output of a planning problem as we saw before is effectively a graph and so the data structure we can use to represent that is an adjacency Matrix that's kind of the standard well one of the standard two standard ways to represent graphs as data structures anyway so here here's an example one this is going to be our actions as rows our actions as columns when there's a zero that means the two actions don't have an edge between them they don't connect when there's a one they do so in this example case we see

action one connecting to two and three two and three connecting to four four connects to nothing because four is the goal the goal never connects to anything else so the bottom row here is actually always going to be all zeros if you were to draw this like we did earlier it would look like this right so that's our adjacency Matrix now we're in a position to define the objective fun function and this is a function that we want to maximize or minimize uh to help us reach our optimization goal there are a lot of different ways you could approach this particular problem but what I've done is I said I want to maximize the number of

actions I have connecting to my goal State and that on its own is not going to do anything helpful it's just going to have every other action connect to the goal State H so that your the whole Matrix is going to be ones basically uh it's it's not super useful the way that we make it useful is by now applying constraint over what connections are ultimately allowed so we're going to need some helpers and some other things to to make this work first we want a count of the conditions that each action has so we already know which conditions it has we want the count uh so we'll call that a vector we'll call that Vector U uh over

uh the actions that the uh or the conditions that the action has we also want a helper Matrix that we'll call L for whether or not a connection between two actions is legal uh we're going to Define this with uh disjunctive logic so there are two cases where an edge is legal one is if the target action has no conditions then it's allowed to connect to our source our initial state right the other is if the source action satisfies the condition of the Target in that case a connection is also allowed so that's L sub and then the last piece we need is account of for a given candidate graph are all of the conditions of a given Act

satisfied or only some of them and so we count up the ones that are satisfied and so the final restriction we have here x sub J uh that requires the uh Edge to be legal for it to be selected and then it also requires the count of conditions that an action has to equal the count of actions that are satisfied for that condition and if all of these things hold then we can construct the graph it will satisfy the planning problem not as well as the the graph based approach but it works right it works pretty well uh if it was a big enough graph it would become horrifically inefficient and so again you should never do this but you

can do it you can turn almost anything into a function and and this is actually so I have kind of an ulterior motive Beyond functions being fun for talking about them uh show of hands who recognizes this function I try to make it a little bit obvious with the first two but I'm talking about the last one so y equals Sigma Lambda Sig Lambda X hands really all right I I expected way more than that um this is useful who nope wrong direction who recognizes that oh I don't believe you everybody has seen a neural network drawn this way absolutely all of you I do not believe you um so this is the way that we

normally normally see like a basic multi-layer perceptron uh Illustrated it's kind of the standard way but the fact is it's also this right a neural network is just a function when you train a neural network you're optimizing a function you're using different methods than we would use for our graph just now like the neural networks is usually not doing a lot of uh discret optimization for example but they're closely related right we talked about extended family of AI methods before like they they all kind of go together so this is one place where you can see how close these things are to each other even though they can be used in very very different ways so that's

all I wanted to mention okay let's move on to uh more of a it's going to become a real world example it's not going to start out as a real world example so just bear with me you are William Adama commander of the Battle Star Galactica and it is your job to save Humanity from the sylon threat but the syons have just FTL jumped into space near you and they're uh attacking and attempting to wipe Humanity out is that a 10 10 okay 10 minutes that's good we're a good spot they're going to try to wipe you out now you have uh very specific kind of optimization problem you're facing you want to optimally assign all of the

weapons that you have at your disposal in a way that's going to minimize the threat of the enemy that you're now facing right so let's say you have uh machine guns and cannons and missiles and Fighters and bombers and whatever else you might have available how do you assign those to the syon threat to eliminate it or at least minimize it so that you're able to survive and Humanity can carry on well this is actually one of the classic optimization problems it's called the weapon Target assignment problem and it goes a little something like this you have some number of weapon types which called W and for each of those types you have a count C greater

than or equal to zero you have some number of targets we'll call those T and each target has a value that's some real number again greater than or equal to zero those uh combinations of weapon types and targets uh they have these two Associated values so there's a kill rate which is the rate at which a given weapon type is able to kill a Target and then the flip side of that uh the survival rate the rate at which a given Target is able to survive and attack from a certain kind of weapon right that's the foundation here then we have our decision variable which is again a matrix this Matrix though is not a graph

it's just a count of how many weapons of a given type are we going to assign to a given Target to minimize the syon threat and so when you write out the uh objective function here it looks a little bit more complicated than the one we had before which was more more heavy on the constraint side but all this really says is look there are two things we care about the value of each Target and the amount of damage we can inflict on that Target targets that are high value but hard to damage might not be worth selecting targets that are very low value but easy to damage might not be selecting finding a balance between

these things is the the point of the weapon Target assignment problem there is one one constraint we need to add here which is just you can't assign more weapons of a type than you actually have but other than that no additional constraints on this formulation of the problem so this time let's actually implement it in Python so I'm using a library called pyomo uh it's mainly developed by the folks at Sand Labs if you know them um it's what's generally called an algebraic modeling language which means you get to write python that kind of looks like the equations that we just wrote out uh and then it does all the heavy lifting for you behind the scenes and can solve your

optimization problem so here we have to CED all those same variables I just mentioned I populate it with super random data because the data doesn't really matter for our purposes and uh you got to declare that variable you have to make sure that it has the proper Dimensions so we give it the two iterables the weapon types and the targets and make sure that the Matrix has the right shape uh and then we register our objective function and our constraint function but the really interesting part is how we write those functions so you can see the mathematical notation again here on the right but you also have the python version over here on the left and

notwithstanding the fact that I refuse to write code that uses variable names like X and A and C and whatever um it's basically the same thing right you've got a sum over a product you're looking things up with indices but the python code looks a whole heck of a lot like math notation and that's the point of these kinds of languages this code by the way never executes uh it gets uh introspected by the Poma system to figure out what the mathematical structure of the function is and translate it directly into uh the format that we need to pass over to the solver same thing with the constraint right the python looks just like the math minus uh

naming schemes okay so then we have to just uh instantiate a solver I'm using scip uh until early this year it was not an open source option so I wouldn't have used it but it's actually I think by far the best open- source option for hard optimization problems so if you're dealing with like non-convex stuff nonlinear stuff mixed integer stuff sep is really really good and it's aache now so you can use it it's fantastic going to aggregate the results going to print them out and this is what you get right this is our allocation of weapon types to the various Targets this is the Matrix that results so assuming we have sufficient Firepower will be successful

we'll eliminate the sylon threat but of course if you're a BSG fan you know uh that's not the end of the story because all of this has happened before and all of this will happen again okay I promise I would make this relevant to security so let's let's go ahead and do that what what the hell are we talking about out here well if we change some names get rid of weapon let's call it control as in security control get rid of Target we'll call that attack is in Cyber attack so now we have a control attack assignment problem and this is very much the kind of thing that security teams actually deal with day in and day out

they have a whole bunch of things they're being targeted by there are a whole bunch of details of how those attacks work they have a whole bunch of products they pay millions and millions of dollars for and uh they're trying to figure out how do we spend our time in the most efficient way to deploy and manage these products to minimize minimize the risk from the way that all of those attacks work right this is actually sort of a fundamental problem security teams are are working against and so you can formulate the security problem as this kind of optimization problem and when you do that you run into interesting things like oh you know that six-digit standard MFA code thing

that everybody does is actually not very useful it doesn't add much security at all um a fair number of people realize that at this point so whatever no big deal um but everybody loves yubik keis everybody thinks yubik are like the be all IND all of anti- fishing right you can't get fish if you use a yubik key that's a goddamn lie uh so it blocks some things but it's uh completely vulnerable to other things there are no credentials in oo based fishing so yubik doesn't do anything and DNS hijacking like one of the critical features of not just yubik but let's say like uh UF and web offin for example is the domain validation but if DNS hijacking is in

place and you don't have certificates pinned you don't realize that the domain you're talking to is not the domain you think you're talking to you still get you still get like credentials stolen or um certificates intercepted signed messages intercepted uh whatever right and we can just go down the line like OCTA is great except for when OCTA Is Not Great uh firewalls are great except for when firewalls aren't great uh more firewall stuff I spent years working on firewalls so it's a natural example for me that's kind of how it goes right and so if you think about security as weapons and Target it turns into a really interesting mathematical optimization problem and if you solve it

you will find some interesting and unexpected approaches to doing the work of security like boots on the ground work of security and uh you can reduce the risk to organizations by pretty significant amounts how am I doing on time five minutes oh in that case we're going to go ahead and talk about logic programming I did not think we'd get here so this has not been rehearsed um logic programming it's it's really cool it's a kind of a meeting in the middle of uh the previous two systems I tried really hard to get a bot that looked like Benedict Cumberbatch and I I failed mid Journey just wasn't having it so I'm sorry about that but I still

think this version of Sherlock looks pretty cool uh the idea of logic programming is pretty straightforward you Define relations and then you can these are symbolic relations right and then you can substitute concrete values in for parts of those and you can basically just Traverse them in whichever direction you want so if leave things completely symbolic like a plus b equal C I can ask it for uh Well turns out to be an infinite sequence of substitutions for those symbols that will satisfy the relation I've established and so it'll just do a and b and c equal these things it'll it'll run Forever Until the heat death of the universe right but I can also substitute

uh some values in and then it will give me answers that achieve those values it's doing a lot of the same kinds of work of other systems but it's built into the language itself when you do it right so um if you've never heard of any of these there's like prologue or the subset of prologue datal log um there are a couple languages that Google has built yetal log was the original but the current one is called logica and they use that in production for their Knowledge Graph you do lots of cool stuff with it um one that I really like although it's uh built on top of scheme so it's very lispy and if you don't like

parentheses I would advise you to stay away uh it's called mini conin uh it hasn't been around that long but it's super cool there's a python version uh so what I did here I definitely don't have time to talk through it but what I did here is once again take our planning problem show how to represent that as a series of logical relations and have it be able to navigate those to Output a planning graph based on relations and so you can do that by declaring uh the facts uh that we take from our actions and everything defining relations like okay well here's how you say that an effect satisfies a condition here's how you say that an action or one of the

effects in an action satisfies the condition here's how you can say that an edge is valid just like in the equations we saw before there's a logical disjunction here so there are a couple different ways to do that um you can do recursive relations so you can define something as being an ancestor if it's anywhere behind in some kind of chain you can use that to build the concept of reachability so from our start or our goal actions in a planning problem is a given action reachable and if it is we want want to use that for certain things um is an edge between two actions reachable well to say that it is we need

both of the individual actions to be reachable and so on you throw all that stuff together and you uh hit print and it will build your planning graph for you again it's not perfect it's not as good as doing it the uh quote unquote correct way the oldfashioned way it's kind of like a weird hacky way to do it but logic programming is pretty cool especially for exploring data I thought I had five minutes five minutes ago man you you're really confusing me but that's good this means we're going to have time for Q&A so I rush through the logic stuff happy to answer questions on it but before I do that my robots are

hella cute and so you should applaud them thank you very [Applause] much do we have any questions here I your guess is as good as mine man real quick I'm in AI governance so I know this stuff but awesome we've seen a lot of things in agent based modeling uh being promising I just curious if you had any thoughts on that I'm sorry can you read the second part oh agent-based modeling are what are your thoughts on that cuz I'm seeing a lot of value coming from that for uh controls and risk are you talking specifically about like the the current sort of rash of concern around llm based things and prompt injection or just

generally like in financial uh Financial crime it's being used to find things that wouldn't be there absent the agent based modeling synthetic data things like well give give me an example oh okay well what you do it's be kind of hard to say it in short term but agent based modeling You' heard of synthetic data MH okay basically that that's it's created by the same method and what you do is you you take an existing ground truth training data set and you learn from it and you add other features and you just you create these little agents they run around and it's kind of like doing um virus prediction you have these agents that run around in

the society and then you can determine where the virus is going to go and how fast and how much same thing with financial crime just let it cut loose you have these little agents that that run off and do accounts and they're evil some are good and then you see how it all kind of comes together so anyway that's something I saw I've seen a lot of I just curious if it's not something you have a direct yeah no it's it's it's a it's an interesting approach I mean we see people do that kind of simulation based stuff um in a lot of areas one one of the things that I think is cool about

logic programming actually is that you can build up um a logical representation of a network literal Network or something more figurative like a financial model and then reason about where gaps might be exploitable within that model and uh the way the things work like just substituting for symbols right you can just iterate over all the gaps that it finds so another approach to that but yeah I mean just basically turning agents loose and letting them bang on things is of course a useful approaches that's kind of like uh the chaos modeling stuff that Netflix was famous for 10 15 years ago yes sir yeah cover a lot of ground so

ums yeah so that's a really good question right we dealt with a very simplistic version of the planning problem like just Boolean variables and uh actions always successful so the question was how do you plan under uncertainty which is a great question because literally uh so the the seminal text uh in the AI field for undergraduates at least is uh AI for the Modern Age by uh was it steuart Russell and Peter norvig um and there's a chapter called planning under uncertainty it's all about this this kind of thing and uh there are a lot of well there there are a few different ways to approach it uh one of the things I like about this kind of of model is

you can just uh let it fail when you're doing everything in parallels so in the security context as I constructed it uh if you run into a roadblock you can just backtrack and go down a different path but as you get into more abstract kinds of States like coordinate systems and other things that like Robotics and realtime systems use that starts to turn into a quite problematic thing to be able to deal deal with so like the uh the self-driving car example they run into lots of issues there they end up having to use a little bit of machine learning to uh filter down some of the planning possibilities so that the actual decision- making system system

which is based on Reason Not statistics can make a smart choice but yeah it's a it's a real problem and here you go I think they're on the fourth or the fifth edition at this point yes sir interesting he follows on from a question I have is so these are great when you actually have a reasonable Assumption of what the constraints are with logic Bas programming or that you know what the um what the connections of the graphs are right but as we know in security you know a hidden constraint or a constraint that's missing that should be there causes us all kinds of problems so it's great when you can Define when you can

Define this space well what kind of solutions or approaches have you got when you think that there's kind of variability in the space or maybe constraints that you don't know or um parts of that you know your waiting or your connections within a graph that should be there but you're missing yeah so those are the cases where I would generally make up data right I would estimate what I think some of the unknowns are and generate synthetic data around it and then feed it in the same deterministic kind of system or if I'm using machine learning which I still use a lot uh feed it into something like that too but uh at some point you have

to make some kind of assumption you can't build over the void and uh yeah I mean you can look back to history you can do all kinds of stochastic things um Monte Carlo stuff like you have options there but at the end of the day there are guarantees like the things you make up might not help so that's a tough one all right I think we are almost certainly out of time yeah all right need to cut it thank you all for coming there's my email if you want any of this code or you want access to the slides or anything just shoot me an email happy to share thanks guys