Decision Analysis Applications in Threat Analysis Frameworks

hi everyone thank you so much for coming to my talk decision analysis applications and threat analysis frameworks again I'm Emily Shaw go just a little bit about me to get started because it kind of informs where this talk came from so this stemmed out of my thesis research from my master's degree I just received my Master of Science in public policy and management with a concentration in cybersecurity management from Carnegie Mellon University this year prior to that I actually got my bachelor's degree in psychology and political science from Carlow University in Pittsburgh so I kind of took a little bit of a pivot in my career path which sort of allowed me to bring the different perspectives into

this research so just a quick agenda here I'm going to go through the background of how I came to this point of feeling this research was important my methodology myself performing this research the methodology of the framework that I have developed some future direction is that I hope this research can take with me or with others in the future and then I'll wrap up and if we have time I'll take some questions so one thing that is tends to be a problem in cybersecurity is that human factor and historically the human factor has not been considered enough in security that's starting to change however still to this day when we think about the human factor insecurity we

think of the human as the vulnerability so we think about the computer grandma who clicks on the phishing email you know that sort of thing so the human is very much a part of the vulnerability end but unfortunately the threat actor is still that hacker in the hoodie with no face we don't really think about the threat actor as a human who is making decisions in real time and the fact that you know we have imperfect information as defenders but they have imperfect information too and how do we use that to our advantage in security one area of security that's really becoming very important is research on apts advanced persistent threats and these are threat actor groups that are

usually attributed to attacks either because they take credit for them or because their mo is consistent and their consistency has allowed them to be a great subject of security research and finally because it's an interest of mine I wanted to look at the behavioral and decision analysis aspects of security and one a very important area of that I'll just give a shout out to here is the game theory perspective there is some research out there on that that I found that was very interesting it's also very math heavy it's not something that your average person in security or in any field other than economics or math can look at and say oh I understand how this is relevant to me so that was

something that I felt was a gap that I wanted to begin to fill with this research so the first thing I had to do is kind of find an approach to take with this research to start with I wanted to get access to aggregated pentest data because pen testers emulate thread actors they emulate all different kinds of thread actors so that would have been a great way for me to look at and say oh here's what's happening here's something that like an organization will sign off on because it wasn't you know a real breach or anything like that turns out that's actually very hard to get an organization to sign off even anonymize I mean I had great contacts through my

University who did everything they could to help me get this data that was not going to happen so the next thing I tried to do was what if I could just get one what if I could look at one set of pen test data from one engagement would this work no no one would let me use this data so I kind of had to take a step back by this point I had committed to this as my thesis and I'm starting to panic a little bit I'm thinking man how can I do this that you know I can still reach the desired object and so basically what I ended up doing was taking a very theoretical approach

using publicly available information so what I ended up doing was I decided to focus on a PT's because of the consistency of their behavior because that's what a lot of other organizations use to develop threat frameworks and I'll talk about that here in a minute so I decided to take just some specific examples so I looked at apt 10 and hurler group charlie group I looked at because it's very simple their attack strategies are usually very straight forward that's going to be the one that I'll demo here in a little bit because it's shorter and I can do that apt 10 I also modeled just for sheer magnitude because they have so many attack vectors and so many different

things that they do I haven't included in here just for the magnitude of it it was unpleasant so I wanted to incorporate elements of existing frameworks rather than reinventing the wheel so the first thing I looked at is of course what everyone is familiar with the Lockheed Martin in cyber kill chain seven steps it's very linear it's usually only applied to a PT's generally is a very good model to stick to but it has many flaws that are kind of being addressed and one of the ways or one of the frameworks trying to address these is the mitre attack framework which I'm only looking at the actual attack not the pre attack this was actually just

there was a talk on this yesterday it was very interesting and I shout out to them but so basically it takes those last four steps that are effectively the same as the kill chain but it goes into more detail so what are those specific categories of what a threat actor can do once they're acting on your framework so basically what I wanted to do was I wanted to take those existing frameworks I wanted to incorporate that aspect of human decision-making and the fact that attack might not be linear if you foil one step of an attack that doesn't mean you're done and that was something that while it's alluded to and the other frameworks it kind of means that like

you have to almost start from scratch so I wanted to think how can we not start from scratch how can we take probability or other things that the the threat actor is doing and kind of include those so the first thing in an organization to apply this framework is identify top threats so I picked my threats just as ones that were interesting but for a particular organization you'd want to look for ones in your sector for example there are specific threats to the financial sector or to government something like that perhaps to a particular geographic area or to a size of an organization and there's reports being put out every year on what those top threats to look for are and then the

next step is to research that group's attack methods now that's something if you're working specifically with a PT's there's you know great repository on that from rapid7 from mitre from types of organizations like that if you're looking at something a little bit more obscure you might have to work from your organization's own historical data were you breached in the past by an attacker is that something you're worried about again what did they do so then the next thing that you'll do is to form the decision tree which I'll show here in a moment to understand what the primary and secondary attacks so for example perhaps a group's initial attack vector is usually spearfishing but they have other

things in their back pocket if that doesn't work and perhaps once they've fish their way onto your network they have something that they might implement on your account but if they can't do it because you don't have privileges then they try to move to a privileged account things like that so once you have this street put together that's where you start to work within the organization to synthesize with known weaknesses in your group so here I'll just actually moved this so this is the example decision tree that I made for Turla group I hope you all can see it from here so basically what I did was I started pretty much their initial tag vector

attempt as always spearfishing that made them very easy to model so if they're successful then they move on to the step of process discovery etc and so forth and if they fail then they move to the alternate method of trying to brute force credentials so basically as you're moving through this tree what I went ahead and highlighted here in red is the points at which the attack would be considered failed if they were unsuccessful because there'd be nowhere else for them to go and then in green I went ahead and highlighted at what point would you be breached would their attack be officially successful so the way this can be used within the organization is

to then look at would you would your organization have methods to defend against things within process discovery or lateral movement are those controls in place if those controls aren't in place then you know that that is something that is of concern within the organization and should be a priority for mitigation I also feel like this was something that I kind of came upon as I was working is that the critical nodes are really those places where you have a red and a green so those are really the points where you can say okay if they reach this point and they're successful then I've been breached if they reach this point and they fail they're done so

those are things that are I feel like as an organization it's really important to focus on now here's the tree that I went ahead and made for apt 10 I put this up here just to show how complex this can get I don't expect you to be able to read any of that because the print super tiny but this was actually even a slight simplification because spearfishing isn't always their initial attack vector but I went ahead and did this assuming that it was and even with that we're looking at many many layers so this is something that can really get into very high orders of magnitude very quickly but it's still all functioning in that same way so each

branch ends with that critical green red node and there are many places of attack failure along the way to look at so some of the limitations that I encountered while working was one of them was obviously I didn't get to work with live data I didn't get to work with something organization specific or with a simulated attack that had actually occurred that would have been something that someone could work with in the future and would be very interesting because then you could see how an organization could actually apply this to their own situation instead of just working with a general apt also this is very theoretical and it lacks math foundations I am NOT a statistician or a

mathematician I'm very interested in those fields I feel like a future direction for this would be to start to include things like probabilities that is not something that I was able to do with this time and if anyone has expertise in that I'd love to hear from you sidenote so really then the question is what can organizations gain from this so like I said it can be used for prioritizing mitigations what are those critical nodes where an attack will succeed or fail and prioritizing those as top concerns and also within that just also seeing what are the other steps along the way so the it's also important to know not to get too caught up in those critical endpoints because

if you can stop them earlier in the attack all the better but really to kind of consider all of those things it's also you can personalize your corporate policy in this way so really to say I'm gonna look at these threats these threats within this sector are the most concern to us basically at that point you're not just working with so um one of the things with the attack framework is you have this big grid of a whole bunch of attack that's that can be very overwhelming this way you've simplified it to a very linear process how can i end this branch how can I end that branch how can I stop this whole tree entirely and although

I've modeled it with a PT's I can't stress enough it can be used for many other types of attackers as long as you have some data either from your own history or from open source or from a partner that might have access to that you could very easily model the insider threat or the script kitty you could do those things very easily all right so at this point I'll open it up to questions I also have my Twitter on here my DMS are open it's just my name if anyone wants to reattach me with questions later

so first of all things for the talk to questions or comments the first one is based specifically for the epic is that you have kind of a history record of their incidents do you think about giving a weight talking about the probability of moving between one leaf to another that's where I would love to have someone with more of that statistics probability background to work with on this that's a direction I would love to see it go and I tried a little bit to do that but I did not have enough personally I did not have enough confidence in those numbers to include that into the actual presentation yeah so yeah absolutely that's one idea the

other one is you have used the binary decisions kind of trees I can see you know cases where you might want to look even at multiple kind of options for each and every steps rather than just do I did not include one where I did this I did actually have some that I played around with where I had three or four as a tree actually I remember now I didn't include them because they were just huge and you couldn't read anything but that's absolutely yes that is very true that like this is not necessarily binary there's always more than two it says great Jim

so thanks yeah fantastic talk I just wanted to mention I've just been in a talk by Fox IT Paul polls and he was talking about a unified framework that brings together mitre the kill chain and some red team activities to fill in the gaps between those frameworks and that could be really useful to taking this to the next level also seeing how things branch between trees because he made the point that very rarely from a kill chain perspective when you read team do things happen in that order be fascinating to take that kind of analysis and look at it against that and if you want to talk about someone who might have some data to play with thank you find a few months

then yeah let's definitely keep that conversation going because that was a great presentation I'd love to use that framework thank you

Thanks if there's really good idea that you have here what would be neat is to add some percentages of impact so the cost of these different type of breaches at these lots of information at this point in time but what's really really neat about this is that it gives you the ability to communicate to business leaders in a way that they can understand it relate it to the money because that they use this same type of framework and game theory like you mentioned as well as cost analysis for you know if you're gonna buy this company versus buying that company versus you know it's great this is really a good way to communicate to business leadership yes hey great job

I'd a question about like how you um like how long did it take you to do this once you have like this source data and you're like you want to go from this source data into having you know one of these decision trees built how long did that take and do you think like is that something you could operationalize over time in a way that's like cost-effective so for me actually it really didn't because I I basically just pulled from the two sources of mitre and rapid7 for that data on the attack vectors so I mean I did one of these in a day the more complicated ones I took a couple of days just because I needed to walk away

from the for a little while but generally I would say it all depends on how good your input data is if it's very compartmentalized and it's very easy for you just say oh here are the you know three initial attack vectors then that's a very quick thing so I think to answer your question I think the operationalizing comes not so much from building the trees as from inputting that data if you have a good repository it's very quick to do

thank you so for your perspective how well do you think this will scale such they can still be analyzed visually and have you considered options for analyzing it once it's too big to analyze visually sure there are decision tree software's out there that could be used to kind of produce a comprehensive report that isn't the visual tree I like the visual tree I think that's very good to hand people who aren't mathy aren't really into the nuts and bolts of it I think it very much depends on the situation certainly if you're working with a lot of uncertainty a lot of unknowns for example if you're working with something that's more of a broad vector of the insider threat for example

that's very very broad and you might find yourself dealing with something with too much of a scale so I think I think what's really important is just getting everything as specific as possible so that you have that good repository of data and you have a very narrow scope when you say just yeah but when you say specific are you referring to the ontologies you used to quantify the notes yeah so attack or whatnot yes so I think if I go back here I went ahead and specifically mapped this to the terms used in The MITRE framework for example so I would say you know brute force can mean a lot of things so if you're working in a situation where

you have to protect against multiple types of situations then maybe you have to make that more specific so that kind of is your question now do you vision this potentially what do you think it would benefit or do you think it would have a negative impact to go beyond tree grafts to either a cyclic or cyclic grafts the more general structure they'll be very interesting I think the the benefit of this is that it still kind of sticks within that linear visual format that people are kind of used to seeing with things like the kill chain I think the cyclical would be more compact so I think that's very interesting to see because you know if

you fail one thing then you go back a few steps to try again

again great talk one thing this kind of reminds me of and of course every dimension in probability is Bayesian Nets mm-hmm and you know if you one thing that's often done an amazing nets for things like this is to annotate these with controls and mitigations pointing into some of those attack vectors so you can then show what controls mitigations you have in place and how those reduce the ability of an attacker to successfully exceed those so that might be under the way to to bring something probability to this yeah great point thank you

it was a guitar um so I'm kind of new to the cybersecurity part of the kill chain but kill chains already use other places I guess and my experience has been with electronic warfare and I'm just wondering if if you've had any exposure to that I mean then you know electronic warfare has been doing kill chains for quite a long time and I I just wonder if there's some maybe experience or or ways of thinking about the kill chain you know we spend a lot of time trying to figure out how do I break various links and you know there's lots of different ways to present that information and I would say it might you know it might be

hard finding somebody who won't wanting to talk to you about that but I still think there's many plenty of information you know that's out there on the web I mean it's funny that Lockheed Martin is the one that you know documented this kill chain because obviously they are involved in you know electronic warfare stuff and they do it there all the time so anyway just a an area another area may be it is just to see what lessons learned came from or half could come from that that might help you in the presentation and figuring out where the weak links are thank you I appreciate that I don't know anything personally about that area but I'd love to hear

more about how that could be applied thank you

all right thanks so much [Applause]