Kelly Shortrige - Big Game Theory Hunting: The Peculiarities of Human Behavior in the InfoSec Game

so it's called big game theory hunting the peculiarities of human behavior in the InfoSec game which I understand is a little vague if you don't know Marian she's like an amazing reverse engineering malware analyst she's not working at Intel actually and she had a great talk called I think it was big-game hunting about like I think was like again reverse engineering malware and so I wanted to riff off of that because there's a great presentation so I'm Kelly I'm a product manager in the InfoSec world so I do the like strategizing and vision stuff related to building the security products I'm really trying to think about like what's the actual value that people need as a

security practitioner rather than just like bolting on features I spend most of my weekends and evenings looking at things like behavioral econ lately looking at like cross-domain analysis from like climate change looking at different models resiliency to see how it applies to infra sec I think in general cross-domain research is super important people don't do it enough like not just an info ii 'try domain but it turns out there are lots of smart people in lots of different areas so it's good to learn from them and this is game theory um so the talk is about Game three but it's really going to be about behavioral game theory but a traditional game theory like Game three or 101 you

see a lot of times particularly on Twitter people conducting these sorts of analyses it gets really old really fast and I don't think it's a good framework for us so it's time for hunting some game theory if you aren't familiar there was a meme I think during the election where someone said something that was not related to game theory and said it was time for game theory and it's been kind of a meme ever since so it's actually time to debunk game theory so question for the audience do you believe bug free software is a reasonable assumption yeah yeah thumbs-down correct and do you believe wetware like our brains is more complex than software yes

okay so traditional game theory relies on the assumption of bug free wetware so this is a terrible assumption behavioral game 3 assumes there's no such thing as bug free so like most of you in the room if you're kind of offensively minded you probably agree with this assumption mary gomen who was a Nobel Prize winning physicists also said think how hard physics would be if particles could think that's why I think it's a really interesting problem area then Dan query Dan Gere he was quoting someone but he said amateurs study cryptography Professional Studies economics so that's a nice pat on the back for me and this is what you're going to learn today why traditional game theory isn't even a

theory and is also unfit for strategy making a new framework that I'm proposing for modeling the InfoSec game based on behavioral based insights and finally new defensive strategies that exploit either your adversaries thinking or learning and I'll explain what both of those are so let's go hunting to find out why so first what is game theory actually discovered this after troopers when I went in I kind of assumed everyone knew what game theory was because that's definitely the impression you get when you kind of go to conferences looking at InfoSec Twitter turns out nobody really knows what it is but we like pretending like we do so this is a refresher you don't have to

admit whether you do or don't but I think it's helpful so game theory is a mathematical language used to describe canary scenarios a conflict and cooperation so it really is more about language in theory it doesn't actually based on fact it's all theoretical and thinking about these imaginary games so I use it as an engendering tool like really something off of which so like think about other things so you can dictate optimal strategies so again that's why it's a language you can frame what a certain game or a certain conflict looks like but it isn't the theory of the conflict itself so game theory applies whenever actions of players are interdependent so basically I make a move like your move is going to

adjust or be different depending on the move I make so different the different strategic scenarios include a bunch of different types of games which I'm about to enumerate as well as to print solutions for each depending on the approach you have so first zero-sum games so for example in chess there's one winner if well one loser it's pretty straightforward in poker the amount won by someone is equal to everyone else's losses non-sterile some games so some people think the stock market is as your ISM some came but it's not like everyone can benefit everyone's like 401ks can increase though there's a separate discussion about how passive investing in index funds are may be problematic for that

area basically as the economy expands like everyone can benefit so that's why it's nonzero-sum the prisoner's dilemma is also nonzero-sum I'm gonna dig into the person er still and I'm just very briefly in a bit but it's not like one person wins one person loses and the like loss and gain completely cancel out there are negative some games so like nuclear war both people do not benefit at all everybody it's much worse off climate change is also kind of like a really terrible game of chicken and everyone is worse off but they're positive some games like free trade is often thought of a positive Sun game because everybody benefits it's not well Green Administration aside like most

people assume that like it's not a complete like one negative one when you're conducting free trade then also equal rights that's when when everyone's better off there's lots of empirical evidence for this too so there are also in these games again thinking about game theory is a language one of the their different types of information so complete for it's incomplete basically you know the types of opponents you have and their motivations incomplete information incomplete is when you don't actually know who your adversary is and what their motivations are so this case you know the type of big kitties that you are facing perfect first imperfect information so perfect information is you know the moves that

the other person's making in imperfect you don't then finally information cemetry versus asymmetry so in symmetry each like opponent has the same amount of info about the other opponent and in asymmetry like they each have different levels of information about each other there are also defender attacker defender games also called dad games and these are sequential games in which setzer players are attackers and defenders as you might expect the Foundation's are in traditional game theory attackers in the common theory you want to be maximally harmful a lot of times it's a terrorism use case they're looking at like what's the most amount of destruction that I can cause they have some sort of specific goal in

mind obviously that's not really true in this domain and many other domains so it tends to be very narrow and then it also assumes people are risk neutral and it's like I said want to be actually harmful so both of those assumptions don't really hold in the real world but the first move is defenders choose some sort of defensive investment plans so they construct like a big wall like whatever you want to think about it but they make the first move so then the second move is attackers observe the defensive preparations let's say like hardening infrastructure and they choose their attack plan so like I said the goal they often think about is like for example

there's a lot of stuff with them what is the Konan burg bridges and it's looking at how can you bomb like this one bridge but nash equilibrium is also used to solve these games and this is bad because the nash equilibrium is actually pretty simplistic and not very accurate for the real world the nash equilibrium is looking at the optimal outcome of a non cooperative game so basically when you have two players trying to beat each other it's it's good as a concept to analyze whatever the outcome is but it's not very good for anything in practice so players are making the ideas players make the best decisions for themselves while taking their opponents decisions

still into account so thinking about how the other player is going to play but it's it's not really thinking long-term either I should clarify so the prisoner's dilemma the basic idea is that in an ideal world everyone would stay silent so it's basically you and your partner get arrested now the police are questioning you you're not allowed to talk to your partner what do you do so if they both stay silent they'll serve a really small amount of time that negative one negative one they both confess it's say like two years and two years but then if one person confesses the other refuses whoever stayed silent actually gets four years in prison which is worse so even

though the optimal outcome is that both players refuse in practice they're probably going to confess because the if you look kind of across the row it's still more advantageous kind of as a whole for them to confess than not it's the problem with the Nash equilibrium is it's based on a priori reasoning so it relies on theoretical deduction verse observation or experience it also assumes rational all-knowing players a lot of times people don't even know the game that they're playing I think that true in the InfoSec game though I try to codify what that means then they assume others decisions don't affect you but that's super unrealistic because in my opinion a lot of times if someone makes

a certain move it can cut off entire like decision paths for you as well but a lot of people have applied the Nash equilibrium to InfoSec over the years particularly at the more I will say business minded conferences you'll see a lot of talks about how Nash equilibrium justifies this product or this vendors existence just not very rigorous so some of the conclusions that I was able to find just from a Google search defenders should play extremely fast so the attacker drops out of the game like for any defenders in the room like the idea that you constantly are playing faster just totally unrealistic another conclusion which was probably my favorite is that it's better to invest

in some sort of security but not to invest regardless of the attacker strategy so who knew security is important yes yeah so the idea would be I'm trying to they didn't they were very example light but I think the idea was like for example if you think that an attackers in your database it would be like just shut down all of your network and then figure out what's going on it's like immediately even if you aren't sure so and then the other thing is like applying tons of mathematical equations to the problem which no defenders ever gonna do so so a new defensive framework the idea here is that now that we know game theory is a language used to

describe some sort of game or conflict it's not just a theory we need to explore like what sort of framework does work what sort of theory does work and I'm going to be leveraging behavioural insights so using game theory for its expressive power I think is really important so it's also important to look at data outside of game theory so again looking at kind of cross-domain analysis like what insights can be cleaned from other domains when they've been researching this sort of stuff but first I think it's important to figure out what game is InfoSec I argue it is a dad game so it's continuous defense and attack it's nonzero some I think sometimes people think of

info ii zero-sum but i really don't think so particularly in the case of like nation-state espionage realistically it's probably not going to affect your day-to-day operations very much it probably won't affect your stock price i mean maybe if you're reachable like Equifax is the first time it really affected the stock price of a company so it means to be seen if that's kind of it's an unique case like in target's case it affected their earnings I think one quarter but not very much beyond that then I think there's incomplete imperfect and asymmetrical information I don't think we know nearly enough about our opponents we can't track their moves we don't even know most the time like

what type of attacker is and we certainly have asymmetrical information it's also sequential and dynamic so it's not just like one round and then you're done it's a kind of ongoing process I also think it's a uniquely potentially tricky game the closest is like a piece war game in the sense of nuclear conflict though that's don't compare it to nuclear conflicts it's completely different but it's similar as far as the opacity of information and that is sequential so another question have you heard InfoSec described as a cat-and-mouse game yes fun fact until last week I had never really thought about what this meant and I always assumed that the defender was the mouse but it's obviously the cat he was never

actually able to catch the mouse so that was me a couple but I know now but the problem is traditional game theory doesn't actually allow for those or most characteristics of the InfoSec game because it seems people are rational and they aren't and it seems static versus dynamic environments so again you have kind of like you think about the cat and mouse they're always chasing each other traditional game theory doesn't assume that and also the fact that you can't ever be one step ahead of your adversary I think a lot of people recognize that a lot of times attackers are one step ahead of defenders but traditional game theory doesn't allow for that then also just

empirically speaking deviations from the Nash equilibrium are everywhere but the problem was traditional dad games extrapolate from Nash equilibrium making it hard to model this so the first time that behavioral insights from a dad gamer like a behavioral econ Studies sudden gank dad games came out was in March and there hasn't really been much sense in so I think it's gonna be something hopefully we see more of over the next few years even John Nash said this I feel personally that the study of experimental games is the proper route of travel for finding the ultimate truth in relation to games displayed by human players so I think if he even understood it I think it's important to think about

so the behavior based framework so I think it needs to be experimental how do people actually behave and then it's also good to think about the fact that people just the way our brains work we predict our opponents moves by either thinking or learning which I'm about to explain so thinking is just modeling how opponents are likely to respond so this is before you've made a move it's thinking about what they're going to do our brains work like volatile memory so it's kind of like DRAM which needs constant refresh cycles and it's not very efficient so working memory is a hard constraint for human brains it's very hard for us to have a large number

of like higher-order beliefs maintained so thinking about say 10 steps in the future just as far as we're thinking about it and not writing down it's very difficult for us to do so a lot of times we only think about what's the immediate kind of step ahead and then humans kind of suck at recursion so it's increasing the number of steps really strains our working memory so it makes it really difficult to process them learning however is predicting how opponents will behave based on their behavior in Prior games around so this takes over once you have prior experience to leverage you instant to learn through error reinforcement learning which just means trial and error learning and then people

have learning rates so basically how much experience is factored into decision-making there's also a very very small random variable they're just much smaller than people think a lot of it is kind of like the past experience that we have but there's enough that clearly for our lizard brains they wanted us to keep trying something a little bit new so it's a very small factor there then something I find cool which is not gonna be exploring at all in this talk I hate to disappoint you is that dopamine neurons and code errors so I think there's a lot of interesting stuff to think about as far as the economics of attackers and defenders but I don't have the money for like an MRI

machine so someone does have the money please give it to me and I'll start to do cool studies but until then no dice so as a case study there's a study by of X alone Bou Claire which basically not an InfoSec but looked at - and consecutive security games and the traditional kind of physical security context across four different strategies the idea was that you have different learning rates for attackers again how much they learn from prior experience and they tested the number of prevented attacks for each strategy so basically there was a four by four payoff matrix matrix across those four strategies they were just going to examine the number of prevented attacks based on that matrix

so what they found was a thick strategy prevented attacks ten to twenty five percent of the time which is like really really terrible a game theory strategy prevented them about half the time a random strategy also prevented them about half the time then the cognitive modeling strategy which again is thinking about like what was the prior behavior of your opponent prevented sixty-one to seventy seven percent which in numeric terms 22-250 four more attacks out of the 200 so that takes into account the fact that again attackers will favor certain options or avoid certain options depending on whether their actions failed or succeeded in the past there's a paper I have in my references called know your

enemy you can see that as a reference so what it means as attackers aren't completely unpredictable they're also not like I guess normative decision makers oh sorry

it's not been nearly say like I said there's been no study of behavioral economics period physec as far as an academic study so no nothing has gone into that well I also appreciate the questions I do it there's a lot of material so I encourage I will try to pause for questions but I'm also very aware of the fact that this is a very like 50 to 55 minute talk yep but in general there's not enough data like an info SEC at all I think for behavioral related stuff the point to that to me is like don't be replaced by a random security strategy algorithm if game theory is just as good as a random like

you might as well kind of like do whatever and see if it works so the implementation so a lot of this is nice in theory but how do you actually do this on a practical level so I'm gonna walk you through five parts the first is a SWOT analysis which you see you've seen Silicon Valley I sometimes like to think of myself as like the Jared of InfoSec like kind of doing the boring things but I get really excited about it so we're we're gonna do a SWOT analysis as well as a perceptional SWOT analysis which is super fancy then going through thinking exploitation and learning exploitation then the concept of minimax and then finally looking ahead for what

would be awesome to do in the future so a SWOT analysis the 101 of a traditional SWOT so it's developed by a Harvard Business School in the 1960s it's a like super common in business strategy in general so it looks at the objective and organization or projects to see like what are the strengths weaknesses opportunities and threats strengths and weaknesses are all internally based so like what are the internal factors that make up your strengths and weaknesses opportunities and threats are taking in mind like the external context to see like again what are the opportunities you can pursue what are the external threats to whatever you're working on so I think you need to model a SWOT for

yourself in relation to your adversary but also do the reverse the adversary in relation to you again trying to really think about like how your opponent is thinking so a fantastic game theory professor at NYU named Adam Brandenburger with whom I had the pleasure of speaking one of his quotes is the primary insight of game theory is the importance of focusing on others of putting yourself into the shoes of other players and trying to play out all the reactions as far as head as possible and in the vein of Toby's talk like you can play devil's advocate that's exactly what you need to be doing in this case so this is just a very kind of

high-level example swat4 kind of like a traditional defender some of the strengths basically you have an understanding of the target environment hopefully because it's your environment you have a motivation not to be breached I think that's definitely a given the weaknesses are you may have an adequate budget that's kind of changing given the level board focus but I think budget problems are something most people have experienced a lack of personnel I think everyone's familiar with there aren't enough people to hire InfoSec then finally limited employee training thinking about you know employees click on phishing links whatever else they are learning properly that sort of thing is kind of an internal weakness the opportunities are leveraging things like

containers to allow for easy tear up and tear down I just gave a talk about resilience which talks a little bit more about that so I encourage you to check that out if you want to learn more about that sort of stuff and then as I mentioned there's increased support attention to get more budget the threats though or attackers can also use new tech for scalability like I know a lot of what we've been seeing is they're using like Google services Amazon obviously stuff like that even Dropbox as far as like new ways to deliver attacks which is bad it's also really hard to keep up with the pace of new attack surface it's hard

enough to kind of have asset management and then as you get more devices more software whatever else makes it really difficult so perceptual SWAT this was developed by the same NYU professor so the idea for perceptual SWOT is that things like core competencies can actually be weaknesses weaknesses can be strengths so if you're familiar with judo it's someone sometimes called the judo strategy so basically like how do you use your opponent's weight against them so thinking about yourself first the other what are the perceptional swats so you need to think about what do you think is a strength what's your perception of a strength how is that actually weakness and then that's gonna be the reality and then like what

weakness is actually a strength for your opponent obviously what the opponent thinks or what you think as their strength could actually be a weakness and that presents an opportunity than what you think is a weakness or they think as a weakness it's actually a strength of theirs the idea here is um so Andrew Grove actually the former CEO of Intel said that for example the biggest core rigidity is actually top management management went through prior conditions to get to that point and they don't exactly understand kind of the new realities and the new challenges that are being faced so basically the moment that the environment changes the people at the top are the wrong people to be

there so what you think is a strength is actually a weakness so I think things like compliance and fix security guidelines can sometimes be seen as a strength to at least get a bare minimum but it's also a rigidity right and then moving on to attackers I think a good example is an attack core strength is having the time to craft an attack so like mikus talked earlier like a lot of it is about patience problem is you can also leverage that strength with strategies that lead attackers down rabbit holes and waste their time another one is having access to known vulnerabilities the idea is you can confuse them with fake architecture or fake looking systems so they can't be

certain what you're actually running so thinking exploitation so by talk it's your nights is mostly going to be about these sorts of tips and tricks like how defenders can also exploit things particularly their attackers way of thinking through problems so this is a part of that so belief prompting the idea is that you just increase the players thinking by one step it's part of the idea of as I said before like you can only have so many steps in your head so this is how can you improve that so what you do is you prompt the player to consider like how their opponents who their opponents are and how their opponents will react over time and I think for security you have

to model assumptions around the capital time tools risk aversion and as Toby said like you need to think about like is there a specific sort of adversary you're looking to emulate and think about you can't just think about kind of a generic attacker without having kind of like something specific to test in mind I take this idea further which is that you should bottle out just entire decision trees which I'm going to talk about shortly and general there's been a lot of research to show for example I think it was that if if someone's trying to predict their opponent they'll get it say right 30% of the time traditionally speaking 70% of the time it's wrong but

if you have them perform some sort of belief prompting that actually switches and they get it right 70 percent of the time so there's good empirical evidence for this so your goal is to ask if I do something an X how will that change my opponent strategy how are they going to react so a generic belief prompting guide how would attackers preemptively bypass whatever a defensive move I make so if I implement two-factor authentication what will they do instead what will the opponent do next in response what are the costs of the opponent's offensive move like something super fancy probably they're not going to do they're gonna try something simpler like fishing at first then also

what's the probability the opponent will conduct the move but just also can be based on things like cost or time I think additional things like how do you think our adversary chooses their delivery method obviously what assets are they going to go after what countermeasures will they think that we have and then as an example let's say a script kiddies lands on one of our servers what are they going to do next let's say they perform local reconnaissance they escalate to whatever privileges they can get the counter would be something like privilege privilege separation and don't hard-code credentials which we've seen recently people still definitely do this leads to the attacker has to exploit the server and there's a risk

that the server crashes so you're slowly raising the cost of the attacker that way and thinking through everything that they're going to do so with decision tree modeling which is attempting to codify that whole process so they're just representations of sequential games in essence which i think is better than the payoff matrices that show kind of a static point in time so what you want to do is model these decision trees that include both offense and defense and I'll show you a visualization of them so it's not too conceptual you also want to theorize the probabilities of each branch which I think first you want to use kind of like fuzzy estimates which i think is totally fine you can refine

those over time based on whatever data you see or other hunches you have and I think one of the big benefits which all really emphasize in a bit is that it creates tangible metrics to deter your self justification so again back to Toby's sock you don't want to have that kind of like circular thinking that hivemind thinking so if you put here where our assumptions you can then go back and be like look we were wrong here and you can start to think through what do we need to do instead a quote that I've used in a lot of my presentations it's by Dino daizo V who's great I definitely recommend you read the attacker math presentation he put

together but he said attackers will take the least cost path through an attack graph from their start node to their gold node so the attackers have a minimization function and that's one cost so for the decision tree which I don't have my little laser pointer a full admission I use that mostly as a toy for my cat and so it's currently in the like cat toy area of my apartment so I completely forgot it which is bad but anyway so uh you have this kind of like you start it a reality this is the reality of your world you have say a 25% probability that skitty's or someone random let's say hacktivists that's not in vogue anymore apparently

it's 25% of the time the nation-state I think this is probably high for most organizations is 10% of the time and then 65% of the time it's gonna be some sort of credit criminal group the idea here how far does this [Music] okay this doesn't extend very far so I apologize but um as you can see here the first step remember in a dad game is you have some sort of defensive investment plan so let's say you invested in yellow which is doing nothing I definitely don't remain recommend yellow security but lots of people do it so let's say you yo load you don't have any privilege separation obviously it the attackers next move is like okay

well we have route this is great these people are idiots so let's say the next move you make is also yellow and you don't have any role separation they're like okay there's a database on the box let me use it and then you continue to Yolo and you don't have any Tokido cessation or segmentation of let's say these are like creds they want to get on the box that means they win they get to their target in contrast what you want to do is think okay let's say implement privilege separation let's say it works I don't know sixty percent of the time what you'll then see is in as a counter the attacker either has to skin for

whatever reachable data they can get from their current privileges or they use some sort of let's say known exploit which say works 50 percent of the time then what you can say is okay if we know they're gonna do that then we'll implement something like G R Seck which is as someone said like G R Seck makes it so unusable for the actual user it makes it impossible for an attacker as well so let's say it works like 98 percent of the time so the other option is you can use something like set comm that works say 50 percent of the time if you do that route let's say they use a one day which is like not super fresh o

day and that works let's say 10 percent of the time or they use it 10 percent of the time and then they can also have O'Day and that gets them to a win with G R Seck they have what I call elite o day because it's generally you get extra props if you have something that also works with G R Seck up and that say like they'll use that two percent of the time so the whole idea with this here I'm happy to answer questions about this at the end is really thinking about how can you force the attacker to that very far can be right or left depending on how you go like how can we force them to

have to use elite O'Day so it's me when I think about how do you create a decision tree I've actually done this for consulting a few times and it seems to work pretty well as first thinking about what assets do our attackers actually want it's also thinking about like what do we what about what assets do we care but it's really like what will the attacker tried to get and the important important clarification there is also may be attackers want some sort of asset you haven't you don't really care if they take it that's valid it's part of your own threat model and like building in or drawing in business context you need to be aware of what that is the second

thing what's the easiest way for attackers to get those assets obviously the yellow branch that's like a very easy way then what countermeasures are on that path think about what like defensive procedures you currently have do you have red limited privilege separation do you have like IDs anything like that then what new path will the attack or take given that mitigation so you've if you have IDs what are they going to do instead and create a new branch for that my idea is you repeat one through four until you create that path or they as I put it have to use a day all the way down like again force them to use that elite Oh days like what

does that path look like what are the mitigations on that path then you assign very very rough probabilities so you can kind of think about what's realistic I've gotten questions before like how do you draw this out so I use PowerPoint it's very inefficient obviously good if you're presenting in PowerPoint but you can use white boards and then camera snaps right a huge do not erase draw dot IO and Gliffy have plugs into confluence which can be useful if you're a Jewish or like Atlassian shop there's Google Docs they allow for like insert drawings so if you're using a lot of like Google slides Google Docs that's useful and then Visio I hate Visio apparently I

learned after presenting this at blackhat some people actually like Visio that was shocking but you can also use Visio but I really think strong that that decision trees help create a feedback loop to refine strategy thinking it's thinking through your assumptions thinking through how you're thinking about attackers allowing you to go back and check those assumptions as you get new data as the reality changes so in particular it helps for auditing after some sort of incident so you can think about okay where did our thinking actually go wrong like what mitigations did we have what we overlooked it also helps ensure I think a big problem in info second technology in general is that we take additive only approaches

and we don't really revisit what do we currently have like there countless times I've heard of people saying like yeah apparently we're using some technology from say 2006 but we never thought to get rid of it and you're still even maybe paying for it um but it helps ensure that you've actually mapped that out it also helps if there's like again as I said a historical record of your decision-making process which is super important I think I have another talk where I talk about like the how to make decisions and groups I think it's also really important from a group decision-making perspective but this is kind of like an objective collaborative thing rather than like oh well it was my

assumption that whatever you can't point fingers as much it's like no we agreed that this is kind of like our threat model this was our decision tree and it makes it more neutral it also mitigates I think the doubling down effect where it's like okay well we had whatever sort of strategy like with Equifax let's say they this was obviously not true but uh obviously fire I believe they were a testimonial for fire I so it's like okay well if we just had fire I maybe that wasn't the best strategy and it helps to show we thought that that would mitigate whatever was going to happen obviously it didn't so let's rethink what the branches actually are in what

countermeasures we can put in place instead the big ideas defenders the defenders advantage as you know the home turf I realize that's not always true even for a fortune 500s but that should be the strength which you tried to leverage so again it's you want to visualize the hardest path for attackers and how can you force them onto that path leveraging the fact that you understand your own environment then looking at the commonalities on trees like during some of the exercises I've done things like two-factor will come up on almost every tree like it's almost always it's useful somewhere so it's thinking about like how can you get the most ROI for whatever security product

you're using by cutting off the most attack as possible so another idea is with these decision trees let's make them the new quote-unquote nice report so it's a new request if you do outsource penetration testing which I believe tobe in date said probably not to do you can ask them to do it you have a red team I think it's very valuable to then ask them like not just how did you get to whatever the goal was it's also what did you do that didn't work what did you encounter like what countermeasure actually works what were the rabbit holes you went down that didn't pan out because maybe there's something that you don't even realize is working and if they can

articulate that that's great it helps you again to see that sort of attacker perspective or alternative perspective of your defenses and where to improve so learning exploitation again that was about thinking exploitation which is before you've actually had prior experiences with your opponent this is a has two specific types of exploitation I would say so there's information asymmetry exploitation which is disrupting the attacker learning process so attackers learn just like everyone else they factor past experiences into their model how can you give kind of like make that process not work then there's learning rate exploitation which is basically you can make those past experiences unreliable and play on the fact that they have that learning rate

in to how they learn and introduce unreliability into that data and start to actually preempt their moves which is more of the long-term thing but what you're trying to do really is make sure that their model isn't accurate so again exploit the fact that you understand the local environment better than attackers one way is through falling falsifying data when we think of asymmetry and InfoSec I think we often think of the fact that attackers know way more about us than vice-versa but I think there's actually a chance to reverse that so there's been some research that looked into jammers and I think it's what it's got energy harvesting communication systems which is not obviously the InfoSec domain but

what they found is attackers are really really reliant in that case on the disclosed power level of the device in order to actually jam it somebody came do is actually start falsifying that level of power and the attackers can actually conduct their attack appropriately so the idea of leveraging that the McCrone campaign is a case study and keep in mind I don't think we ever got confirmation that they actually did this but they said they did so let's assume that they did they allegedly used phishing tarp hitting so they when they signed on to phishing pages they used true and false logins and passwords they also planted fake documents like I think one was they had like a Silk Road order

which was very obviously bogus they also used to fake languages in their documents like Cyrillic just to make it very hard to verify everything really this plays into the Twitter phenomenon last year which was waste his time 2016 but it's for hackers so if you weren't aware of this the idea was that let's say like a guy's not texting you back or stringing you along something like that you waste his time you have him spent like by you would dinner or something else or just in general like different strategies to uh waste his time similarly to how he was wasting gears the idea here is doing that for attackers the goal is to really like

throw a wrench into the attacker scientific method so they can't test their hypotheses so the goal really is to make it so the attacker doesn't know why they're failing and have them keep expending resources to find out so some of the things here like kind of attackers probing the system you can create custom email rejection messages and then what you can do is create a honey doc on let's say the a block policy and you post that somewhere so when they google that and they hit that honey dot you can be like okay someone's actually probing this system like they clearly got that error message we know that we should be like a little more

careful or be watching out for that yeah it basically for any of these you want to track when it was accessed the idea also is a long-term if you use these honey documents honey tokens like thinks Canaries that look to describe legitimate policies or technologies that would be useful an attacker recon and particularly if you can have kind of go through the different steps an attacker would need to conduct you can start understanding kind of like the timing and pacing of your attackers non-determinism is another aspect of this so uh non-determinism basically means like there are different behaviors at different times and you can't expect the same results every time a SLR is a non deterministic feature it's one of

the more I would say famous ones and securities but it's also highly deterministic in that it works the same way every time but I want to amplify that and extend it to kind of a higher level in thinking about security so the goal here is to screw up attacker profiling so you want to raise the cost of attack at the very first step which is recon so you make the attacker and certain of your defensive profile in your environment as an example a lot of malware is now designed to look to see if they're in a VM or like a malware analyst sandbox and it won't run because it doesn't want to be caught so a good

strategy is to make everything look like a malware analyst in box because then it will never run a better strategy is make it look like a different malware analyst sandbox every time so you can only afford so many like 4 to $50,000 appliances so it's a lot easier some way if you put wolf skins on the sheep as I put it the idea is you mix and match hollow like a very superficial meaning that it's not actually the program but it's just artifacts of the program but like sketchy-looking from the perspective of the attacker the malware artifacts on like a normal physical system so think about your typical employees there aren't too many tools around this but

this is one of them and it really just helps you emulate virtual artifacts on dividual physical machines the unprotect project also has kind of lists of what those virtual artifacts are specifically what malware is looking for most commonly this is also a list so you can add in things like Wireshark fiddler but these are a bunch of executables that have been found work it's been found malware is looking for like vmware all that stuff you can also those are like typical address prefixes so you can change that you can just create like different dll's you also um yeah more stuff that you can create you can create like fake Drive stuff like that and again the idea is

that you mix and match these just to make it really really confusing because I have attackers like okay why isn't this running like why are we not able to do this across like all of these systems it's gonna be really hard to figure out like what the machines actually are debuggers are also an example of something that you can emulate so they registered like certain driver objects so you can pretend that you have those and they it's pretty nice in that a lot of times when you're looking at the reverse like Mao or like people writing malware that are like okay how can we look to see a debugger they'll find oh yeah you know it's always you know a

function of it's a 1 or a 0 as far as like what the like what the structure is set to and so that means you can be extra reliable on the other side and just always have it be a 1 rather than a 0 so it makes it pretty simple and also low dll's from AV engines and make it look like you have either like all antivirus or a bunch of different types which can also be confusing um it's pretty easy to find these but these are some of the most common of like endpoint protection agents or AV agents that I was able to find the idea here is to you could have a forwarder DLL so basically

you have one DLL which loads a bunch of different other ones again you can maybe choose at random you can also deploy like the lightest weight hypervisor you have because malware also looks for that um I haven't used any of these but these all seem to be pretty good but this is just like added wolf skin then minimax you may have heard it also as maxi min it's by john von neumann who's like a notable polymath basically assumed that each player chooses their strategy in a position of complete ignorance concerning like the other players choices and so the other player has to choose safely the idea is you minimize the possible loss for a worst-case maximum loss

scenario so you want to minimize given like you're assuming complete catastrophe um so Dan Gere had a paper that actually looked at this for calculating risk via evolutionary risk management functions which partly inspired my resilience talk on Thursday and it examines mini FAQs max functions so you want to find the minimum of the sum of the expected cost of protection the expected cost of non protection so basically if you have like a a cost curve for like the expected cost of non protection than expected costs of protection they're going to be slightly different so it's like the cost of a breach first the cost of the security products you're spending on which you're spending money and some of

the conclusions from here your dan gear cited that one ideas you don't have a monoculture I talked about this idea of diversity a bit more than the talk from two days ago it's on my Twitter if you want to see it's very beneficial for protection there is a trade-off with efficiency here and I think that's very much so up for debate it's not the context of this talk stochastic decisions or random decisions can be better than deterministic ones again as we saw they're at least equally as good as game theory and it could be helpful to throw off attackers then from the imitation game the idea would be so in the imitation game basically you have

the Enigma machine you can like figure out what the Germans are doing but you don't want to act on that information all the time because they'll figure out the fact that you know you're obviously having access to that sort of information you also don't never you don't want to use it none of the time because you want to have some sort of advantage so it's like what's the balance between using it some of the time most of the time and they talk about that in the imitation game and finally looking ahead so this is thinking about how to use these sorts of behavioral strategies this framework going forward fluctuating infrastructure using emerging tech which i think is i

don't think it's been solidified yet but seems to be called infrastructure 3.0 like containers is one way netflix is chaos monkey another thing i talked about that other talk makes it really difficult for attackers to persist so the idea with chaos monkey is basically it kills instances at random so you basically have a lot of like tearing up and down again it makes it very hard for persistence which is great as far as defending against attackers this notion of fluctuating infrastructure you never have the same sort of environment at any one time so again it really really hurts attacker profiling I think the dream is modeling at actor cognition verse model tracing like really getting towards

predicting attacker actions I don't think we're anywhere close to there we have to start being able to collect the data so I think thinks for example makes these canary tokens you can actually create your own I have a bunch they're super useful but it helps in the prior example of again like you have some sort of penny tokens to try to figure out okay what reconnaissance activities as an attacker conducting you can start to model out kind of like the process that they're going through again the timing and so forth the idea is also again that preference has changed based on experience so what you can do is sometimes like not block a certain attack or action and see do they perform

that detection action again so you want to incorporate what's called the post decision state which is like the attacker has either received like a success or fail feedback you want to figure out how they're going to respond to that also they hire the attackers learning rate it's easier to predict their decisions generally the learning rate is slightly variable between people so it's good to figure out and start to calculate how much those sorts of prior decisions will factor into their decision making you also don't have to assume like initial attacker preferences based on empirical research so in that prior example with the random equals game theory results I think of as initial preferences only change to the

outcomes by one percent or something so it's pretty much negligible so the the brief formula here is basically the the change in the expected utility of an offensive action basically like how much an attacker wants to use a specific action it's basically the learning rate times the feedback like I says tests or failure - the currently the current expected utility so basically it'll change based on whatever the attackers learning rate is how much they incorporate prior experience into the model times like success - like how much they already wanted to perform than action so for example if the learning rate is 20 percent which is roughly reasonable and then you just make like a success one and then a loss

negative one then if an attacker successfully performs an action learning rates 20 percent it means that the attackers 20 percent more likely to do whatever the action is going forward then from here based on what you actually see like how likely they are to continue performing certain actions over time you can start to adjust that learning rate start to predict so the tricky part here is that they may respond more to losses than to gains I talked about prospect theory one of my other talks there hasn't been a lot of work on that yet and I think we're so far away from this point right now we don't have to worry about it but it'd be

great to get to the point where they do or we do have to worry about it um so yeah you basically want to track the utility values for each attacker action and then for detected or blocked actions the attacker action and outcome are known variables so the utility is calculable obviously if you aren't aware it and attackers conducting an action you can't get to that point but at least then you can kind of start predicting if you're able to actually see an attackers actively conducting some sort of campaign hopefully you can start to predict what they're going to do particularly if you start to see commonalities between actions of compromises in the past versus the

current one that would be super helpful so in conclusion despite the meme it's no longer time for some game theory we've also learned that game theory is a language not even a theory so it's me now you can sound extra pretentious at cocktail parties by saying that so if that's the one thing you take away that's fine with me so you start with a SWOT analysis to gain perspective of yourself in the context of your adversary and vice versa you use thinking exploitation to improve your threat modeling and you use learning exploitation to beleaguer your adversaries by messing with their ability to profile I think this is a collaborative effort so we have to work

together to build strategies based on this behavioral framework I know that generally collaboration across the industry hasn't nest Shirley worked well but I think this is so this is such a new area that I think being able like even since I gave this talk at blackhat like hearing people's strategies that they've already been testing in their own organization I think starting to talk to each other about how can we leverage these ideas is really really important then uh the next step is how to begin model tracing attackers again the idea of the honey tokens tracking what their actions are that's super useful then finally hopefully we can start to predict attacker behavior so I recommend try these at home make

your blue team empirical like honestly the worst case is like random is just as good as game theory you're probably doing something maybe based on game theory because a lot of strategy is so you might as well give it a shot and uh the quote I always like to close on is Dan gears which is good enough is good enough good enough always beats perfect the hard part is obviously determining what's good enough his goal was like self sufficiency not expertness and having a culture of measurement but the ideas even if however you implemented isn't perfect it's probably gonna be better than what you're doing so take the risk and try these so slides from black hat already

online I'll also post these again so you don't necessarily have to take a picture now this is my various contact information with that thanks very much

you