Christopher Peacock: TTP Pyramid

Hey, let's turn it down. You start the talk. We do stuff like that. Uh previously I worked on beyond intelligence in space where I did so tier three analyst uh incident response. I was leader I was uh intelligence lead. I missed anything else that you do in your job. >> He took my job over. Um so yeah pretty much anything blue team consulting I did there. General Dynamics I started there so tier 2 and that's where I actually started doing purple TV engagements we had um got a pin test coming up my sister wanted to look awesome so it was like hey you know figure out what the pin testers are doing I go out like hey you're running

this thing called test so we tested our AP doesn't do anything with it and so we're like okay we need an ed we went down that rabbit hole and we started actually building better detections around it and that's how I got into purple teaming. Before that I also where I was a network engineer. So when we talk about uh purple teaming one of the things that we're talking about is the threat intelligence process. So what we want to do is we want understand the target organization. So when we understand the target organization we start understanding their threats. If I am a K through2 school, you know, if I'm the county's uh school uh board like the um IT analyst,

I really don't care what North Korea is doing to South Korea, right? Like that's just not in my current landscape, but maybe I care about ransomware because of hitting schools lately or something like that. So that's where we need to understand our organization, what our threats are. Now from there we identify an adversary. We gather a threat intelligence. We extract the procedures that they run. So TTPs have been lumped together a lot. They're actually drastically different. That's what we're going to dive into because people just throw around the word TTPs, but they're not actually communicating at each level. And then you want to analyze, organize, create a plan, emulate the adversary. Once you emulate the

adversary, what you're doing is you're checking multiple things. So you can check things such as do we have detections around the different procedures that were ran. From there we can also go out and we can say all right did our analyst respond to those alerts that were triggered. We have any detection gaps we can conduct detection engineering based on those procedures that we've observed in our environment from that. So that's one of the things that we're talking about when we want to get to a procedure level. We're really talking about trying to get down understand what the adversaries are doing and validating our controls or building um detections to fill in those gaps. Threat intelligence is not piping your

IOC's into your SIM. If you say you make actual intelligence by piping IOC's into your SIM, that is not threat intelligence and that is an embarrassment to people who actually do intelligence because it's a very difficult thing to do and there's a lot of great people who are driving. Um, this is actually probably like classified or reclassified like private now because JP2 actually got taken down off the military website. Uh, but thankfully it's still out on the internet. And what we have to know is when we're looking at intelligence, this is where the IRC's are is the data, but we're actually scoping down to where we're trying to get intelligence on an adversary. So we want to understand

their operations. We want to understand their procedures. We want to understand who they're targeting. And so like this says, we're really understanding the threat actor's uh motives, their targets, and their attack behaviors. Oh, and I told Dave I was going to speak in tactical intelligence. That's how you really make it actionable. So, one of the things that we have to understand though when we're talking about intelligence is a threat. Often times I hear people like, "Oh yeah, we have a threat because we're vulnerable here." Well, that's not actually a threat. A threat is made up of three simple things. And the more you understand this, the better you understand threats. the more you understand the threats, you understand

who's coming after your organization and what to prioritize and address. So, first off, we have hostile intent. With hostile intent, what we're looking at is, you know, do we have data that the threat actor wants? If we don't have data that the threat actor wants, we don't care about them. All right, the hostile intent could be to make money, it could be just to, you know, deploy a wiper. So, we have to think about the hostile intent. Then we have the capability. You know for a while ransomware they weren't doing that great of procedures that difficult of procedures but then as they grew into a billion dollar industry their capability really went up. They started developing

or procuring zero days something like that. So the capability is going to go up. That's going to increase the middle part which is the threat. One thing about hostile intent too you can actually control this. So if you are doing something that adversaries are targeting and it's not that profitable for your business, guess what? Stop doing that small little business. Then the adversaries go away. So that's one thing to consider. And finally, the area that we control most from defenders is the opportunity side. Opportunity is what's out there in our organization. Do we have app um app blocking enabled? You know, do we actually patch at our edge? These are areas that we can control and

we can limit the opportunity presented to uh the threat actors. So the more that we understand these different segments, we understand what threats are potentially coming after our organization. And then Andy Piaza put together threat box, which is a great area to actually look at and try to prioritize the threats that we care about. And as you can see, it goes by intent. So, as intent goes up, it gets more important. And as capability goes up, it gets more important. So, you know, if I'm working for a DoD uh client, maybe I'm worried about China or Russia more than I would, you know, Venezuela because maybe they don't have as much capability. I mean, maybe

they're supported by Russia some, you know, they might have some of those capabilities, but that's a different conversation. We can talk about that later at the after party. So when we talk about intelligence, we need to go and actually understand the timeline of what the uh kind of the history of intelligence, what it's adapted within cyber and the kill chain came out. Then we went to AP1 uh pyramid of pain and now we're kind of where we evolved to MITER attack is where most people are trying to think of intelligence in cyber. If you're not familiar with a cyber kill chain, I definitely recommend familiarizing yourself with it. If you ever want to work in stock tier one,

you're probably going to have to understand this and try to categorize it. One of the areas that we're going to focus on though is actual actions on objectives. So installation um is one area that we can also look at when we're trying to catch stuff. But for the most part, when we're looking at delivery and exploitation, we're kind of relying on our vendors for that. You know, we're going to do our patch cycles. we're going to rely on our AB er vendors and our email gateways to really focus on there. But when we're looking at um you know prevention is nice but detection is a must. So that's where we really want to focus um when

you're actually trying to do any sort of purple team engagements, emulations or detection engineering, you really want to focus on those actions objectives. The other thing to bring up with the um with the cyber killchain that a lot of people don't know about from cyber threat intelligence, I highly recommend going out and reading up on the diamond model. What this actually does is when you dive into cyber threat intelligence, mapping the diamond model at the different stages of the killchain actually allows you to align up and track adversaries. So you can actually if you have enough collection and if you're doing enough instant responses, you can see different adversaries and that's how you gets or activity groups

and stuff like this. Also too, if you don't need to do attribution, don't try to do attribution. It's really probably not worth it. If you want, I can give you some dice and you can roll them and it'll tell you China or it'll tell you Russia or it'll just give you random attribution. But if if you can go and you can say the Simpsons did it and it changes nothing, don't waste time on attribution because you just don't need to do it for the most part. AP1 report. So with the AP1 report, what happened here was Manion private um defense contractor actually started um doing analysis on an activity group which link back to China.

One of the things that we when we look back across history in defenses, we look at hey AV caught something. Let's remove that malware. Malware is bad. Get it out of here. That's not the way though that APC wonder started showing us. We started seeing that there's more than just malware, right? There's actual human operators behind the malware that's taking actions on objectives. The other thing is you want to get down to that human element. They're going to have certain actions. I don't know how many of y'all work in a large organization such as the military, but you're going to have certain approved actions that you can take. You're not going to go out of that scope. Otherwise, you're going

to get a rep. Also too, it takes a long time to get tooling. Sometimes procurement, like in large organizations, it can take 2 years. Like if they want to buy an EDR, it might take them 2 years because they have to vet through everything. They have to go through that procurement life uh cycle. Then training too. So like you know the US a lot of our operators, they might go to SANS or some other training like that. So you're going to have certain training. You're going to have manuals. we're starting to get down to the human element that they're actually going to do procedures that they repeat and catch. And like I said, manuals, the

conc manual is a great example of that. And it's kind of hard to see up here, but um like all these are just slightly different names that they're registering for their emails. So that's just one example of how APC1 is going and they're just doing something from, you know, playbook. Also, this is a screenshot from a remote desktop uh s uh session which I still don't know how Mandy actually got unless they just packed AP1 back. But that's an interesting conversation to have as well. And then when we look at procedures though, we actually look at AP1 versus Conti. So this was back in 2011 um or no my bad 2013 I believe 2012 and

this is from Conti 2 years ago. What we see is AB doesn't do detections on procedures. AAP does detections on malware and it doesn't even do a good job of inmemory malware. So that's another topic. But when we get down to it, we actually see stuff like net group domain admins, right? Hi is still running this. This is like a decade difference and it still works in most organizations because we're not focusing on actually catching procedures. So that's one of the things the main takeaway is look at procedures. We have to figure out how to actually catch those. And that's coming down to one of the things that we're at right now. A lot of

people are still down here. They want the hash values. I've been asked time and time again, hey, there's a ransomware incident at one of our suppliers. We want that hash of that ransomware so that we can then hunt for it in our environ. Well, guess what? That ransomware group, they're going to generate a new hash every time. If you're trying to catch on hashes, you're not really going to catch anything just because of they they generate so much and so fast and they're going to be different across all the environments. Don't me. you should you should have some sort of IOC um pipeline to check your SIM, but like that's all it's good for is is is just doing a check. It's

like doing a compliance check like let's just make sure we're at least meeting this stuff, you know? And then same with IP addresses. It's trivial for an adversary to change their hashes. Like I've seen tools every time they download it's a new hash. You also have uh polymorphic malware that's going to change throughout the cycle. IP addresses, it rotates so easy. Often times you're going to have an adversary that's just turning up and then burning down a bunch of different IP addresses. Domain names too will autogenerate that. And then when we when we get to network and host artifacts, this is where we're kind of relying to on our actual vendors. So if I'm looking at my

network, maybe I have Palo Alto, right? I'm trusting Palo Alto to build out this stuff for host artifacts. I'm trusting that my DDR my AV is going to, you know, go after those. But then when we get to tools in TTPs, which this lumps together back in the day from Dan and Bianca, that's where an EDR can't necessarily scope down for our environment because if they do it, they'll have too many alerts in other environments. Um, so we really have to go out and actually have to tune this for our environment. So automate all this. Trust that you're doing this. Focus up here. Now when we say TCP, so we just break that out. And

I want to get down to procedure level. First um we're going to talk about the hashes. And this is just an example of why hashes for lack of better word suck at doing detection. Okay? Like literally. So David Biano who just the pyramid of king all these over 80 million were only submitted by one organization. So most of the hashes do not cross different organizations. So you're doing a check for a hash that you're just not going to see elsewhere. And if you get other areas where it's a worm and worms through the world, yeah, that might have the same hash. But day in and day out, you're just not going to see a shape.

So this gets us to attack. So we know that David Biano said, "Hey, we need to move up from hashes, IPs, then we need to get up to that TCP level." So then miners started working on attack. This is where most current intelligence lies and there's a little issues with it, but I do I do enjoy it and I love the attack team. So it was two years after the AP1 in the pyramid of pain. But one of the issues is it's a one-way function. Often times people say I want it actionable or I want to make attack actionable. And it's like okay cool. How are you doing that? And they're like well you know we get the navigator and

we put colors on it. It's like all right but that's not what attack's really meant for. What attack is meant for is you have procedures and observations. What what it's meant for was is when people went to do digital forensic incident response, they would then take those observations, they would map it to techniques such as the sorting hat and then they could communicate that to other teams. So that's what the attack was really meant for was just a one-way function going from observations to techniques. It's that simple. So don't try to overthink attack when you hear it. This is what it looks like. So you see here we have proc dump uh with else to

elsass being dumped using proc. That's going to be T1003. That's how you use the attack. You actually just map what you see to the technique ID or you get the software cast software ID 00002. Um you also have you know exploitation for privilege uh escalation. So this is how attack is actually meant to be used. So that's what actually making it actionable is we're going from the observations to attack. We're not trying to take attack and do some magic fairy dust. That's not how it works. But this is the current state that we're in right now is everyone wants that 100% attack. uh this past year they did a recent evaluation and all the err vendors said

they came out and they said we have 100% attack coverage and you really can't do that because you don't know what the adversary is actually going to do. Yeah, you could run one test for a technique. Okay, cool. But that technique might have, you know, 50 or 100 different procedures for what this leads to is we get stuff like this. I call this the checkpot balance. So someone will come in and be like, uh, okay, for this technique we ran five different tests. Let's mark it off. Okay, cool. We got coverage on that. But the issue is if you're running five different tests and the adversary is using these other three procedures for that, you're not going to catch the

adversary. So this is the issue. And that's why I say it's a checkbox policy because it looks nice, management likes it, but you're not actually seeing what the adversary does and you don't actually guarantee any coverage over the adversar's procedures. That brings us to this where we actually need to break out TTPs because everyone's just talking about TTPs, TDPs. Oh yeah, we're going to look at these TTPs. And most of the time when they do that, all they're giving you is they're giving you a technique. So if I say, hey, you're doing a technique of credential dumping LSAS, that doesn't tell you what the actor is actually using. You don't know if your detection capabilities are actually

going to be able to detect it. You need to figure out the actual procedure so that then you can uh test your ER to validate your detections. If you don't have them, then you can build it. So with tactics, this is like the bottom of the pyramid of TTPs. We really can't make tactics actionable. They're just too broad. So what I kind of equate them to is they're buckets. We take techniques and we put them into buckets. Otherwise, the attack navigator, it would just be terrible. It would be a giant blob. And then like I said before, once we get up to the technique level, this is where most intelligence uh that we see is

being communicated at, but it's just not actionable enough. Like I said before, if I say credential in LSAS memory, okay, how do they do it? If I tell you that they're smuggling in, you know, a file via email, okay, how are they doing? Is there password protected zip? Is it going to be in an ISO? Is it going to be a link that then gets clicked and then download? There's so many different ways that you can skin technique. You have to understand how the actual adversaries operate. And this leads us to this fallacy. We get procedure assumption. Since everyone's talking at the technique level, if I come out and I say, "Hey, the adversary is using

process discovery. I know the adversary's TTPs. They're doing process discovery. All right, but how? So, what happens if I go and I run this atomic test? If you don't know what atomic uh red team is, I definitely recommend it. It's a good starting point. By no means does it mean that you have coverage over your adversary, but it's a great starting point to actually start building a base off of. Atomic Red Team is a great uh tool to have. You could run different procedures or techniques. But if I say the process discovery is what the adversary is doing and I run get process. Okay, maybe I alert on that. But what if the adversary is actually using native

API? Well, now I I did all my detection engineering and my alerting on get process. It's not valid for the adversary anymore. So this is where I can't say that I just have coverage at the technique level. I have to say I have coverage at the procedure level. We need to understand that. The other area that we get to is is people go out and they say, "Hey, I want to emulate AP1." They go, they go to MITER uh attacks navigator and then they'll grab these techniques and then they'll just run atomic test. There's multiple issues with this. The atomic tests don't actually uh test what ABT1 did. The other issue is is AB21 actually a valid

activity group anymore because of the way that kind of merge. So that's just another thing. I ask AP21 all the time but they just don't exist anymore. So know that in attack like we said the technique level the techniques they can hold so many procedures. So don't have an assumption at the technique level that this procedure is ran. You can't do that. Like I said, you can't go down. If someone tells you, hey, the adversary is doing this technique. You can't go down to procedure. You need that procedure level data. And that's where you can actually validate your detection engineering on. So when we really focus on it, we want to get up to here. So people keep saying

TCP TP TP that. try to actually get when you're talking to your that moves that's not when you're talking to your incident response team when you're talking you know to management you want to get up here to procedure level are we actually understanding what our adversaries are doing at a procedure level can we emulate that in our environment preferably in production so that we can actually validate you know our our production rule set. So that's what we want to focus on is getting up to the procedures when we actually talk in our environment. We don't want to be talking about TT ttt. We want to talk about procedures and that's where we want to be at uh if we're actually operating a

good sock and good program. What what this looks like when we're talking about we want to understand the procedure level intelligence. We're talking about actually seeing what they ran. So here we see who am I/all. We also see like an IP config and we can take this once again. We can put it over there and we can throw it on an attack navigator. That's great. We can take these and map them to the techniques. But if I were to just give you this list, would you ever actually think that they ran command/hall or that they ran this exact nsookup command? Now, you don't want to necessarily do engineering on the exact that a little bit later,

but you can't go from that to this. You have to start here when you're doing your intel collection and when you're doing your detection engineering and your validations. And what this actually helps us do is it helps us focus on the human element. Uh I I work with Malware Jake. I don't know if any of youall know him. Malware Jake has a great story about this. there was an adversary who kept typing uh a command wrong like they kept misspelling it because either like translation or something like that. So they actually wrote the textured rule for when this one command was mistyped the certain way because the adversary always did. Whether they were copying and pasting from a manual or

something, we don't exactly know. But they actually wrote a detection rule, they'd be like, "Oh, there they are. There's fuzzy kitten again. You know, they're in the environment again. We don't know how they got in, but they they they keep running that misspelled uh command. So, there's human elements behind this that we need to focus on, you know, they have that certain training that they're going to fall back on. If you are if you're a threat actor, like if you're doing ransomware and you just ransom 10 companies and you know you ran this, this, and this and it worked in all 10, you think you're going to run it in the next company >> probably. So, you've had these built up

habits. I say um I haven't counted how many times I've said um up here. Hopefully not too many, but that's a habit in life. We all have habits. So, that's one of the areas we want to think about. There's actually a human behind these activity groups or adversaries and threat actors and they're going to keep doing the procedures that they do. One of the things is they might swap out some procedures. That's fine. If you're actually doing detection engineering at a procedure, when they swap some of those procedures out, you'll still have those those alerts that they'll still trip because of, okay, they run these 10 procedures, but they swap two of them out. All

right, I still have the other eight that I can alert on. That's cool. Also with the cont playbook example, this is another copy paste uh example where they actually copied and pasted it and they were supposed to actually fill in the actual IP address, but instead they just copied it from the manual. So I mean this this is how basic it's getting that you know people are just copying and pasting for manuals but they're defeating billion dollar businesses because the businesses don't understand the procedures that the adversary is running. So what does this look like when we actually want to make stuff actual? So recently there was a a zero day all right that Microsoft reported on that's

cool. I'm not going to really catch a zero day being exported. That's why it's a zero day. But we see this activity. We see that MSHTA is going out to the public IP address. That's a simple thing to test. I can actually run a command and see if I can catch that in my environment. So I understand the procedure that the adversary is doing. They were to use a new zero day. Maybe they were to use MSHTA again. I can catch them. You know, we see who am I execution with it being piped out to a text file. I don't know too many system administrators who are going to pipe it out to a text file.

They just want to know what they're running as right now. So, you have different things like this that you can scope in on when you actually are studying the adversar's procedures. How do you go about actually gathering this though? Well, like we said, reports reports like that with the Microsoft one are great. The defa report report is another big one. Now, they're not going to give you like, you know, Cozy Bear or Fancy Bear activity, but they give you commodity malware when it's doing what most people should be concerned about. Deeper report is a great example. Um, that's actually where I discovered about the isolink example. What adversaries are doing now is they're taking a link

file, they're packaging it in an ISO, and then when the when um the actual victim mounts the ISO and clicks the link file, it then will execute a malicious ELLL. So that's just an example of going out finding reports and actually looking for those procedures in those reports. It's kind of hard to find reports that give procedure level, but if you look for it enough, you start picking up on it. The other area to uh gather this procedure level intelligence to test in your environment is your incidents. Your incidents are some of the best areas to go in and test because if it got so far backtrack that incident and then try to detect it earlier. So that's

another area that you can look at too from there. Honey pots. How many red ters do we have in this here? One or two red t. How many blue ters do we have in here? We got two blue teamers. What does everyone else do? >> How many How many people are just starting out in cyber? Okay, good. All right, thanks. I don't know what like a third assume manage management. Okay, red teamers though like I give this talk some of the parks at like um at Defcon and adversary relation red teamers they'll come up and like man I know a honeypot when I see one I'm going to get right out of it. I'm like all

right but how do you know it's a honeypot? Well, what do you have to do? You have to actually enumerate that honey pot first and you're going have to get onto that honey pot. So honeypotss are actually a great area because you're going to get some of the initial uh actions on objectives that that adversary takes and then you can do detection engineering on that and now you're catching them at the front of their of their life cycle. And then also sandboxing. One of the best things that you can do is you know when you actually get your uh like malware samples from your email gateway on that sandboxing often times they just do like a quick

sandboxing. Uh that's an okay uh area that you can do detection engineering or detection validation on because you want to check like hey if those emails if something happens to our email gateway and that email gets through can we still detect it? So, you want to extract those procedures, run them in your environment, make sure you still would detect that if the email were to get through. The other thing though is you can actually take uh these email samples. You can run them yourself for a longer period of time um and then you'll actually get possibly some more actions. I've actually seen it before. We like threw some samples into a environment and like uh

we actually threw like a sample into an environment and like the analyst forgot to like reset the sandbox and then so like an hour or two later there's actually an operator operating that malware sample and it has like laterally moved to the other box in that environment. So if you take those email samples and you actually run them in a sandbox for a longer period of time, you might actually get some actions on objectives from that operator and now you actually it's someone who's actually targeting you because you pulled it from your email gateway. All right. So for red team emulation procedures are crucial for emulating because we want to actually emulate what attackers are doing. Now, one of the

issues that we run into is this red team will tell you these aren't, you know, these aren't really cool procedures that adversary would actually run. So, that's when you need the CTI whenever you're doing the procedure extractions like we showed earlier. Uh, document where you got that from. It'll help you if you're ever talking to a red teamer because when they go, the adversary would never do this. That's an advanced adversary. They would never do that. you want to be able to go, well, here's the Microsoft report where they did it. So now, please go run this in our environment. Um, so it always helps to actually be able to site that. But one of the areas that we

need to think about is adapting procedures. So like we said before, there was ISO that was a link that went to run DL32. Anyone know of a different way that you can execute a DL without run DL32? SVR. Okay. Well, there's multiple ways like reg SVR that you can do uh to execute a DL. So, if you're when you go to your red team, you say, "Hey, actually adapt the procedure some, you know, someone's going to use the ISO link method, how else can they execute?" Because maybe I'm catching run GL32 in an ISO, but I need to think about what other processes I might also need to flag on that the adversary could use.

So, I want my red team to adapt those procedures. You know, if I'm reading a report and it says, "Hey, the adversary disabled defender. What if I don't have the fender?" Well, then I need to swap it up. I need to actually say, "All right, they disabled crowd strike or whatever else it is." And that's where we want to test also variations and try to break detections. If you're doing uh red team, you know, try to actually get around the blue teamer's detections. If you're doing blue team, you want to hand over your detections so that the red team can try to actually bypass them. They can be like, "Hey, you shouldn't scope in to this command within the set

because we can just change that command or we can obuscate." You know, they can actually help you uh change your detection to make sure that you're not scoping too. How taking procedure level intelligence emulated attack drives detection engineering is this process. So like we said we get new procedures that the adversary is doing. We're going to bring it around. We'll emulate it in our environment and then that kicks off detection engineering. We get the direction. Then you go out and do collection of logs. You'll process those logs and then you'll assimate it back to the red team because you want to actually validate that your alert works. You want to validate that the sock can

actually respond to it as well. So this is an example of how we can actually go out, take procedures, run a small, you can run a small purple team cycle or you can run a large purple team cycle. It doesn't matter. Maybe you only have three procedures to run. That's fine. Maybe you have a full 20 procedure set. You can also do that. But that's just how you actually make it actionable. And what that looks like is in this example, we ran a few procedures. I believe this is from Conti and we mapped it to our alerts. We see that we only had three alerts. Well, if someone's actually um doing echoing the user domain,

I probably want to flag on that because if if my system administrator doesn't know what domain they're on, that's going to be suspect because my system administrator should know what domain they're on. So, I can flag on that. You know, net usage is huge. I might go apply on that net config workstation run system info, you know, IP config. That might be kind of hard, but if Betty and accounting do it, maybe I can flag on it because maybe only help desk should really be doing that in their accounts. So you have different areas that you should focus on here where we ran this, we see alerts, now we can go out, we can actually build

detections around it. One area I often get asked about is, okay, where do I go to actually look for this? Well, if we ran a procedure and we mapped it to a technique, I can now go to miner attack. If we actually look, this is a technique 10 59001. So, if I've gathered the procedure, mapped it to a technique, I can now actually see what log sources I need to go look for in my SIM or my EDR to start doing detection engineering. So, if I were to map to this technique, I know I need to go look at command execution, module load, process creation, and script execution. I know where I need to go focus my efforts to do detection

engineering. The other area that I often get asked about is what logs should I have? Well, of attack is actually able to show us um which log sources are applicable to the most um techniques and sub techniques. So what we see here is command execution is actually 200 is applicable to 243 techniques and sub techniques. So from a visibility standpoint, if I don't have command execution, I'm going to have, you know, drastically lower visibility. If I have it, I'm going to have drastically more visibility across attack techniques and thus I can detect more procedures. One of the other areas I can touch on is text. If you ever need to actually map out your coverage of your logging, this

is a great tool to have. I just like mentioning it because it's it's a fabulous tool. Um, but by no means should you ever actually should you ever actually interpret this as defense. This is what what it uh so you take the tech you pipe in all your data sets and then it gives you a nice you know guey to show manager and you get this nice purple coverage and everything but this is only visibility by no means should it be skewed as we have coverage over all this. It's like just because you have visibility doesn't mean that it's going to generate an alert by any means. So that's one thing to uh consider when we talk about

detection drivers. We have the operational uh capability. That's just how good is the analyst and the tools that they work with. If the analyst is working with a tool that's hard to work with for the longest time, Crowd Strike, you could really actually build custom detections in Crowd Strike. They finally changed it. Um, but that's one of those areas where if the tool can't be used by the operator, then you're not going to get great detection engineering from it. You have to have that data collection as well. If you don't have that data to actually review, then you're not going to be able, you know, you can't see anything. So, we mean that data collection. And finally, a threat

understanding. If if you don't know about like lawbas if you've never heard of lawbas I highly recommend checking out lawbas project living off the land binaries and scripts. If you don't know that like an actor is using lawbas or that that exists you're not going to go out and try to find it. You need to have a threat understanding. For the longest time, like when I first got into cyber, you know, it was AB bad, you remove AB. And then every at that point in time, like all the thread actions and all the red teamers were just powers, like using PowerShell to just rip apart environments because no one was actually looking at PowerShell. So that's understanding the threat. You

have to do that. And the way you understand your threats is by doing that procedure level intelligence. So what do we want from an detection engineering standpoint? We need to leverage right team to test, verify and then augment. So we can actually just do small tests. If we know that the adversary is coming off of winord and spawning cmd let's actually replicate that in our environment validate do we have detections? Do we not have detections? And then if we have a detection gap let's build one. You know Excel spine run DL32. These are just basic areas that we can start at if we're just starting off uh actualizing intelligence with purple teaming process. The other area is indicators of attack.

So when we're actually trying to catch procedures, we're not catching IoC's. We're not catching hashes. We're not catching IP addresses. I'm going to look for like in that previous example, I'm going to look for red DL32 coming off of Excel. That's an indicator of of attack. So you really want to focus here instead of IOC. So if you're just starting out, try to communicate this to whoever hires you or whoever you're working with that you want to focus here and not there. Automate those IOC's. You know, you you can purchase or you can there's plenty of free stuff out there now where you can just go and you can grab some IOC feeds. You can throw them in your SIM.

They'll do the automated check. Boom. call it a day and focus on the actual indicators of attack. That's where you want to spend most of your time if you're on the blue team. So, with monitoring and response, I have a lot of words up on this slide. I'm not going to turn around and read them to you. You have to validate this though. So I've seen before where actually um there was mimikat on a domain controller. What you have to know about mimic cast is it's a credential dumper. A domain controller is probably one of the most critical pieces of of is one of the most critical servers in an environment. So mimikat's credential number is on a domain controller. You

can get all the passwords, everything, you know, and you get an AV alert. Anna says, "Hey, AV removed Mimikats from the domain controller." And then they closed the case. I don't know why people are laughing right now. It's because of how the heck did the adversary get to the domain controller and then bring in this tool. So we have to validate our response. Validate responses as much as you can because you don't want to be, you know, the monitoring team who tier one, you know, closes the alert and you have a threat actor in your environment for maybe a year and then eventually, you know, you get a letter from like a three-letter agency that tells you that

you have Cozy Bear in your environment or something. Um, this is just something that I I put up. I know we're not I'm not trying to sell anything. This is um my one of my guys I work with, George. He's a SAS instructor and a couple other people. They wrote this. It's the purple team exercise framework. So, if you actually want to make stuff actionable, like we've talked about, this is a great framework to actually deep dive into it and you can actually implement it with any framework. I will tell you this, do what works for your organization. When George wrote this, and this is free. Um, when George wrote this, he was at City

Bank. City Bank has like hackers like across the world. They literally they're like everywhere. They have a whole team like 40 plus. And so they he wrote this for a huge organization. You know, a lot of us don't have the funding that City Bank has. So adapt with any sort of framework, adapt to your environment. Um that way you know just take the pieces that work and then you can do it that way. With that that's the end procedure level purple teaming. I hope you learned that you know you need to get down to procedure level. We don't need to communicate at TTP. What does that mean? We don't know. Is it a tactic that

you're talking about? Is it a procedure? Is it a technique? We don't know. So we want to get down to that procedure level. Uh and then I'll open it up for any questions.

All right. >> So, are there any vendors or anyone that's sharing this stuff out that does a good job of getting it procedure level that we could, you know, take to our manager examples or that we should be looking for? Uh, that's kind of a nice thing to look at. You know, uh there's really no one who's giving out procedure level feeds. Um, and then one of the things that you get with that is people who also they price fix. So they're trying to compare IOC feeds to procedure feeds. Uh so there's really no actual good feed yet. We're trying to develop one that's >> Yeah. What's up? >> Yeah. So uh I know you're not trying to

sell anything, but what site do we do to like deal with all of this? Like what what benefit should we get out of buying? I'm not I'm not here to try to fender pitch anything, but what what ideally what any sort of purple team engagement does >> is you go out, you understand the adversary, you extract the procedure level intelligence. So if we go here, we're going to extract procedure level intelligence. We're going to develop, we're going to analyze that. We're going to then build an emulation attack that then we deployed. Once you run that attack, you then can do all sorts of stuff such as validating alerts. Missed alerts, like we said, will drive detection engineering. And

then after you've done those two steps, then you can come back, you can rerun your emulation, and you can verify response so that you're not castroller guy. >> Yeah. Yeah. I wasn't trying to sell that. We already

>> Yeah. What's up? >> Hey, so how would we go about leveraging frontline responders like tier one analyst to be able to access the type of information gathering? You mentioned that your own incidents are probably one of the most important impacts. What is the type of information that you want them to be able to record? key so that we can add that into this, >> right? Well, one of the issues is that you're going to run into is your tier one operators will not be proficient enough probably in digital forensics or incident response. Um, but what you will be looking at is if you can, you're going to want them to extract different uh procedures that they see within that

cycle. So when they actually see fishing come through and they're analyzing the event, all right, what did what? And this is where you get into like the diamond model. So you want to map to diamond model. Uh there's actually a decent tool out there that does this. But with the diamond model, what you're looking at is you're trying to align software and activity. So like with the example with Excel spawning um 32 or Word documents spawning CMD the more you can get like those type of artifacts document then you're looking at

Any other questions? >> All right.