BSidesYXE 2024 - Jason Maynard - Red and Blue Teaming and the Powers Gained! Adversarial Emulation.

that's awesome all right red and blue teaming so this is going to be a maybe a little bit more advanced than maybe some of the other stuff right this is really about elevating our understanding of of the adversary and then defending against it with some breach attack simulation capabilities that we'll talk about so real quick anybody using miter the miter attack framework kind of yeah so we'll talk a little bit about miter what it is just to make sure that everybody's in the same uh Lane uh and then we'll go through some examples so I'll I'll do some breach attack simulation you'll see the outcomes the results I am using a little bit of Cisco in this right

there's some open source stuff it's Cisco because that's the tool I have access to you use whatever you want um but I'll show you a little bit about um how you go about doing that at the end of the day you know you probably it's all doom and gloom it really it really is and it is really bad out there there there's no doubt about it um all protections fail doesn't matter what vendor you go with doesn't matter if you go with a single vendor if you can which is very difficult to do uh the bottom line is those controls will fail so we'll talk a little bit about that like I said we'll get into miter attack um

and I believe miter attack will drive a defensible architecture that you can stand behind um we'll talk a little bit about the tooling and then we'll get into some uh scenarios sound good ready excited yeah so I've been doing stuff for a while been around I've I've you know I spoke at Cisco live I've spoke that uh you know used to be called uh Vancouver uh private privacy and security Summit i' I've many different bides so I've I've done a lot on the community side I do have a YouTube channel there is a lot of agnostic stuff on that channel just talking about miter the the P uh the uh what is it the P why can't I

think of it right now pyramid of pain right yeah it sounds cool hey cyber security has some really cool terms uh but pyramid of pain but also some stuff specific to Cisco all right all protections fail so you may have heard today you know you need endpoint protection endpoint detection response you need all these different things in order to secure the organization so on the client side right on the campus you'll have things that are plugged in you have users that plug into Wi-Fi they'll plug into your network and you'll have some level of protection on them whatever that might be you might have at least endpoint protection but hopefully moving to EDR capabilities or endpoint detection

response um and you have that installed but you also have other devices on the network that you can't install an agent try to install an agent on a thermostat right you can't right it's smart supposedly but you can't put any control on it so it sits on the network it's open it's available and if you're lucky you might have some detection capabilities on the network now you might be saying of course Cisco you're saying the network right you're trying to make it relevant in the conversation but that's not true the network provides a tremendous opportunity for Telemetry and the ability to defend so it's an opportunity um and listen as an adversary if I get

on an asset what do I need in order to move in in the organiz the network right I have to be able to leverage the network to move forward in my journey and if I have capabilities there that at least can see what might be happening that's an advantage and then you have workloads and this is where the challenge becomes because you have workloads everywhere you have them in a traditional data center in a physical server you might have a virtual server you might have a container it might be in cloud service provider one might be in cloud service provider two it might be in all of those areas right so how do you secure that well maybe you have some

controls that are installed on the workloads themselves with some limitations so all good but again there's Nuance so for example there's a workload that sits in the data center somewhere and it's Healthcare and the um the folks that uh manufacture that particular software won't allow you to install an agent on it right because it voids their SL so there are going to be gaps and that's all I'm showing here this is a simplified view there's always a whole bunch of other controls in play here but what happens is is that an adversary will get in and they'll land on that one box that's the first thing now I think I heard the comment that the

adversary has to be 100% right and I hear a lot of people say that I tend to disagree because I think the defender sorry the defender has to be 100% correct and I don't believe that I think the defender does not have to be 100% correct the adversary does so it's the opposite sorry um why I say that is is that an adversary needs certain things in order to be successful right so they need to do certain things things to get onto your box and then onto your box onto your box and then on your box onto a workload and then on that workload to some a lot of things have to come in line for that adversary to be successful

if I stop you from getting on that first box everything else doesn't matter because you can't move forward now you might say well then I'll put all my investments there and we're done right no because that's going to fail but if I can stop the lateral movement or at least alert on it then I know something might be happening so that's what I mean by the adversary has to be 100% correct the uh Defender does not so the advantage is the defender that's my thoughts around it so anyway the adversary gets on and they move laterally within that Network right because guess what this whole side is on the same VLAN and you have an endpoint

protection product that had a password on it or maybe it didn't and I uninstalled it and then I've moved laterally within that layer two Network I didn't even have to cross any other control point in your network I was able to move around in that entire segment so that happens and then maybe I get onto iot devices right there's some vulnerability I was able to scan for and then eventually right because this is not where I want to be I don't want to be on the end point I want to be on where the crown jewels are right I want to cause Havoc whether that's ransomware steel intelligence or or intellectual uh data doesn't really matter uh but my

objectives is not the end point that first unless I I hit the gold mine which is going to be pretty rare right like I I hit a box I got access to it and has everything I ever needed and I'm done right which is never going to happen so the point is is that now this all takes place let me just build this out a little bit more there's things that have to happen oh wait I went a little bit too fast there let me just go back a little delay in the click I I don't know if it's the Google huh okay thanks buddy yeah so when you look up here um what happens is is that

you know that initial access might have been spear fishing right there's an attachment the user clicked it and bad started to happen but again as I mentioned there's other things like I have to do privilege escalation I have to evade your defenses I have to move laterally in the network these are all tactics the adversary might leverage in order to be successful or techniques so game over you've hear about it all the time right the there's mentions about even local companies in Saskatoon that get hit and I like to comment on the the SMB like why we're small we don't get no right you're connected to the network if I can get access to a small SMB that

might have a bigger relationship with a bigger organization right then why go through the front door of that big organization when I can go through the back door of that small organization that doesn't have the controls or the staff to support them I'll start there and hopefully can make my way into the network uh for that bigger organization and while I'm there because now I have access I'm going to steal everything I can right I'm going to cause chaos once I'm done whatever I am uh trying to do so what happens here is if we're threatened form we understand the adversary we understand how they might be targeting us now we can start to defend against it and so very simple if

I need lateral movement what if you can't do lateral movement so maybe you do get on that host but you can't move to any one of these other hosts you just can't right there's no way of lateral movement in that organization now again there might be holes in this but a lot of cases you can drive this outcome so let's assume that happens but the problem though is is that asset still needs access to workloads right because if I secure that where they can't talk to anything well what are they doing right there's workloads that they have to communicate with and because they have to that means that that firewall or whatever controls in play here has to be open that means

that that workload is susceptible to vulnerabilities right like if there's a vulnerable uh uh U um software running on that system I can compromise it right if I could escalate privilege on the endpoint that H happens to have uh credentials there I could elevate my privilege on that particular workload so I can't 100% stop bad from happening but if you hear about resilience and things like this this is now starting to drive towards that because yes you might get on that asset but you're not going to move anywhere else in that organization you're not going to be able to move anywhere on that campus side and yes you have access to that workload but you're

not going to be able to move beyond that so yes small compromise I scope contain hopefully remediate smaller risk but the business is still operational but I need to understand what the adversary is able to do and so I I kind of stalled on Pyramid of pain it's even in my deck for for some reason I had a a brain fart maybe it's Saturday at 2 o'clock in the afternoon I don't know maybe forgivable thank you um so anybody hear about the Pyramid of pain yeah the one in it not maybe the that's a different show um yes so so anyways you got you know uh things at this lower level of the pyramid domain names hash

values you've got um IP addresses these are what I believe are are commodity based controls right it's a bad IP address it's a bad domain it's a bad URL they're usually a symptom of something right if there's a connection being made to a bad IP there's prob probably something on that asset making that connection right so it's a symptom these are things we need to block these are what I believe are commodity based controls but your firewall is going to do it your endpoint protection going to do all those toolings are going to hopefully mitigate it against this this is what the industry's been focused on is that bottom part what we need to do

is climb up higher in that stack and and the Pyramid of pain the higher up the stack you get the more pain you cause the adversary that's the goal here so if I need Powers shell on the Windows machine to move laterally in the network and I can't access Powershell as an adversary now I have to either be more creative and use a different scripting language on that particular operating system or I have to bring in my own tools right because you've stopped me from using Powershell so that's the advantage if we can start taking away a little bit of this and that um as Defenders I think we'll have greater success so miter attack I'm a big

believer of last three years I've been out promoting um the miter attack framework I do think it it provides the defender an opportunity to elevate their ability to defend I really really truly believe this and so what miter attack framework is is is a framework um but what it is is it's about real world not what if stuff right it's not pie in the sky and all this magical Fair dust stuff this is real world examples of the adversary being successful in organizations and it's community community-led and everybody can contribute to it it's nonprofit um these are the folks that do the common vulnerability exposure right miter um so it focused on adversarial tactics techniques and common knowledge and

really it's the who is the adversary who are they right what are their goals right the why um and then the methods the how that's what it is it's a great big table of all the tactics and the techniques and how they go about um uh compromising organizations in order to be successful and then everybody's fed in information about what is that particular adversary's ability to do and then here are some of the mitigation capabilities that you can Implement in your organization to to remove risk there's about for in in the Enterprise framework there's uh 14 tactics um and you know there's probably I don't know hundreds to thousands of of tactics and techniques and sub techniques right

there's a lot of them but in that there's only 43 mitigations so it's not that overwhelming a lot of people say well wait a minute it's overwhelming to go through but here's why it's important okay this is an analogy this is I don't know if you guys probably don't have this here uh because you guys are it's crowned for energy right like it so so say Alberta or Ontario right there there there'll be somebody knocking your door usually at dinner time right when you're sitting down they they they look like this maybe less suspicious but kind of like this right they have a badge they look you know presentable and you look through and you open the door and they

say hey I'm here from the electricity company um let me see your bill and I can tell you how to get a better rate over five years if you lock in right and so yes so you open the door and you're interested enough but what if their objective was to gain access to your home right get a level of trust the trust is you open your door and then they Rush In And they steal your TV your jewelry all right you've got a big dog maybe they're not getting in your door right but but they make their way in right they've fooled you because you've had a level of trust there what happens then they go to another house down the road

what do you think their level of success is is that door opening of someone's home pretty high right and guess what they go in they do the same thing they go to a different neighborhood okay knock knock knock you open the door and again trust is gained and again I steal jewelry whatever it is right whatever my objectives are so that's the success of it but what happens is each one of these people most likely um have called police right and so the police have went to each one of them and interviewed them said hey what's going on here well there's an individual with a suit looks kind of suspicious but not really right had a

badge right clipboard or an iPhone or iPad right I guess today it's on iPad right and and I opened the door and then they rushed in and he's they're capturing all the details and then what happens is they send out a bulletin they say hey be careful suspicious persons right but there's some Nuance here but what happens is if you look at it this is exactly aligned to the miter attack framework the robber is portraying to beat the uh from a trusted Energy company which happens to be the adversary okay then you have the uh the they want to gain access so what is their motivations here right the tactic and then what is the technique

right they have a badge they look presentable everything looks good right they knock on your door right at dinner time where you know you're coming in you're in a rush so you're probably you're not as alert maybe because you want to get back to the table anyways that's the technique they're using fake credentials to get in so that's exactly what miter does for us as Defenders it gives us the recipe right the ingredients of of what the adversary is going to do and there's nuance and the Nuance might be that that individual might be a a girl right versus a man might different tie might be wearing a ball cap I don't know right but there's

Nuance in this so there's Nuance in the miter attack framework and adversary might have different changes in how they might achieve an objective but guess what the fake credentials the knock on the door right all of that is still the same and then what happens is they knock on the door in a new neighborhood they're aware of it what happens they call police and whoop whoop right game over so tactic again is the why right so for example if an adversary wants to achieve credential access so I want to get your username and password that's the tactic and the technique to do that might be is I've got some creative software that gets installed on your

machine and dumps credentials and feeds it back to me or I send you an email that has a fishing link that asks you to sign in right whatever that's the the the the technique so when we start looking at a defens of architecture how many people have analyzed the adversaries that Target their specific vertical and I'm going to say not very many would raise their hand right so not really asking you but kind of asking you and so what I mean by that is if if you go and start looking at healthc care as an example and I'm going to show that in a second it's going to give you an indicator of the adversaries that have had success in

healthcare that have Target spe specifically Healthcare itself and so here's the 14 tactics in the Enterprise framework everything in red is a capability of that particular thread actor which is called gamron I think if I'm saying it right but initial access they're using valid accounts um persistence they're using Office application startup privilege escalation wait valid accounts again right they're using it twice to escalate privilege and to get initial access but now I know their techniques right for the tactics that they're trying to drive towards and now I can start mapping that out so in healthcare there's 14 threat actors that Target healthare so if you're in healthare if you're in uh retail you can go into the miter attack

framework you can type in that vertical and it'll give you all the thread actors that are associated um there's some really cool names like Tropic Trooper magic but some of these might be focusing on certain geographies like maybe Saudi Arabia or the Middle East if you're in oil and gas and maybe they haven't targeted you so again you would pick these and then focus on the tactics and techniques that these adversaries are doing and you might be saying well wait a minute these Advanced adversaries are not going to Target me well no they are right because they've already proven that they are and when you start implementing controls for these adversaries guess what they apply

to anybody else that tries to use that capability right regardless so now I MA this out I've got all these little red boxes here are the capabilities of all 14 thread actors so if you look here reconnaissance all the way down to to impact they have a capability so when I look at the techniques now this is on the the right side I can now start lining them up and looking at where my controls should be now for example if I look at in this case all 14 um thread actors they'll use valid account accounts and again not all of them will use the same things but in combination valid accounts for initial access valid accounts for persistance

valid accounts for privilege escalation and valid accounts for defense of Asia yeah so they're using valid accounts in four places so how do I mitigate against that how could I stop that piece from happening anybody oh did it already it yeah guys it's the answer's there I know it's 2 o' anyways mitigate with maybe risk-based authentication so some level of MFA right multiactor maybe some risk or conditional access to say where are you connecting from right but the goal here is now I need two factors username and password bad it better to be passwordless but that's another story and second would be now they need MFA right they need something that they have on them in order to get authenticated so

now I start removing initial access persistence privilege escalation and defense Evasion for those four pieces now that does that mean I'm done no because look at for initial access they have driveby compromise exploit public placing application so they're going to use these other things but maybe these other things are less likely to happen than this one so maybe that's why this is a priority or it's a priority because it it's in four places and if I could take that away and I really do believe this provides us things like time-based defense I'll talk a little bit about that but how do how do I get around that as an adversary if I have your username and password but I

don't have the device how do I get past it anyone because remember that control is great but the adversary is always trying to look to get past that control so what if I MFA bomb you or MFA fatigue right like what if I constantly you get a message on your machine that says hey you're trying to authenticate accept or deny deny hey you're trying to authenticate accept or deny uh deny hey you're trying to authenticate accept or deny it must be my Office 365 except you know that's exactly what happens right that's why I saw have it right because he he knows people that have done that and maybe you've done it by accident on yourself because I know I

think it sometimes and going wait zero trust it's always asking me right so anyways so what happens is so how do I build a mitigating control against that well the adversaries here they get the code right the user has this pop up on their machine but they don't know the code the adversary does so that's how that again when you start building it out now you start adding the controls in the right places you make it very very difficult for the adversary to be successful right adversary knows 7518 the victim has no idea what that number is and hopefully the victim will click on I'm not logging in and now you have an alert somewhere right okay so

you may have heard a little bit about attack chains uh earlier on today a little bit this is an example of it this is the opportunity as a Defender this is getting into the weeds right but this is the at the the opportunity so Step Zero there's two things that could happen here reconnaissance and resource development so reconnaissance could be I'm trying to get your external IP addresses so how do you do that how do you stop me from getting that you can't like you can abstract your email and do all that stuff but you can't stop me from finding out right um and then resource development might be compromised accounts so what's stopping me from compromising your

personal uh LinkedIn account right and you might say well so what that's not my company data no but now I've have friends now I have that account with that individual and now I see all of your contacts and maybe I send a message over LinkedIn with a PDF that's weaponized to say hey read this article great article right I just read it and now click bang bad happens right so there's things that you can stop the bad from happening but for me to compromise that Social account you have no control over that right that's that's a personal account so now everything moving forward though is an opportunity there might be some little things that you can abstract

but you can't stop initial access step one execution and we'll get into the details a little bit here in a second persistence right how do I stay on the box over a long period of time I've got privilege escalation how do I get more credentials or access to that system defense evasion right how do I get past your control so it doesn't know that I'm there doing something bad now if I'm logged in as you guess what I look smell feel just like you right for the most part um Discovery and the lateral movement and then you go to collection exfiltration and then finally impact impact could be stop a service could be ransomware right it could be Data

Destruction could be all kinds of different things right what am I'm going to punch you in the face with now well that's really what it is right bad happens that when they're at step 10 it's bad it's really bad so now that we see this so initial access I know it I don't know if you can see that clear there but um initial access might have a couple of things here right and so you know there's external remote Services they use that for initial access so there's some remote service that you get access to that I might be able to to to gain access to that system for whatever reason so how do I mitigate

that well that maybe that's an exg firewall capability right you just minimize that external remote service access it from the get-go right um valid domain accounts we talked about it risk based multiactor authentication so red means I've got a preventative capability in play when I look at um step two execution they're using wmi Windows management instrumentation so if I take your ability away from using that great but I might not be able to do that if Powershell is used administratively in the organization and they need access to Powershell or sorry you need access to Powershell and you can't restrict it well that's on the machine I have it access to it if I'm on that machine as

much as anybody else so um in this case maybe you've got some EDR capability right some behavioral element to help prevent that then you look at things like um again external remote services so how can I well maybe it's a a a ZTA or Zer trust access functionality right so you're trying to limit access to that resource over the network but the bottom line is is that there's multiple opportunities here I don't need to hit every one of these if I stop this one right the external remote Services there then anywhere else they might use that that might be very difficult for them to overcome but the other thing is is that now they have to try something else in

their Arsenal and when they do that guess what now I've got time based defense my tooling gets an opportunity to catch up to the adversary if it's missed it and second I'm in a box and I'm trying to punch my way through the box guess what I'm doing I'm creating noise right the more noise I create the more opportunity you have for your tools elsewhere to find it or detect it at least you know they're there so as they go through green happens to to be maybe I don't have a mitigating control here I can't roll outad MFA right now I just can't so what do I do well maybe we talk about user training um maybe it's push

push notifications because they have been getting past our our MFA because you've hit accept right you didn't hit decline again keep going through this and we start knocking off some of these not all of them but we start knocking off some of these and finally wh sorry one click two finally we have something like this now this is a simplified view of an attack chain right an adversary needs to be successful all they might have multiple options so you might get blocked and they overcome it with another capability but at the end of the day the adversary has to be 100% right in order to get to step 10 a Defender does not and that's

what I mean by that okay fun excited still there no all right okay breach attack simulation so I think uh the the couple of talks ago maybe the last talk she had mentioned pentesting right great great thing to do for an organization once twice a year it's probably the minimum that you're going to do it um ton of value in pen testing this is not a knock towards pent testing whatsoever but pentesting is only as good as the pentester and it's point in time right the moment it's over it's stale right where breach attack simulation comes in is this is a constant method that I can evaluate my infrastructure constantly right weekly daily every change that

happens in the organization I have the ability to test it for at least um a a certain level of common attacks that an adversary will use right pen testing could be very customized and nuanced and like I said there's a ton of value in it all right commercial tools for breach attack simulation you might have heard of attack IQ Metasploit Cobalt strike canvas um but there's a ton of Open Source and I would encourage you if you haven't went and downloaded Caldera um do that again these are from miter attack folks right they they've created Caldera a tool and we'll showcase it here today that you can leverage they do a breach attack simulation tests in your organization

it's open source it's free and there's a couple others that you can leverage all right so now we know the tools let's talk a little bit about it what is the value of it well first off it allows us to identify vulnerabilities in the organization right safely we can do this constant test that simulates the attack just like the adversary does um and it helps uncover those weaknesses once we identify those weaknesses we can start putting our controls in the really cool thing here is I can be very um creative in the testing that I do but I can also do testing against the miter attack framework so if I'm looking for a tactic or a technique I can actually

pick that Tech technique within Caldera or attack IQ and say execute that attack against this particular asset and let me know the results was I able to defend against and we're going to show that today um it it highlights where you're um able to detect and prevent so ultimately what is the most important piece detection or prevention I didn't hear anybody but I'll I'll detection right because you can't prevent unless you detect and never going to 100% prevent now everybody wants to and every vendor will tell you get our stuff and we'll prevent it the reality is nobody's going to give you a signed agreement that says we 100% protect you against everything 100% it's

not ever going to happen so but you want to be able to detect it and if you detect it there is an opportunity potentially to prevent it right not always because it depends on where that detection has taken place um again it helps you prioritize your Investments as well so if you find an area weakness that gives the adversary a tremendous opportunity to be successful then that's probably the area that you want to put a control in place right that investment now has been um articulated at that point in time compliance again if you have uh any Regulatory Compliance that you're doing this is evidence-based testing you've validated the control is working um you could demonstrate your

ability to prevent detect and respond to security incidents and then ultimately you can avoid fines and things like that right and then continuous Improvement right because I'm able to do this more frequent it's a tool that I have available to me now I have um the ability to simulate these exercise freely and because I'm doing these more often again I'm able to shore up uh my capabilities when I find weaknesses and the other thing that it does is it helps the team understand the adversary and how they may operate so if compromise happens your team now starts learning of what to look for now next because they know where they are in that stage of an

attack in order to be successful right if you don't have eyes and ears on the network as an example and you know they've popped one box you probably want to look for lateral movement right because they want to move from that initial access a access that they got on that initial uh asset so again it gives your team the ability to understand that they've been there they've done it um they start getting a good understanding of how the adversary operates now here here's a couple of scenarios so in this one we're going to use Network detection response I would say probably most organizations aren't doing much in regards to network detection response okay I think EDR is

being leveraged and point detection response um you've installed you know uh Microsoft crowd strike you cisal secure what a sentinel one right you've installed those tools but do you know what the network looks like do you know what normal looks like can you detect an adversary on the network un unlikely in most organizations unfortunately we're getting better but we're not there and and this is xdr and so the point here is we got Caldera open source free tool to use and we're going to live off the land does anybody know what living off the land is right we all do right it's using the tooling that's available and not having to bring in our own tooling if I

have to bring in my own tooling that means every control that you have in the organization has an opportunity to mitigate um so I'm going to use uh living off the land so Discovery is the tactic the technique is network service Discovery can we detect it what do you guys think oh I tried in your no I didn't okay so we have an agent that has uh Caldera agent installed and forget about the sound I know it sounds dramatic doesn't it and so then what I did was okay let's maybe I know nothing we talk about generative AI so I go on to chat GP and I say hey create a Powershell script that uh scans

the entire sl24 network based on the network configuration that you are able to pull from the machine right and then I want you to scan all common ports for that entire segment so I just asked chat GPT to create me a Powershell script that allows me to run it on an asset and scan the entire network so I don't have to bring in a tool to scan your network I can use Powershell to do that and not only do I ask it I say hey I'm an idiot well maybe not an idiot but I'm not familiar with Powershell so explain what each command is and so it went through here you go use it right and you know what it

took me a little bit what I find with these types of tools uh so far is you can take scripts you can create scripts but they're never 100% perfect right there there's always nuance and there's one thing around new bands that Dash band that was causing problems so I said exclude it moving forward right the script would never run um but then I said well okay that's great but that's in clear text can you encode it for me and so Powershell or sorry chat GPT said well here's how you encode it all you have to do is take this new script I'm creating for you put your original script into this new script and then run

it with the power encode Command right and listen the encoding is not something that's going to be hard to determine what the actual command is but it's obic nonetheless and so now I have these two Powershell scripts and now I know how to run it on the machine so you might be saying well wait a minute how did you get access to the machine well valid account doesn't matter I got on the machine Let's assume I've made my way that far in the into the environment but now I want to scan the entire segment that that asset is on to determine if there's other areas that I can elevate my um capabilities so I paste that

in paste the script and I go ahead and run that in Powershell and I get the encoded command right so again copy open up Powershell paste and now I have this encoded command hit enter and there it is so now I can take that power uh shell uh encoded command and run it in Powershell with Dash encoded Command right and you can't see that string and clear so okay there we've got it we go ahead and run this and you might say EDR is going to pick this up well let's hope it does but EDR is only as good as the asset it's installed on and it's only as good as the platform that's available but you

can't scan the network without the network knowing that it's being scanned that part you can't hide from doesn't matter if it's a thermostat a light bulb or anything and so the network was able to see that there was an internal port scan that maybe the device without an EDR capability wasn't a ble to detect and again it's just showing you now that I've got some visibility and there's a little bit more to the story in this this case because I did multiple tests on this particular asset it starts building out the attack chain and guess what Discovery credential access defensive asent and command and control it starts associating all of these independent observations from all these

different tools and starts collecting them together to build out that story for you and now I know even though I didn't stop it I I know an adversary is in my network scanning the the environment either that or a skilled user and then I need to have a talk with that user of why they're doing it right but I know something's happening and I thought well wait a minute I created that Powershell script using chat GPT and I knew nothing about it let me go into Caldera and I actually came in here and took those scripts there's some customized there was a lot of error that I had to go through and fix as I go

along around syntax but now I could put them in here in Caldera and I can run this tool now native with Caldera now Caldera is there's an agent installed in the machine right or any breach attack simulation tool you install an agent on a machine and now that's how you run the test right there's this is the the head end and then you have clients and then you run your test right so I was able to come in here and now I can create an operation I can schedule this to make sure that my networking and other tools are picking up this noise anytime and my team is aware of it so again it's like having having a

mini pen tester in your back pocket right I'm constantly testing my infrastructure for attacks um and simulating that so here I'm just grabbing that encoded command and I'm saying run that and then also run the the the clear Tex and the reason why you want both is you want to see if your tooling has the ability to tell you what's happening in the environment so can it tell you what the encoded command is great if it can that's pretty good and second it should be able to pick up the clear text that that one's a little bit easier or a little bit more trivial and I go ahead and run that it's doing exactly the same thing that I just

showed you running it in Powershell so there's all the IPS on that Network and if I scroll down to the very very bottom um I can see the open ports on that that Network and once I know the open ports then probably I want to know what versions and software that's available on that asset and then I'm going to look for exploit kits or sorry exploits against that vulnerability uh and then take advantage of it that that's ultimately my goal so that's the first one and again I'm using open source I can come and look at this analysis I can build out reports from this and it's all free so pretty cool and again you can

test anything like there are all kinds of miter attack capabilities built into the tool all right so we're able to detect it perfect that's exactly the outcome we wanted now this one I um this is a little bit different I'm using aack IQ I'm not going to show you aack IQ until the end if we have enough time but I use aack IQ um and attack IQ has a test that looks for this particular vulnerability this this content right web script security view for players blah blah blah right I know my intrusion prevention system doesn't have the signature to detect this I didn't want to show that because if I showed that then it wouldn't be

valuable here right if I'm going to block it and so when you do your testing you want to do it in two modes you want to do audit first because you want to see what you could detect if the attack was to run its full cycle because if you don't and you block lock it earlier on you might be prone to a whole bunch of other tactics and techniques that you have no idea your tooling has the ability to let you know so you run it not at first and then you determine wait it catches everything fantastic and then you run it in prevention mode to say how quick do I stop it right hopefully as

soon as possible so this one here I go to chat GPT and I say okay it's pretty cool all right so anyways audit mode so I go into audit mode um so I can see this test come to fruition and I see that it's been successful this is just an indicator that's all that is so I know that I failed to detect that test they were able to get through so then I go into chat GPT and because I'm using snort I can say hey create an open uh uh sorry create a snort signature version three look for specific parameters give it this message and write that signature for me now when we talk about augmenting

your team or your sock that's pretty powerful because now your sock can use AI to get them 98% of the way there maybe 90 right but get the the way there and then maybe they have to do some modifications and that's what I had to do here so I ran this I go ahead and hit start and again I say listen I don't know anything about snort signatures can you explain to me what each line does so I can use it as a learning tool and so it comes out and it gives me the signature right away I say well wait a minute tell me what each line does and it does that right fantastic check on time am I still good

guys yeah all right so now I I it comes out so it says alert TCP home network to external network HTTP ports it tells me that those variables are right flow to server established right the flow to the server has to be established 3 minutes left oh we're not getting to the the the second part all right so anyways draw this out okay we may get to the other part anyways so now I got that signature there's a a little bit of modification I have to do here it wasn't perfect but like I said it got me 98% of the way there uh I won't bore you if I can find my mouse again maybe I

can't weird anyways so I make the adjustments and then what I do is I go into the the firewall I add this new rule and I'm going to run the test again again and this time it's going to detect that test let me just see if I can fast forward it oh there it is oh no I knew it and so what happens is I've imported so just believe me on that I've imported the S yeah yeah no it worked so I imported it yeah yeah all right let me show you I'm just kidding I got to hit play if you want access to this stuff after just email me I'll get it to you um even

the videos and look at this it says intrusion would have blocked because the the intrusion policy is in IDs mode not prevention but now I know it detected it it's got the message that I created and I use chat GPT to augment my my security team's you know uh ability to create signatures on the fly with maybe limited U uh expertise so Victory there so fantastic right we're able to detect that I'll come back to that in a second there was the thank you cuz I knew I was going to run out of time but um here's the last piece right so if we have 13 minutes we'll get close to done it this this you can do this for other tools so

that we're going to use a Tac IQ that's a commercial product I don't care it's nothing to do with me um I'm using cisal secure endpoint today that has nothing to do with me it doesn't matter use the tooling that you have um but there's a couple things it's going to look for Behavioral prevention detection and then sample analysis and again I just want to show you this with a a better appreciation of it because even with these tools they might be wrong okay they might not be 100% accurate so anyways this is the miter attack framework at the top those are the tactics below are the techniques that we're going to test in this particular

scenario and what happens here is let me just fast forward a little bit um what I'm doing here is again this is our our xdr but it doesn't really matter I'm just proving out a couple of things right so there's no assumptions that something's missing I'm showing you what client's installed and what is enabled right so I telling you a little bit about secure endpoint what's enabled what's installed Microsoft right there's a a collection of telemetry for the device itself now that I know that that that's there and what modules I have I'm going to go into in our case we're going to go into a policy and we're going to make sure that it's running an audit mode why

because we want to run the full test right we don't want to block anything too early on and there's the version that I know that's just been updated I'm just coming into the endpoint and very quickly I'm coming to see that that audit policy is in place that audit policy is in place fantastic again use anything that you have available to you I go into attack IQ um and this is a SAS based service as well and I come in and I have a uh to come in and create a new assessment I create a new assessment and I you can build your own here you fully customize everything in here that which is really

really cool and um I grabbed that Baseline test uh for protect policy against cisal secure endpoint like crowd strike would be in here most likely in other tools right not just Cisco and here's the test right so the test kind of goes into behavioral protection dump Sam um this one's look collection of screenshots keystrokes uh clipboard data and here I can customize things so if I need to add active directory or anything like that I could put any any of those parameters um or if I want to customize it so it's it's very very flexible in that way um and then what happens here is um there's behavioral uh detection evaluation couple of and then some

samples so I'm going to go ahead and um create the assessment here and I hit create new I know you're excited me too all right now I add the endpoint so the remember the endpoint has the endpoint detection response tool but also has the attack itq agent or the Caldera agent that would be installed on it okay and then you have to make sure you exclude that because your your anti virus mechanism might actually block that so you might have to put a white list for that particular one I go ahead in the tool I run um the report now the Integrations on this which is cool it'll actually go back into the tooling to

pull the reports back to validate whether or not you had success the thing is there was a little bit of nuance so prevention you can see it was all red for the most part nothing was preventing why because we're in audit mode remember so not prevented not prevented not prevented right and there was a couple that were prevented so I I have to go back and look at why but it doesn't really matter but then I saw this one down here that says not detected and if I look at it it's got to do with dark Comet and I thought well wait a minute I'm going to grab that hash and I'm going to park it for a moment right so

it did it was successful meaning it landed on the endpoint and then the tooling deleted it from the endpoint it didn't block it K uh there's about 15 samples that were tested that was one of them that were was successful so keep that in mind I've got that sha I'm going to look at the report and it shows me pretty good detection right detection 100% across except that Ingress tool transfer for that dark com I think it was dark Comet um it was 90% we missed one and again that's any tooling right you might have variances in in the success rate nobody's going to be 100% And if I come in and said pound in my

chest and said 100% of everything you go well yeah he made sure it was 100% I want to show you in reality that it may not be 100% but that's okay but at least I know that now now what I do is I go through here and this is all the logging I go back I actually look at all this these are all the things that were detected like it's Pages just from that 20 uh test report for that particular agent um and so what I do is I come in I I look at what is the details here I want to make sure I'm at the right ver version change and then from there on

that's when I know so the version change was here and now I can see Mimi cats was ran here someone mentioned Mimi cats the SMB Justin I think it was earlier right so SMB or sorry Mimi cats was used here and and um again was maybe successful but detected right we detected it that's the key piece here and you can see all the stuff now ultimately we want to block this earlier on even the Powershell commands are included in here right if you can't see the the the command line arguments then you have a problem um because you don't know exactly what it is that they were doing so if I'm just going to skip through so

this kind of lets us know what took place and then I went back and I looked at that Shaw and are we good okay all right five we're not that good then anyways so I took that sha and I went into our Mal analytics sandbox and I I I put in that sha and we do know about it so we do know that that sha was Associated to dark Comet now my I I had to figure out why the endpoint didn't and honestly for this test I don't care right normally I would but I don't care in this particular test I didn't dig any deeper it could be that the reporting engine didn't actually grab the data

over it could have been some reason why we missed it I don't know and again I don't really care um but I would in a real world so here I'm looking at we talked about attack chain remember that nice little attack chain that went down and showed you everything where the controls are this is a visualization of that right it starts looking at the processes to the name to the email to the the user you got Network detection response endpoint detection response identity threat detection response and now you can see the relationships between bad which is in red to any of the assets that it may have been involved with so hash to an IP an IP to a domain an

endpoint to a user right I get that all collected for me and I can visualize this as an adversary there's lots of stuff I have to do now I can visualize it I can see it as a Defender and if I'm doing breach attack simulation I'm doing it in the safest possible manner right I'm not doing it when an incident is occurring which is the worst place to learn so then I go through and I run this test again okay so I'm just going to fast trck it because of that five minute limit here um I rerun that test again in protection mode and exact same thing but what's different here is I start seeing ered

now this is what I meant by the tooling Aid means that it didn't get a determination one way or another and what happened was the defensive tool in this case happened to remove it so fast enough and earlier on that the tooling didn't know what happened right so which is a good thing but this is what I meant by taking it with a grain of salt right you have to come back to this um what happened here you have to come back to this you could pull this report and validate whether or not you detected it and and prevented so what happens here this is the report look at this missed all of this it says it missed all of

this piece here which is credential access it missed Powershell over here and it missed um that one Shaw right that I talked about earlier and so you would think that you failed that test you might even question your endpoint but when you go back into the the tool I show you in the the video I go back in the tool we actually did block it so it was blocked the report never came back into the tool and I could reach out to attack IQ and figure that out it's probably something simple right but it didn't come back into the tool and so I've just manually modified this to say prevent it right but again at least I

know where the failures may have occurred and then validated um if they were true or not outside of the tool that's it that's it it's really over now well there's not it's not that over I think you have one more session right and then closing yeah but it's almost over anyways thanks