Confidently Measuring Attack Technique Coverage by Asking Better Questions

my name is matt graber i'm a threat researcher at red canary been there going on about two years now and i'm a defender with an adversary's mindset so a little bit of history for me i was active duty navy for some time as a linguist got out started infosec start my career in doing mostly offensive stuff like red teaming and then transition did some malware reversing for a little while and since then i've been primarily working as a defender and what's cool about that is that i'm able to apply an adversarial mindset to everything that i do in detection including detection engineering which is going to be the focus of our talk today and a cool thing is uh my very first

infosec talk was right here at besides augusta the the very first one in 2013 and so well is my mentor chris campbell who brought me in um and really encouraged me to present and that's when we presented the concept of living off the land for the first time so uh it's just crazy to think like how much things have changed and also how many things have not changed at the same time so um yeah it's it's been a wild ride all right so what does this talk about i'm going to attempt to do three things we're going to define a framework for an adversarial approach to detection engineering so what i mean by adversarial i mean applying an adversary mindset to

building better more resilient detections getting getting as much coverage as we can or shelf life out of our detections because detection engineering it's not and there's no end state to it it's always a process and the more coverage that we want to squeeze out of any of our detection analytics it's best to apply an adversarial mindset to be thinking about how can we build good initial coverage and then bypass that rinse repeat and then we're going to actually apply this framework to to a specific attack technique in this case mshta so mshta is a built-in utility in windows signed by microsoft that allows an admin or an adversary to execute script code specifically vbscript or

jscript and as we'll discuss and discover some variations thereof as well and i thought mshta would be a good example because it's a relatively simple utility it has a ton of options that we as the tech engineers would want to consider but it's also relatively tightly constrained in scope for for the length of time that we have here and also i want to encourage all of us certainly myself included just to get in the the process of asking more and better questions and so we'll discuss uh some of that as well so some aspirational goals of detection engineering in general not even applying like the um the adversarial mindset per se you'll hear me harp on this all the time

and i have to remind myself of this if i ever start to get overwhelmed with a technique that is initially too broad and many of you are familiar with the miter attack matrix some of them are really freaking broad like there's a sub technique for powershell like there's nothing powershell can't do so like if someone was to approach me and say like hey matt i read canary do you detect powershell um i mean like i can't in good faith just say yes without caveats right it's so broad and so we need to start getting into the mindset of scope and being deliberate about the scope that we define and build coverage around and another goal is to continually

increase our confidence in our ability to detect and attack tech being so i've already mentioned once i'll say it several other times detection is a process it's not an end state and so the best thing that we can do is just over time gradually increase our confidence like even if we're going to apply some of the techniques that we discuss here today in approaching things with an adversary mindset finding all the variations of any given technique we're still not going to have absolute confidence in 100 coverage like that's just not possible like this is one of the reasons like security researchers are so great uh anyways because they're continually challenging our assumptions and it's not my full-time job to be a

security researcher so like i don't have the time or resources to be challenging all of the assumptions of our um corpus of detection analytics so um there's definitely give and take like as detection engineers we can start that process um but many of the security researchers out there can help and and also um like malware developers like can help build up that confidence over time so we do our best and then detection coverage lies on the foundation lies on a foundation of assumptions all right again this this comes back to confidence like we can only know so much like there are known and unknown assumptions the known assumptions ideally when we build detection analytics are things

that we would document and the way i think about those known assumptions is those are anything that we document that we're saying deliberately fall outside of the scope of our detection analytic right um and that and that's okay so if attackers started coming around and abusing all those assumptions that we don't have a cover for uh coverage for again that's okay we can go focus on that some other time when it's prioritized accordingly but we we can't just cast a wide net and try to detect everything we have to be realistic um confidence is derived from our ability to define those clear scopes again and measure coverage within that defined scope and so with regard to measurement

we're going to be talking about testing and validation quite a bit a little bit later on once again i'll be harping on this detection is a process and not an end state if you can confidently say that you detect power shell are you just going to go move on and just never care about again because like you have some arbitrary confidence that you'll always be able to detect it in all cases like but that will never be the case okay so now i'm going to describe the methodology that i apply um in many cases to detection engineering applying that adversarial mindset now you're not going to have to follow any of these in sequence think of this just as like a cycle

and i'll go back and forth between them depending upon the scope of the technique but the first step is to define an initial scope and this can be really really easy and we're going to walk through this process step one why don't we go to miter's webpage and actually learn what this attack technique is and i'll show you how i kind of read between the lines when i'm defining an initial scope for a detection analytic we'll use things like open source intel but then we'll also transition into the next stage which i call mapping or like identifying key terrain this is where we can apply concepts like static and dynamic reverse engineering to identify the technical components that

um that make up that attack technique so if we can understand the actual components yet the relevant executables the dlls any uh specific like consistent network connections registry modifications if we can be confident that within our scope those things are not going to change then that's where we can really start focusing our analytics because that's where we can apply those detection choke points because if any of those components are not successful well they're required in order for that attack technique to be successful so if we can focus our detection optics on any one of those finite technical components then we can prevent ourselves from expanding our scope unnecessarily and also really hone in on adversary behavior

the next step is classification so here's where we will identify technique variations and you'll hear me say this a lot think of a technique variation as like a sub sub technique so if we keep talking about scope right we want to keep con like start with a relatively expanded scope but try to constrain it appropriately over time and so a technique variation is we're going to talk about mshta there's a handful of different ways to invoke mshta right you can use it to download an hta file remotely you can execute it from disk you can execute it in line right there's all these different ways and there's some additional ones we'll get into i consider all these to be

technique variations and so we can identify those through open source intelligence right like whatever the adversaries are doing like if we're talking about coverage like that should be the minimum set of coverage that we have is against in the wild threats but we'll also want to go against like uh theoretical variations as well um for those theoretical variations is where we extrapolate uh those variations through reverse engineering and also applying that adversary mindset to understand what an adversary has direct or some level of control over for each one of those variations and we'll go into some examples next this is where we really put our adversary hat on and consider what an attacker has direct control over

all right so i mentioned that with mshta you can execute a script locally on disk well does that script have to have a specific file extension mshta.exe does that have to be mshta.exe or can that be relocated renamed right so we'll go into this and put our adversary hats on and think about all the ways that we control so many things that might challenge some defender assumptions next we'll we'll apply everything that we learned previously and tie all of that to any relevant detection optics right are there process process start events module loads network connections all the stuff we're typically accustomed to but it's going to be more tightly constrained to what we've learned through our research process and we're

just going to be that much more confident as a result in those detection artifacts and not make as many assumptions as we would have if we just detonated a sample in a sandbox and observed in like a procmond trace uh what what popped out on the other end this is definitely more of a deeper or holistic approach next is uh to validate so this is where we're actually developing code um i mean if we if we were a red teamer we'd be developing a red team tool to uh to leverage this technique but as detection engineers uh we're think of it just like we're building like unit and functional tests to test this one technique so we're taking all the

technique variations that we identified and we're baking those into code and we're so we're implementing parameters that allow a defender or a detection engineer to flip all those bits that an adversary would have available to them as well and then we can write unit tests and functional tests on top of that to really measure our coverage and i've got some code that i'll show you that that we use to do that and then rinse and repeat so again this is a cyclical process you may have gotten to the last step of validation um only to discover or realize like oh yeah there's this one additional variation that we might want to get coverage for so you may have to

do some more reversing some more testing some more open source intelligence and repeat this whole process until you're confident that you've defined your scope appropriately and can move on all right so um i think we can all collectively uh just get better at asking questions and uh the the general this morning i i think made some good points about like challenging assumptions and uh that this certainly applies here so this is like the canonical question that is very reasonable from like say a prospective customer or you know the latest threat drops on on twitter um there's a whole lot of fun around that maybe um malware authors are starting to weaponize that and so there's a

legitimate concern hey vendor do you detect this thing right well of course you would love to give a confident yes answer but many of us know that's usually not possible without caveats all right so do you detect mshta so well what do we mean by detect this is not what i'm going to be saying to the prospective customer this is like what i'm thinking in my head because i don't i don't want to be like well actually like do you actually know what mshda is like it's not you know it's complicated it's not it's not that easy all right um so what do we mean by detect mshta what do we actually need to give that affirmative yes or no do we

detect it well the next question i would ask to myself is well what is mshta like i know intuitively what it is i know that adversaries abuse mshta.exe but is that it or are there other avenues of executing hta script code i don't know but we'll investigate that and what is required to see mshta based on the scope of my specific definition so you see i've highlighted detect and c so my philosophy is you can't confidently detect something if you can't confidently see it in the first place and this is what um identifying those technique variations is so important for is uh and identifying the relevant optics for each variation so if you can't see

um what those would look like in in your optics uh then you're not going to be able to confidently discern benign from non-benign in the first place so we have to have good optics coverage in order to have good detection coverage and then what is required to discern benign from non-benign in my mind it's generally easier to scope the set of benign behaviors than it is to scope non-benign and when i say non-benign i'm including suspicious and malicious categorizations right suspicious implies some ambiguity whereas malicious is like yeah that's overtly bad that we can all agree on right next if we want to confidently label something as malicious is there any supplemental data beyond the original scope of our

optics that might be used to enrich the the event so that we can be more confident that something actually is malicious so that we're not just generating false positives all day so do you detect mshta heck yeah i do but with some reasonable caveats all right all right so let's apply this methodology so i mentioned the very first thing that i do in the the scoping phase of this adversarial approach is like literally just go to the website miter attack all right so let's read between the lines here a little bit so um first thing that popped out to me is mshta all right that first question was that i had to myself is what is

mshta well it says here adversaries may abuse mshta.exe to proxy execution of what dot hta files okay those appear to be microsoft html application files and that's nice they have a link there so i can go maybe reference that and learn what the heck these things are in the first place but but uh some some questions i have already are is it just mshta or is mshta maybe just like a wrapper for some other core um like maybe like functionality built into a dll that i might be able to reference somewhere else and then hta files is that it or can you supply other file types file extensions i don't know yet so and javascript and vbscript okay

i've seen that a lot in the wild but is that it i don't know and what is javascript and vbscript in the context of mshta so we'll investigate that apparently you can also execute inline scripts and they give a cool example of that there they can be invoked directly from urls so right there i see two unique technique variations inline script execution and execution from a url so again think of those as like sub sub techniques in mitre parlance where were we okay and then mshta stood out to me right does it have to be mshta can mshta be renamed relocated to notepad.exe i don't know it's certainly worth trying and building some test coverage

around this stood out to me so i'm like trying to think as as a defender if i was to build some like command line coverage around this like is vbscript always going to be consistent or are there other options i mean i mentioned that javascript is supported so what would that look like and are there any others i'd like to investigate that independent of if they're abused in the wild or not and then lastly this stood out to me this might be something that i might want to investigate the the validity of whether or not it bypasses application control solutions okay so next we're going to move on to open source intelligence and actually see

how things are being abused in the wild and as i look through all these again i'm trying to enumerate those technique variations what stand out to me is like unique features of each procedure being executed here so procedure like the ttp tactics techniques and procedures we're looking at specific procedures here particular instances where the techniques are invoked so this first one here looks like that um that inline script execution paradigm this looks a little bit different like in the one in mitre it have bb script so this one has javascript colon and then script code goes in there again i'm going to ask are there any additional script options that i can supply there is there anything else in that script

code that might be required so as a detection engineer what i really want to get to the bottom of is what are the things that an adversary has the least amount of control over so in the case of inline script execution is it required to have the script engine colon script code in there if so i might be able to key off of that but i can't be confident in that detection analytical logic unless i capture the set of all possible script engines and so we'll we'll investigate what what those are comprised of all right next uh this looks like being downloaded and executed from from a url this one is a variation of something

that we did see in the wild which i thought was really interesting i don't recall if it was an actual crl file but what's really interesting and this will come up in a little bit is you can embed hta scripts in any arbitrary file format you want and mshta will read it from from the end of the file so you can just tack it on to any file type you want including a crl a certificate revocation list which is probably going to be signed anyway so there's a potentially really good evasion opportunity okay uh this one was really interesting when when i first saw scratching my head a little bit like how this even works

but it should make sense once we start reversing this how this how this is possible but you see that the syntax is somewhat similar in the sense that the script engine is there so script engine colon right so i'm noticing these patterns as a detection engineer the difference is the syntax is a little bit different that follows a colon and it's also run dll32 executing which is super interesting okay i just made this one up but this was like a theory that i had like what would prevent someone from renaming mshta to notepad.exe and instead of evil.hta renaming it to notes.text is that possible right we don't have to just rely on open source intelligence like

we're also intelligent ourselves to create hypotheses that we could easily test whoa sorry okay some other ones i'm just throwing in here for reference when i was researching this again this is where like security researchers are constantly like challenging our assumptions this was a cool one lethal hta so i didn't realize that hta was also an implemented com object which means that hta itself actually made for a great potential lateral movement technique so this is something that we might want to consider in our research and our tests as well to build out coverage for and then finally i've already mentioned this so a polyglot file is a one file on disk that represents different formats

simultaneously so this was a really cool article where again it turns out you can just append hta content to any file type uh whether it be a pe a certificate revocation list like it doesn't matter mshta just wants to read it from the back of the file so there's some really interesting evasion opportunities in that scenario there so we're taking our notes and we're annotating all these different technique variations and all these variations are going to have their own distinct detection scope there may be some overlap but many of these look different um because like there's going to be different detection optics unique to each one of these variations all right so here's the fun part

actually digging in and learning a little bit about how these things work um identifying the technical components so i'll be glossing over a lot here but it turns out like if you just throw mshta in into ida it's a very very simple wrapper for the run html application export function in mshtml.exe like that's that's all it is it just wraps that function um and uh when i annotated the uh the parameters or the arguments that were being passed to it um what stood out to me it was interesting these four arguments mirrored the um the function signature for uh if you want to write any function that you intend to implement for run dll32 these are the parameters that you would

implement in in your function and that's what i was seeing in ida and so it was cool that um using that information i was able to arrive to the same like a similar conclusion to what was revealed was being used in the wild that um uh what was it yeah the run dll32 instance like it didn't really make sense to me at the time how that was even possible but now that i could confirm in ida that run html application was indeed uh designed more or less to be used with run dl032 um a security researcher or malware author at some point had the same realization and started abusing that for probably evasion reasons all right

so we've expanded the scope of our detection analytics a little bit right so we started with mshta.exe which was mentioned in the mitre website and now we've extended it to run dll32 as well all right so here's what we've identified thus far run html application in mshtml.dll i consider it to be the lowest level interface to the execution of hta content like if you go and reverse that function like it's very complex there's a lot going on but sometimes we have to draw a line in the sand and say like enough is enough like is an attacker going to go to any lower level than that function to execute hta code that's probably unrealistic so here's where we could

probably draw that line in the sand and not go any deeper than what we did can run dll32 be used to interface with html application well malware authors already confirmed that for us but if they hadn't we already had that suspicion anyway based on the implementation of run html application how it implemented its parameters what other built-in os components reference one html application so is it just run dll-32 and mshta so we'll investigate that and considering an adversary you could just call run html application directly what would prevent them from just implementing their own code to call that function instead of using the built-in utilities so first i would ask myself like is that actually being done in the wild

and what does it buy an adversary to do that um that just because i've come to the conclusion that it's unrealistic that they would do it doesn't mean that they won't but it helps me scope my detection analytics so uh because i came to the conclusion that it doesn't really make sense for an adversary to do that because they would have already had arbitrary code execution why would they want additional hta code execution on top of that then i tell myself okay i'm just not going to worry about that it's not going to be in the scope of my detection analytic and i'll move on if they start abusing it later on then i'll start

caring and maybe focus on that but i don't care about that right now right so would not scoping this constitute a detection miss and i would say no if an adversary started abusing the run html application using their own custom code i'll say no we didn't miss it because we like it was a documented um assumption right so we didn't catch it but it didn't actually miss our detection analytics because it wasn't in our scope all right so if it's not in scope then you just document that as an assumption accordingly all right so this is a really silly thing i do but it's really effective so this answers the question that i had here of uh what other

built-in os components reference run html application so this i'm just constraining my search in powershell to the system32 directory but realistically i probably just do this throughout the whole file system i'll say show me all exes and dlls and then select string is like the equivalent of grep that have run ht the string run html application as both a unicode and an ascii string um and then if anything came up besides these two here then i i would dig into that like rinse repeat throw that thing into ida and try to figure out if if this is something that an attacker could actually abuse so this was a good sign for me that it only

returned what i expected mshta and ms html.dll the two components that i had already identified um but i would encourage anyone to do this like again this is kind of silly like just grepping for strings um but it's i have found it to be extremely excuse me extremely effective in scope

all right that's better all right so next we're going to classify we're going to start enumerating those technique variations and i'm going to try to go through this this quickly so of the ones that were identified at the end of this research were the following local hta file execution on disk and i've asked this before are there any file extension limitations or can i just do evil.text or whatever uh inline execution via protocol handlers like javascript colon vbscript colon we've seen some examples of that are there any more support for multiple scripting engines all right again we've seen that are there any more how are those defined in the os how would i enumerate these

supported engine components this isn't really well documented anywhere so i need to figure that out those polyglot files right there's another variation there remote execution via unc pass is an option

uh i mentioned that lethal hta project so execution via um com and dcom remotely and locally execution via emulation of a double click i'll show you an example what this looks like this was interesting when i discovered it that attackers were doing this they were doing this to to break a process hierarchy so that you didn't have like a direct parent-child process relationship next modification of the um the executor like mshta.exe file name and or path yep now i want to briefly discuss breadth and depth so using this paradigm that we've been discussing the way i think about breath so the breadth of your detection coverage increases with the technique variation count okay so here you can see those uh eight

identified variations uh that con to me constitutes breadth of coverage depth of coverage increases as you develop more tests for each one of those uh variations okay so let's put our detection engineering hats on uh and start thinking like an adversary all right so here is the contents of one of those hta files all right you can probably see like some of the script code that's going on there nothing really malicious in here this is just a test that that i ran but what are some things that stand out to you that perhaps an adversary might have some control over or what do they need in order for the hta file to run so like you probably need the html and

the head tags the htma the hta application tag what are some things they might be able to control maybe that window state attribute okay that could be looks like it's pretty convenient for them to like hide windows um script language okay this one has vbscript what other ones are supported and then the actual script code okay so this is a pretty minimal hta and the things that stand out to me that an attacker probably needs and of course we can validate this is the script language and the script code and then all the other syntactic sugar that's required to execute hta stuff

okay we have another example here all right this time they're doing in-line execution here's something new to me that stands out so in the other examples you saw you saw there's a vbscript colon and a javascript colon now we have an about colon so when i was doing this research like i didn't even know that was possible one of our incident handlers was investigating something and noticed um that an attacker was abusing this reached out to me i was like oh like that's crazy that i didn't know that was possible so i was able to turn around some uh coverage pretty pretty quickly um in in the test code to to support this okay so we've seen three different

instances right what else is required though well there's the executor so mshta.exe but that can probably be renamed and relocated the script language it's probably a requirement and then the actual script code again okay so if we were defenders what might be some of the most robust things that we would key in on in order to reliably identify this inline execution as hta inline execution because could this just be notepad.exe running and then what about that about colon what else is possible does hta colon application have to be there right so we need to be thinking about the things that an adversary has little to no control over and focus our detection analytics on that

because otherwise it's just going to be brittle and subject to evasion all right okay so let's start thinking about possible optics that we could build uh detection analytics off of uh command line is pretty obvious now just bear in mind command line and attacker generally has the most level of control over they may not have complete control over that and like we just discussed what they may or may not have control over so we have to be very consid uh considerate of that because they exercise a heck of a lot of control over what they can supply at the command line next would be process path and file name an attacker has a tremendous amount of

control over that so are there any possible mitigating controls that could be used to identify when an attacker renames and relocates mshta.exe or rundll32.exe well we could consider original file name um raise a hand does anyone know what original file name is and would be able to explain that to me

yeah within the is the property within the file that has the original name of the executable um yeah so like as a defender why would that be useful like why would that potentially be more robust to key in off of versus the process path and file name

[Music] right so they might change the name of the file on disk right but it's unlikely that they're going to change the original file name in the executable um especially for a built-in executable because in doing so they would invalidate the signature so they might as well just be dropping their own malicious code but thank you for answering that

okay so these are the identified original file names for mshta.exe and run dll-32 this is an extremely powerful analytic for defenders if you can baseline this set of uh like known law bins and if you're doing your detection logic off of original file name instead of the more naive like process path and file name where you're baselining it against the two right so like you would expect mshta to be in system 32 and syswow 64 those directories right so if in your analytic if you said okay we have a process event and the file name is food.exe but the original file name is mshta.exe dot mui well that that deviates from our baseline so we'll we'll detect on that

that's extremely effective

okay the script host dlls so these dlls are uh they'll be loaded depending upon which script host is executed so you saw like the vbscript colon javascript calling um maybe these could be relevant in our detection analytics i don't know uh i'm leaning towards no because uh you can get the most you you can get a lot of context from just command line and the original file name but i i consider everything and then i just whittle down what is absolutely necessary to build a detection all right is parent executable path going to be relevant what is hta executing from is parent process windward dot exe is it explorer.exe is it powershell.exe these are all going to be very very

relevant in our analytics is it making any external network connections does mshtml.dll load that's what implements that run html application function this may be relevant maybe not i'm leaning towards no unless we care about the instances where attackers implement their own version of mshta.exe because that would have the effect of loading mshtml.dll into um an executable with low reputation right or or wouldn't be signed um but within the context of my scope i don't really care about that dll load but i'm just trying to be exhaustive here in considering everything that might be relevant within the confines of our scope do we track hta file rights i don't know that might be difficult because it's just a text file

and it could possibly have any file extension right so that might not be very realistic okay so let's talk about atomic test harnesses so here's where we're actually going to validate all these things and actually implement all of our findings from our research so this is a powershell module um that's up on github and the purpose of it is to validate all of our findings in a way that's abstracted such that you don't have to be a subject matter expert on mshta you can just go read the help and flip all the relevant bits to invoke all of those technique variations without actually having to know how to craft the html code to execute everything all right so um the

underlying motivations to this was um i can't detect what i can't see okay so uh yeah i could go detonate some malware in a sandbox i could go run some red team tool but those weren't designed with detection engineers in mind i might get some good ideas out of it but if i don't have something that i have like more direct control over i don't i'm not going to be as confident in what i'm able to see to um infer detection logic and what should i be looking at anyway right like uh as a non-subject matter expert on this ideally like i really don't want to become a subject matter expert on hta like i just don't care like someone else

ideally should research this and just present me with what i really should be focusing on and so that's what i aim to do in uh in the functions in atomic test harnesses okay and how do i distinguish technique artifacts from test runner and irrelevant os artifacts so this is really frustrating for any detection engineer you go detonate a malware sample like with a procmond trace and then you have to go weed through everything and be like is this like an os artifact is this actually related to the technique i don't know maybe i'll go run it again and see if it exhibits any any patterns that might be related to the technique um and then

whatever you run it in that runner utility is that generating its own artifacts that you have to like weed out too so like this happens with like uh atomic grid team as an example like if you're running tests in uh invoke atomic test like how do you discern powershell.exe running the test runner from the actual test artifacts that's that had always been a perpetual challenge for me and so this is something that i wanted to to help solve and then what does it a successful technique look like so in the case of uh hta code how do i know that the script code that was embedded in the hta actually executed for as great as atomic red team is like

it's it's one and done you go fire it and then you have to interpret the results yourself so you may not know confidently that the thing actually ran even though it said even though it claims that that it did because it didn't shoot out any errors um but i really want to know like did the hta code actually run and so there's some automation and some like eventing in play to um to confidently know whether or not your quote payload ran successfully and then uh how can i replicate a technique without needing to be a subject matter expert again like i should just be able to go read uh the help on this function and know like what

arguments i can pass to it in order to have some confidence that i'll be able to detect this without having to form the payloads myself and then must i be limited to attack procedures supplied by malware authors or proof of concept like red team tools those are great for developing an initial scope but they're not holistic so we can miss a lot potentially and then i also wanted to test one thing and test out one thing really well in order to confidently measure quantitatively what our coverage is for a technique all right so let's hop into the demo here

come on now

oh man

freezing up on me

well this is no fun

here i had everything set up so beautifully

um well

all right so i'll do my best to explain things while this uh i try to reload this um so i've created a powershell function in atomic test harnesses called invoke ath html application let me try to resume this here this is where i pray

man

well all right so just gonna have to do my best to explain things here um so what you would do is you would run this function and it presents a ton of different options right so all those identified technique variations that we talked about that we identified earlier all these here are implemented within that function in a very abstracted way so if you wanted to execute an hta file locally it'll do that using hta as the default file extension but then it also lets you override that so you can supply any file extension you want so foo.txt what have you um it also lets you supply any executor file name that you wanted so you rename

it to notepad.exe test that out and it will let you do that so it supports all of these now there's still a lot of parameters that it implements and so with all those options like it can be pretty easy to be overwhelmed and let's say i just wanted to test our coverage on every one of those techniques as well as all of these supported um uh like say script engines that are supported um well there's also um uh unit tests implemented for that function and so i want to say there's like 30 different tests implemented um because for each of those technique variations there's also like different parameters again like different script engines different file extensions

and it tests all of those in an automated fashion and so if you're getting green on all of those tests then um well one you know that each of those executed successfully um and then you can go from there and then see like ask of your edr whatever your sensor analytics are did i actually see that and if you are then you've really built that foundation of optics coverage upon which you can develop confident detection coverage all right so um apologies that the demo did not work out um but you can go to the github um again it's atomic test harnesses try it out ask me questions manifestation on on twitter so in conclusion this methodology can apply to

literally any attack technique i focus on just hta just because it was relatively uh constrained for the um for the scope of this talk but again like scope is key you have to be deliberate in order to not become easily overwhelmed okay a minor attack um i to remind myself of this that it's a guideline and a reference um it is not like the gospel uh and just because it says something does not mean it's true uh if that's the case like please reach out to correct them but they don't always get this their scope may be more broad or too narrow for your use cases so it really is up to you use it as an as an initial guide in

reference and scope is derived from the technical components the level of adversary control that they have over each variation the available optics and whatever your available resources are as a detection engineer like this is a relatively expensive process to squeeze this coverage out of any attack technique and um don't let perfect be the enemy of great all right um that there is no perfection in detection engineering you're never going to have 100 confidence of anything because there's always going to be unknown assumptions in your logic and then finally confidence is misplaced without testing like i could state qualitatively that we have confidence but you know show me the money right um show me those tests that you're running to exercise

all of the identified technique variations and all of these supported parameters for each one of those variations and then do i see it on the other end and if i see it on the other end then i'm confident that my detection analytics are going to be resilient against evasion so with that i thank everyone for your time [Applause]

you