Diamond Model for Intrusion Analysis: What You Need to Know

all right cool all right thank you all for coming um my name is andy pendergast i am uh that's a company i work for cybersquared and that's our product but i'm not going to talk any any more than that i have to talk into the microphone apparently what i am going to be talking to you today about is the diamond model for intrusion analysis so what is the diamond model um it's an analytic methodology we came up with about eight years ago to track group figure out threats um admittedly this was mostly focused on what's now termed apt advanced persistent threat forgive me for saying that i know i feel that ball warming up that ball warm up

who is it for so if if you are a defer analyst if you are if you deal with threat intelligence in any sort of way this will help you i promise if you are a risk mitigation planner get out now because i hate you you but it's useful for you guys too um and as you can see there's there is a paper behind this um it's up on our website but it's published by dtik um so you can get it off dod's pages too it's a great read with coffee it's only like 70 pages i usually like take it in the bathroom with me when i brush up all right so why did we why

why did we create this so i mentioned about eight years ago me and a couple other guys chris betts and sergio we needed a plan um and uh chris and normal chris speak that he actually does kind of look like captain america um told us we needed a plan and then sergio said all right i got a plan let's let's just jump out of an airplane and figure it out we were being you know attacked by at all sides by different yes ap disease and these bug guys here um and we needed a way to sort them out right we needed a way to characterize and figure out all right well these all kind of look the same but they're

not really the same they tend to change up every once in a while and we kind of need to know how to counter them and we need to know how they're different from one each other in order to be able to counter them smartly so hence the diamond was born what is it used for today so it is both a cognitive and formal model for um for figuring out intrusions right the cognitive model is used probably by hundreds mostly in the dod today but increasingly external thread intel analysts people with some sort of threat until responsibility in their role and it is also a foundational concept that has been used in taxonomies and languages standards and

protocols like sticks that are emerging and becoming more and more prevalent within this space it is itself not a taxonomy it does not seek to list out every possible way that an intrusion can be characterized but it is foundational in that concept and it allows the analysts to apply things like sticks in a much smarter way it is both set in graph theory so based and we've used it in our product as the basis for the data model that we use there so um i mentioned it's graph based the reason why it's called the diamond model is because each cyber event you can't throw a ball at me for that is is made up of four vertices

the adversary at the top these are typically the bad guys that want to do bad things to your network and steal your stuff or take your network down they can be characterized in a number of ways you might know something about them might know their email addresses you might know their twitter handles you might know phone numbers you might know where they live and if you do you should go drop a bomb on them or maybe not yeah you might know the computers that they use you might know where they have fingers on keyboard that's really great if you do most people don't but that can be inferred and we'll talk about that more about as we talk about the model as a

whole the capabilities um if i had one word to describe what the capabilities are it's typically malware right and this can be described and and uh enumerated ad infinitum right forever down and there's numerous ways to call these out but at the very top level these are the capabilities that the adversary uses in every event to carry out their objective against the victim the infrastructure is the way that they get there right so they're moving through the internet somehow amazing imagine that and you can characterize that infrastructure based on ip addresses domains emails how they're owned who owns them do they do they control them do they subvert control of them can you go and call the the isp and tell

them to shut them down can you call the the mom-and-pop shop that they're using as a command and control server and talk to them and see if you can't figure out something to look at them a little bit closer perhaps can you call up the webmail provider and ask them you know what they know about that email address perhaps if you have that relationship so there's all sorts of ways to leverage the infrastructure even if they're using it against you to use it against them then of course the victim and this can be enumerated in in numerous ways as well but typically the personas network assets email addresses or the specific data that they're going after

in your network now um there are some meta features that i mentioned this is one event right so the meta features are primarily used to link specific vents together so you have a timestamp when did this event occur easy enough we do use phases this is commonly people typically use the kill chain phases in here to to characterize the phase of attack that was being used the result was it a success was it a failure when you're building the graphs you probably want to know where you stopped if you stopped an intrusion and or whether or not the adversary themselves just you know they had their own fail and couldn't get out of there the directionality um those little

abbreviations itv this is typically infrastructure to victim infrastructure to infrastructure adversary to infrastructure this characterizes where within the attack space or where within the internet the event took place it may not be taking place on your network yet it might be taking place external to your network um and then methodology the class of activity right is a spear phishing message is it a watering hole attack is it a ddos those are pretty pretty easy to understand resources so if they're using an oday it infers that they have some resources to leverage that oda right if it's if it's not publicly available how do they get it these are questions you might ask yourself that can lead you to unders

better understand the capabilities and the resources available to the bad guy right um are they are they out there buying odays are they developing themselves do they have a network of hackers working in a basement in a foreign country developing things are they paying off folks to get it did they get access perhaps to the software manufacturer that is building the product that they've exploited so we don't expect you to know all this we certainly don't i certainly don't so unknowns and uncertainty are welcome you're expected to have knowledge gaps as you're building out your understanding of a threat and your understanding of how to go about defending against them i mentioned the kill chain if there are

any lockheed martin people here please note that i did use your trademark and uh so what you see here is um a couple of incidents and we've we've grouped them and within here you see a kind of standard kill chain breakdown lots of people actually use different flavors for their kill chain right so you might have different phases of attack that's fine the diamond model supports that we don't really care what is important is each of these little events going down the kill chain uh going down the incident graph um these blue lines here represent a causal event so one event causes another right so there's a relationship between those and there's actually a confidence

associated with whether or not you know how well you know one caused the other sometimes it's very obvious sometimes less so and then these purple lines that go across the incidence these are correlations between specific nodes or vertices in in the diamond events that represent commonalities right across events and there are competencies associated with these relationships so you may be very confident that it's the same thing or you could be less confident based on the availability of the malware the the way the infrastructure was used uh maybe it's in the same class c or same you know same net block but it's not the same ip address and it gives you some level of confidence

that they're related but not a hundred percent um and i mentioned that unknowns are welcome so if you can tell i'm not sure how well it shows up here or not but this guy is kind of dotted out it means you're in the exploitation phase for whatever reason you didn't catch that maybe you just caught the c2 coming out and you can then infer based on the knowledge you had on incident one that since these were the same that maybe maybe there's something it can point you to where you need to look at faster to look at incident too right is not rocket scientist rocket science not rock i'm not a rocket scientist this is not rocket science

all right so creating activity groups um basically what we did on the last slide was we created a very basic activity group you saw that incident one and incident two were grouped together and sent three and four were grouped together those are two separate activity groups within the diamond model it is a formal process to create activity groups i like to think of it as fourth grade science class where you build a hypothesis and then you test it so the first thing you do is define the problem right so in this case it's an attribution problem those are not the only problems you can define they do tend to be the first ones that we do just because you can

once you have that you can then go do other things and define other problems with that activity group and i'll talk about that in a few slides but so say you know a simpler way that you typically hear this around the office or you know like oh i think we got hit by apt-1 well how do you know what is apg-1 anyway yeah what makes uh i'm going to skip the second one because i don't think red october is really that sexy anymore but so say what makes um poison ivy 9002 and plug x you know if you saw that on your network what would you be dealing with well probably probably some bad stuff from china right if you're

familiar with what those are this is a 9002 backdoor plug x is a openly available while not openly available but used by several different groups that seem to originate from china and poison ivy is kind of like a openly available back door as well that's used by first stage by many of these guys but how do you know just because these things are used that it's the same set of folks they're after the same thing you really you don't right um but what we're describing on in these questions are really represent the next step in creating an activity group and that's feature selection so what criteria what points you know what nodes on the diamond what malware what infrastructure

what ttps tactics techniques and procedures define this set of activity as being a set of activity right and once i have that i can begin to test it but watch out this is uh the first step and all intrusion analysts all threat intelligence analysts are guilty of this at one time or another i know i certainly am you put together a group of features that you believe in the beginning are like this is what makes apt-1 apt-1 i know it i know these guys i see it all the time and then something comes along that busts your whole theory up right that's okay um we the diamond model doesn't prevent that it helps you to think about the problem

before you before you make those calls but it expects there to be mistakes and it allows you to go back and reassess and so when you're when you're creating a group um you once you define those criteria they become your grouping function so out of all of your knowledge if you want to say okay well i think i understand what apt-1 is based on well mandiant put out a pretty good report and i've got a little bit of knowledge here here and here about them so i'm going to use all that knowledge to create a grouping function based on the criteria that i established and then i'm going to take all of my data wherever it came from my internal

networks uh the the thread intel reports that i have access to maybe i've got some feeds from some vendors coming in and i'm looking at open source stuff i'm going to push it through that function and see what what sticks as apt-1 right and i'm going to continue to do this and it may be and it's expected to be that as i do that i'm actually going to learn some things that i didn't know before that i can then go back and apply to my grouping function and make it broader or create other sub you know other sub functions that would also make something apt one separate for my initial function once i have this this is where i can do

some more interesting things so i say i've got a pretty good understanding of what apt-1 is well now i can start to trim them i can see how they've changed over time how did they change when we did x y and z how did they adapt when mandian busted up their their all their intels and and threw them all out there in the in the wild right did they continue to use the same tools and infrastructure did they drop it all and start all over again and how did we pick them up can you figure out what they were after usually you can do that after one or two incidents you can go down this list you can

compare groups by each other to see which ones have more capability or others or really what's more interesting and what's probably most relevant for anyone in the room is once you have a good understanding of the set of capabilities and infrastructure and you understand their intent you can make some pretty smart decisions on how to counter them right you can start talking to your ciso you can start talking to your c staff about where you've got gaps in your network based on the capabilities that they bring to bear against you and start filling those gaps right and of course it's going to be a cat and mouse game of course as soon as you put

up a new you know some new defenses they're going to try to find new ways around them but we all have jobs for a reason right that's the security industry so and the last step here is redefining so as i mentioned before if you make a mistake you may find that i've got some stuff that doesn't quite fit into my bubble of what i understand one group is versus another then i can go back to the drawing board and reset my grouping function and and make sense of that another kind of silhouette here is like yes yeah i understand i i like to to group adversaries and make little pretty graph pictures of of what you know everything that apt-1

does and i'm just using them because that was the first thing on my slides you can apply this to anything not just apts you have not thrown a ball with me yet you're very graceful yeah i can avoid that one so so whatever you know if you're talking about crimeware the this still applies if you're talking about um guys out of russia trying to steal your credit card numbers from from your network or point-of-sale malware or whatever right that they have their own tactics techniques and procedures they have their own malware they have their own capabilities and they have their own infrastructure that they use and it may look a little bit different than the ones i'm using

example as examples here but the model still works and we've tested it with various you know different groups and whatever methodologies and intents whether they're looking to ddos your network it all still works you can always still characterize them and so once you have them characterized as i mentioned you know so what um the so what is is if you if you start making decisions when you're blind about what what products you need to buy to protect your network or you're a consultant and you're making recommendations to the the customers that you're going on site with and discussing things with if you make bad recommendations because you don't understand the adversary you don't understand the risk that it

presents to the network then you're wasting you're not only wasting security effort but you're leaving the the entity that you're trying to protect still at risk right so it comes down to know thy enemy and this gives you a better way to know your enemy all right and so this is more for the um the geopolitical geeks in the room if there are any um there becomes once you have an understanding of uh you have that activity group you can create what's called a meta diamond which you know if you imagine like lots of little dots here about all the capabilities that they have and lots of little dots here about all the infrastructure that they have

and the various incidents that they've targeted or exploited your network or gotten in um you can infer up here this this relationship between the adversary and victim there's something that they want from you right what is it that they want what are they trying to get at that can help guide your decisions on what to protect right where are your crown jewels and then here the horizontal we're not i don't control the horizontal and the vertical but the horizontal here the capabilities in the infrastructure this represents the the combination of both and also allows you to search if you know there are adversaries out there that want to get your crown jewels want to get your goodies

but you don't know the capabilities and infrastructure yet you can begin to hunt right you can begin to look for them using either known techniques you can look at protocols that are popular to exploit or whatnot and begin to hunt for them and then and then perhaps find some of these targeted things this is more of an advanced move typically you have to have already dealt with the ones you know about before you start looking really deep for the ones you don't know about yet all right so um i've kept this pretty well that's loud sorry i've kept this pretty academic up until now i'm going to go through a use case that we applied it's public it's one of

our blogs that we did last year conquest actually been helped out quite a bit with this that's right so um the the origin of this is that we were out there doing what's called a victim-centered approach we're looking for interesting artifacts of intrusion attempts that might be targeting people that we care about and in all transparency we're just hunting around virustotal to see what what people submitted and we found this um this document that we actually originated from the commanding genera or the g3 of us army pacific exercise division and it went out to a bunch of folks originally um also in in the us army but we found it up on virus total and it was trojanized

and that seems kind of interesting you know who would trojanize a document like this who would weaponize it um and the subject of the exercise uh or the the message was the conquest exercise which was a joint mongolian and u.s exercise so just from this document existing we can infer an uh a kill chain right we can infer uh create a hypothesis on what may have occurred here so probably on the recon face the bad guys were out looking for something that they could weaponize right that would be of interest enough to for someone to click on and then so obviously they do that and and again hypothesized it was likely emailed or staged somewhere where someone

that was targeted would click on it or download it and click on it and then you know the exploit happens we'll talk about what the exploit was in a moment installation and then profit for the bad guys uh we did notice there was a specific um post in the command co control protocol that went back and forth that allowed us to very quickly identify it and associate it with other malware that was known so very quickly once we had this malware term barco fork by some of the av hits that occurred we found another piece of malware that was another stage document or trojanized document that actually dropped the same exact md5 called out to the same exact um

host names there so we can infer now perhaps there were two incidents one containing our first piece of malware and the second one containing the the second the second piece of malware actually had a very similar theme also would have been interest of interest to mongolian parties so pivoting is actually one of the friendliest and easiest use cases for the diamond it's how the diamond model was discovered it's what we initially did when we were looking at this and it allows you to grow your your grouping function there and then also uh grow your knowledge base or your activity group as a whole so we started with that piece of malware right and obviously that is a capability

just by doing a little bit of light analysis on it it bleeds out some command and control domains for us great i love command and control domains because with each pivot i can as i mentioned i can grow my uh activity group but i can also grow my grouping function and each pivot represents as i grow new knowledge it either gets added to my activity group or it may actually be a method that i can enter into my grouping function with a certain level of confidence so then go and broaden out my known activity group as it gets broader and broader and broader as i enumerate all the bads that are out there for this group so i now have three

domains those domains all went to the same ip address so i can add that to my activity group but perhaps with the lower level of confidence because ips you know these domains are made to switch around ips maybe it's relevant for the time that they're pointing to them maybe after they stop pointing to them it's not relevant anymore but it's still a data point that i can use what's perhaps more interesting is they all have the same domain uh registering right they all had the same guy registering him and who is records that's also pretty interesting and with a high level of confidence because these aren't easy to change and once you have them well i guess they are in some

instances if they're black hole or whatnot but one if you have that domain there or that email there you can then go and see what else this guy's registered there are some gotchas if um they're using uh who is protect or other other type services but it's kind of amazing how often these aren't used so this gives us a high level of confidence for any other domain registered by yn right and one point of noticed and this is like the the kindergarten level of threat intelligence um it's where you start after the first pivot if you notice i i didn't look at the activity going on in my network i wasn't staying in my splunk instance or looking

at the logs to figure out what the bad guy did i'm now kind of looking at what he hasn't done to me yet right i'm looking for other points that i might be able to apply back into my network to look for other activity so with these domains you know i didn't have these yet i would expect if this was active on my network to see some callouts here right but i can pivot here and look for other domains pointing to this guy that looks suspicious and maybe look for those on my network right i can once i have this i can look for other domains that i don't know about yet that might be of relevance and point

those back on my network too and see maybe i've got more activity than i know about right so this can support an instant response cycle as it's happening so this is what we did essentially is we went and did some more pivots each one with varying levels of confidence but we found a lot more domains if you notice a lot of them have the same registered level and then we looked at the registrants for these domains the domains were probably linked by common ip addresses happening at the same time we can also look and see if any of those domains are related to malware that we didn't know about yet and sure enough they were so here's uh

if you look at the registrants these are the only ones that had yn in them 79 of course of course these are interesting but now we have a few other registrants of interest as well and as i mentioned at every step you can you can test and say okay well the registrants those are pretty high confidence and you can validate that by pointing it back to malware and it's even better if it's the same hour but you might get you might find that there are other p other types or other families of malware being read that are linked to call out domains that are registered by these guys being used for malicious purposes too so

those are things you would add to your grouping function what we found after we published the blog was that there is some related research that others did and we found that there's plug x malware so we started with barco fork malware but there's plug x malware also using these same callout domains so sure we'll add that to our activity group and even add it to the active the grouping function however plug x malware isn't just used by our bad guys they're used by several other bad guys so we have some domain knowledge that allows us to adjust our confidence level on what we add into our grouping function and so what we found were other

people seeing the same similar activity we're blogging about plug x stuff being used to target mongolians and that sounds like it's right sounds like it might be the same people but actually there's no strong link back to any of the malware that we saw or any of the registers registrants that we saw which led us to believe that actually even though others were linking it to our same activity group we would be more cautious than that and say it was probably a different set or a different set of activity the um we also in our own blog noticed that we did see apt-1 using the same hour at some time however uh none of the other characteristics

matched right so it assumed that even our barco fork malware was used by a couple different groups um and it was just having one link in in the diamond model isn't typically enough to make a correlation to say this is all the same activity and we saw several others here too so i hope that what you've all gotten out of this today is that as you're doing instant response as you're applying if you're doing thread intel you can use the diamond model just cognitively to test your own assumptions right that you can test what you know about an adversary you can test whether or not you can point these signatures at that and expect it to work

expect it to detect more or to keep them out or if you know you need to do some more homework and get better uh visibility over the threat based on your knowledge of it um this is just a primer you know we've gone less than an hour as i mentioned there's a 70 page paper available sergio one of the other co-authors has a summary that's about eight pages so maybe you only need tea you don't need coffee while you're drinking it or while you're reading it that's available on his blog and then the full papers both on our our website and dtx and i think we have a little bit of time for questions so

there are yeah i went a little faster than i thought i did good so it sounds okay so you get a definition of a verb but it sounds like you don't really share it's not like there's a central repository of saying here's all you know here's what we know about these bad guys so okay no absolutely not so this um kind of speaks to our product and i'll keep that at a minimum but so threat connect if you see it on the slides here it's built to be a community knowledge repository that's active right so it's a threat intelligence platform that supports communities so you can suck in and tell from wherever your own internal knowledge what other

people give you feeds that you have coming in bring it all in make sense of it and then you can share that to a community and community share it to each other you maintain things there may be some things for good reasons that you don't share right um and we won't get into the argument right but you can programmatically share your knowledge of a threat or many threats into a community that you trust within our product so um and of course you can have your own you know not not a product pitch there there are lots of way you can maintain an access database if you want god help you or an excel spreadsheet on everything

you have and share that to you right um there's a lot of strength in the structure of the relationships that you can articulate both within our platform and then other emerging solutions stixx is a great protocol for doing this too

yes right to get it done fast fast fast and to force people to actually go through all the steps can you talk about how the diamond model actually forces young others actually to correlate all those findings and actually which then leads to more so the model itself doesn't doesn't dictate whether things are automated or they're manual and the the automated piece is a little bit of you know everybody has their own secret sauce on how they're automating those or how they're establishing uh algorithms to make those um connections automatically or programmatically across different data sets of course we're doing this within threat connect and it is a point of research as we're always learning right and i think there is there's a lot

to be spoken for for machine learning techniques that can be applied here in different algorithms that can be used with different data sets however you know what i wanted to stress today is that you don't need that to do it manually and oftentimes it scratches the itch immediately when you're doing instant response that you can use these techniques cognitively um and apply them can we force anyone to do this no you know it's it's a technique it's like going to class and did you did you take what you got from the class and apply it directly does that answer your question okay

absolutely so the diamond model works well with other established means such as so actually i mentioned the the kill chain right it's there's another presentation i've given on how they work together very well where you can map your knowledge of the adversary from both a kill chain perspective and the diamond model to iden and then overlay that um with your own mitigation capabilities so you can apply it to a i think in the kill chain it's called a course of action matrix so you can apply it to a course of action matrix and find the best bang for your buck and how you could stop them at what phase of the kill chain there is another we work well with act

attack graphs um we we call what a blending of the diamond model and an attack graph and activity attack graph where you look across everything an adversary could do and then you overlay it with your knowledge of from the diamond model in a structured manner of what they have done right and that gives you a highlight of the paths they've actually taken and what you if you mitigate them here what pivots the adversary may make that you haven't seen them do before and you can make some you know informed guesses on which ones are more likely based on what capabilities you know they have versus how hard something else would be so good more questions

going once

so um i could bring up um what's that is wes in the room hey wes you want to bring up clark connect you want to bring up to our connect

so everybody wes uh wes works with us he is our and what's that yeah yeah or i can log in all right uh actually i've got it shred off so i can tether this might take more than a minute

so as you're getting that so you'll probably need him to help you out with the wi-fi here it's okay i'm sure the wi-fi network here's safe for my laptop right yeah nothing of great to prevent analytic bias when including malware families together or you're grouping your threat groups so one of the that's a great question thank you um yeah there you go so one of the axioms that i like to go by when using the diamond model is that and i mentioned it briefly in the slides that one connection typically is enough every connection should be weighted by confidence and you should typically have two or more high confidence links before you bring in you bring something into your activity

group so for those of us that actually have windows and offices where we work there is no finite line for okay high confidence means this low confidence means this yep so everybody in the public sector what's their equivalent so it does take a little bit of so it's iterative right you need to learn if you have no other knowledge base as you're mentioning like if if you don't work somewhere where you already have a mass database of what each group is or what you know what should apply to an activity group um you have to start right you have to find you have to start and maybe you're starting at ground zero right and but the diamond model will let

you do that it may just take a little bit long to iterate through and learn what make what we should make a high confidence versus low confidence but really quick just some things i can rattle off my head before you start linking things together um is is the malware publicly available you know it if you're seeing poison ivy um across your network and you see it in march and you say again in june just because you saw poison ivy does that mean that you're dealing with the same set of actors or the same threat probably not right maybe what other points of data do you have that you can correlate right so um and the the case with the

domain registrants well what do you know about domain registration well if i've i've registered the domain and i've used an email even if it's a fake email it's it's a point of reference that's frozen in time from the point he registered it right and that person controls it unless he seeded it to somebody else as a third party which doesn't typically happen it could but we don't see it happening very often so that's a moderate to high confidence linkage that you can have if you notice the whois registration changes then you go back and you change that in your in your model or in your grouping function um does that help answer some questions yeah

so okay yes i know i'm putting you on the spot so i think um what would be good to demonstrate and everybody wes uh works for my company he is our intrusion analyst threat intelligence analyst extraordinaire um what's with the uh scroll bar on this chrome uh why are you complaining about things i'm just just wondering i have no idea yeah so maybe you show one of your recent shares and subscribers that's fine it has comment and subscriber here so so um so actually a little bit of background on what threat connected we weren't planning on showing it here but it is as i mentioned it has community repositories of knowledge they're linked together if you show this is just a post that wes

has done the data is linked it you know describes everything in human readable form great so you can see what it is but if you go to the browse screen perhaps or click on your paypal phishing backend internet and um so you can describe things you can tag them you can have security labels and whatnot but what's important is the are the associations probably at the indicator level we'll go back to attributes so you can see all of the indicators that are associated to this specific incident every one of these there's a many-to-many relationship here so every one of these pieces of malware can then be associated to other incidents as well such that if you want to pick one that

has some cross associations with different threats or whatnot

how are we doing on time okay no pressure okay so this one uh does not really have a right west doesn't need a microphone what you don't need a microphone yeah so this one um just for an example this this doesn't really have a direct um linkage to another incident but it is the malware's type of malware observed in this incident was the same as another incident and we mentioned that in the description as you can see um so this incident is 2014 1015a and then in the description you'll see that the malware is the same family found in 2014.710a and also sort of going along with the diamondball in this instance um for this we can't confirm that those

incidents are um associated with the same thread actor because we only have the malware piece of the diamond model the capability piece rather that overlaps so um we don't have enough um overlapping infrastructure to create a threat yes two incidents are the activity of the same threat but you can't can you show them really quick how the two and the amount and the associations if you click to the malware

so if if wes clicks on either the md5 or the shot one there and then goes to the associations from this piece of malware to the activity that it's associated with it should be okay so you only have one there okay so ideally you could have um multiple here so you've got one piece of malware maybe it was used a few different times maybe characteristics of this malware where you use go to associations um so you can have multiple incidents here that this would be associated with or you could have it associated to a threat um which is a higher level uh in in diamond model terms that would be an activity group and and this activity would be an

activity thread so this is one incident you can have many threats under this you can have signatures that you've written on it to detect it and whatnot you can have other indicators that are associated with it as well okay so yeah you've got a couple signatures there as well

so the what you're seeing here is by the actual malware sample itself one of the things we're building out here is actually in the metadata whether we can identify common pe section header hashes common end patches that can identify what the variants are programmatically instead of just saying yes this is a poison ivy variant or yes this has a common mutex string or whatnot so you can pivot on each of those

okay yes is that a hand raise or is i just i don't think you really have a question okay okay any other questions all right all right thank you all for your time