Testing Defensive Controls w/ atomic-operator - Josh Rickard

[Applause] thank you thank you all uh first of all they give a big huge round of applause for all the sponsors man [Applause] all right so today we're going to talk about testing defensive controls with uh Atomic operator if you're not familiar I'll explain it a little bit but Atomic operator is an open source tool that I developed we'll go through but I wanted to actually go through some of the process of how you actually test some of your defensive controls so we'll walk through that uh again my name is Josh Rickard um this is actually me in the The Mask uh but I have a background and a blue team specifically digital forensics and incident response uh I like to automate anything and most uh things I love to release open source tools uh hints I call myself an open sourcerer you know I got the drug okay cool um I didn't say I was fun at all the uh my Twitter is uh at Ms administrator it's not Mrs it's uh it's Microsoft administrator back when I uh did admin stuff so github.com Ms administrator or my blog uh it's let'sautomate.it and I just talk about automation stuff uh that's enough remember so security testing what is it right it is the process of verifying all well some of your defensive controls uh and making sure that they either they're in place and they're working as you you expected right it's pretty uh pretty important for for more organizations but there's this entire process that you may have had around how you actually begin to test your controls uh the first is going to be like determined uh you need to actually determine all the goals like what are your strategy around your testing and what you're going to try to um what are you trying to defend against and then you need to actually create a plan and you will actually go in and execute that plan uh examine your results and you know adjust as needed as well as closure or how you can actually improve your defenses long long term we'll go through each one of these faces and and sell bullets and all that but I wanted to give you a high level of where we're at does that make sense yeah good okay interaction is awesome so just yell and I don't care and sorry for the customers uh Divine expected outcomes right so we need to actually Define what we're trying to test and what we're trying to accomplish with with our goal uh with this test so we need to test against certain things uh are we trying to actually test against a specific tool that's out there maybe and I use this example a lot but memicats or maybe some other tool that's out there a specific technique that's in the miter attack framework or your own framework or your own game um your own standards or a very specific threat actor depending on how mature you are in an organization that may all dependable I recommend first just trying to test the basics right if you're if you're just getting started more mature organizations they may be looking at like Charming kitten or apt-34 or all these other different uh types of threat actors they may be trying to have to defend against them because they're in that vertical or in that industry there are different types like I said uh different types of testing that we're going to have to do and it depends on your maturity and where you're wanting to focus uh the General Security we're actually going to just test basic stuff like can I escalate can I create a domain admin account these are pretty basic um attack vectors that we can kind of test our defensive controls or maybe we actually want to even test products that are in our environment there's the opposite side where you get into a more advanced uh perspective where is the adversary stimulation and emulation those are two different things and those are very specific to a variety or a bucket of threat actors uh 10 malcolmitter anybody here knew what it is uh pretty cool pretty cool guy I think he said here in case City now but um he actually had this blog post on medium I had the link at the bottom but he says that emulation implies an exactness to the copy whereas simulation only implies similarity with some freedom to be different that's important that's really good to know because we need to are we trying to emulate a threat actor I mean we know there are two TPS or we tried you know get something along those lines you know and some variety there for our organization so we need to understand that those are different things as well as your General Security controls so what are you wanting to test right you need to ask yourself are you trying to test your endpoint configuration maybe hardening rules that you have on your operating systems maybe you use Group Policy maybe you don't but you try to actually you know prepare your defenses maybe you want to um remove admin access you know what are the improvements that there's a lot of different ways you can approach that but maybe you want to test the product maybe you want to test your EDR maybe you want to test uh some other maybe DLP system or UAB whatever but depending on that we actually can then our tests and our in our stories and everything we'll get into that but uh or you're testing detection rules maybe you already have a detection engineering or at least some scope um where you have rules you have you know logs in a central place and you can actually run and collect data so you may be at that level it just depends right and I just wanted to point all those uh considerations out so the strategy this is really critical for starting out as you mature you'll you'll go in and you'll be able to create these kind of uh tests pretty easily but at the beginning you're probably going to use some sort of third party uh like Atomic routine or if anyone's heard of miter their adversary emulation project again emulation versus simulation but you can grab tests that they have and Implement them in your environment and test against those common and that that's a great place to start because they're already defined for you and they're pretty well known in fact if you're testing like EDR most of them are probably going to block because it's widely known uh if I don't then that's a call to your sales contact um they but they should be protected against this uh but then once you're actually mature and you want to build out you know your threat um defense testing you actually you can write your own tests maybe you get it from a threat and talent report from Talos or somewhere else then you want to actually take that technique and try it in your environment and then see the results and mature your organization as well as your defenses along the way again Atomic red team is an awesome start I shouldn't I knew I shouldn't have used the dark background but uh this is uh a test it's all in yaml they haven't been marked down this is like the markdown rendering uh on their on the GitHub repo but this is the code or the um the script that you would run to actually um create a rogue uh domain controller and an active directory environment the reason they do that just to kind of explain that briefly is you can then change and convince other machines connecting to your domain control this fake domain controller that I'm God I'm domain admin and then you can do a lot of dangerous stuff so this is a pretty easy attack to kind of detect and I use this as an example as we go through so the first thing that I always do is create kind of a user story like what are we trying to to test it's it's good for me I have an engineering software development kind of background but it is really good to have kind of this outline at least give you guide rails of what you're trying to uh test so as a security analyst a detection engineer I want to detect to prevent the use of mimikens to again create a rogue act directory domain controller within my environment so there's not specifics there um we'll get into that but this gives you a high level of where your scope is and you just approach this you just think man you really need to protect against malicious macros like how you decide as a user story and then later you can go and figure out all the details but it gives you a starting point so create that narrative uh you can do the user story format or I personally like uh just give it when then has anyone ever seen that uh format okay given one then is pretty common for a QA or other software development and testing but it allows you to kind of codify some things if you want to later but it's you know given I'm a threat actor and I'm on Windows Server maybe there's other ions in there as well but when I run this command borrowed from a top red team then I'm able to create a rode domain controller and the security team gets alerted that's what you want to happen right or maybe you want it to be prevent and attack right so you may add a statement in there but it gives you a way to actually just logically separate things out and understand a little bit further what your requirements are the next piece is we need to create a plan once we have our strategy we know what we're trying to accomplish what we're trying to attack or defend against we need to create a plan around so to do that we need to find some of the requirements like what is in and out of scope uh is uh we'll go into this a little bit but is it a Windows machines and then it's is Linux like out of there is Mac under there whatever that is you just need to find that at least in your hand it doesn't have to be totally written down but it's a good idea uh deliverables like what is the expected outcomes improvements maybe you just want to give a report that there's a lot of different deliverables that that could come out of it and then you need to start figuring out what those environmental variables are and I don't we'll walk into that so internet scope right is certain products like we don't want to we want to test our EDR or maybe our configuration we don't really care about we just want to see if uh whatever our EDR is causing alerts or maybe a different tool maybe a combination out there but you just need to understand what those are that goes into your you know your environments and all that other stuff certain operating systems again and windows and scope but Linux is out or and also the execution context meaning is this a remote execution or something dropped on the box and it's all local or is it privileged escalation so on and so forth I went to the talk earlier it was about safe breach uh I hadn't seen that before it looks pretty cool so uh shout out to them I don't I'm not paid by them but yeah if you want to give me a free license I'll take it uh Define deliverables right when you do what are those deliverables we're trying to accomplish simply you know maybe we just want to improve our defenses that's great uh if you're more mature again if you have a detection engineering detention rules maybe you want to improve those maybe you already have a rule for this type of attack and you want to improve upon it because there's false positive it's negatives in there yeah maybe you want to improve again that configuration hardening or you just want to report back out to you know your um your Boston man if we did this we would reduce our attack service by X number uh whatever that is you just need to Define that so you know what success looks like because if you don't I find myself especially with ADHD I'll just go on rabbit holes and I'll just keep going so I need to find like what is that stopping point you also have to understand those environmental variables I was talking about where you have the operating system uh maybe even software if you maybe have a gold image in your environment use that uh standard configuration that most people would receive how do you actually verify your success right um do you if you have login great do you have visibility did you even see that activity even if you don't have another are you able to identify that activity it's not that's an agreement that you need to make maybe it's a product again if it's an EDR if you don't secure DVR didn't light up then there's a there's a problem you need to uh understand those those areas now we execute our test so once we've created our plan we know the environments uh most of the time we'll have to set those up depending if you don't have a SIM set will help elastic is free Splunk has a limited license it's good right for basics um once you approve it out scale it up go to your boss I'm like hey check this out we can detect all this stuff there's tons of resources I have a project out there it's called elk TLS Docker it's elk TLS Docker and it is an entire Oaks there uh with security enabled uh and SSO and it's all documents you just hit a doctor to pose up and it will set up pretty much everything for it so there's projects out there and I think elastic just came out with their own so um they're out there so now we need to execute our test this is where we get to do the atomic operator Atomic operator is a python package that I built used to execute Atomic red team test soon it will support that adversary emulation as well it's just not completed and you can actually run these tests across multiple different operating systems both locally and remotely I'll explain that so we have Linux we support Windows Linux and Mac OS again local or remote so from if I'm on a Mac and I want to remote into a Windows I can five mono windows and on the remote into Linux I can and I can execute those those tests you can also use it just via the command line and I'll show you that as well as in your own automation or own scripts your own code um certain importable like python environments you can select your inputs so most of these tests have variables like what is the path for this exe that you want to run like where is the path of mimikens or what user account you wanted to use and things like that so you can actually select those both interactively uh with the the tool and then you can actually take it even further and not run anything and use a configuration file that is all documented and stuff that will actually automate uh that entire process about I've done up to 100 machines with it so you can do a lot if you wanted to this is how you install it it's a pip or pip3 uh most Mac uh we'll have it you know already installed uh you just do pip three install Atomic operator and if you already have it you I added the dash dash upgrade upgrade to the latest but if you were once it installs it installs this command line uh tool called Atomic operator and then you can just run you know you specify the name command the techniques one or more you can actually specify individual tests if you want and then if you wanted to connect to it remotely you would pass in like the host username and password there's other like SSH keys and all these other parameters I'll show you later in a minute actually right here this is how you would use it in a uh your own script so all the defaults are listed there it's kind of hard to see sorry but all this is documented and it's on the website and I'll show you that here but you can run techniques you can select test run pre-rex thing up commands there's a whole bunch of other stuff that you can do but the basics are just Monica and you provide that host and it'll run overloading here's the repository it's under um github.com sorry atomic or Atomic operator.com I'm surprised that was not taken so I took it and then I actually did a canary blog that kind of walked through that entire process as well so once you've executed your your test now you need to examine those results right you need to make sure that that they are working or they the expected outcomes occur and so you need to ask yourself you know did your test execute uh did you prevent it um did you get an alert all these type of questions are pretty self-explanatory but you still need to ask yourself and look into them uh you know it is work um security is not going to be solved in a day so there is some uh manual work that we have to do but you have to actually you know understand what those results and if you don't see anything and nothing happened that's not good you need to go if you don't have a SIM please leave this audience right now joking you need to record what what happened no matter what you need to add whatever that test is maybe you ran it and it blocked everything and you're good still log it because six months down the road you're gonna be like damn did I test then and you're gonna forget you also need to record your results again both positive and negative like be like man I just I just got domain admin in like five seconds and with all our security tools installed you need to fix that so you need to record that and I recommend doing this all in like a central location like don't keep it in your notebook or in your email or wherever put it in a central place and then anything that you do along their way because it's going to take some research you're going to take some Googling it's going to take you know papers reading whatever but you actually you know need to record all those future considerations improvements uh any notes maybe even references um so you can go back or someone else on your team can look at it in the future so on and so we also at this you know once we're satisfied we go into the closure phase of our testing and it's really all about verifying you know we're going to verify that our requirements are met we're going to make any suggestions of our findings at the end as well as provide a general summary and that General summary could just be for you or it could be for people on your team or your entire award depends on you know your situation again when we look at our statement that has a security analyst detection engineer I want to detect to prevent the use of nimikats to create a rogue active directory domain controller within my environment did anyone know that maybe cats could do that anyone know what memicats is okay cool yeah it's pretty cool you can do a lot of stuff with it everyone's like ah just don't passwords it's like you can do a lot more than that there's a lot of stuff in there so we have to make sure that we verify our requirements again questions to ask yourself what are your you know where your security control being able to you know prevent or protect that do you have that visibility you didn't see it and visibility is crucial when it comes to uh defensive security and uh activity identified are there any other variants that we could apply to apply to this maybe you have some reference links or research so on and so forth just just record on them you also need to know what improvements right do we need to make any changes that would help just because you found an issue doesn't mean anything is going to be done about it so you need to also provide like how we're going to fix this if it's a hardening configuration thing then yes if it's a detection rule maybe you need to enhance that detection room and you need another whatever it is and then if you're completely lacking visibility then you need to improve it so just just think about those things when you're when you're closing this out and suggest uh how to fix it you also provide this kind of summary report again this could be for you or for really uh ciso or your ISO or whatever but you need to document your findings I recommend like jira but you could use any sort of internal Wiki or um any sort of don't use teams or shoulder point please but um something like Jerry is probably the best uh you document your test of what you ran the results positive or negative and any of those improvements that you can make across maybe that report goes out iot be like hey guys we need you to improve you know these hardening controls that we have around memory does or whatever you just need to provide those and you can show this is what I ran this is my test as a result we need to fix this so overall this is t