Skill Sharpening at the CyberRange

alright ladies and gentlemen thank you very much for coming to the on adversary simulation this is what I do every day at the Regent University cyber range and I really want to explain that our moat our focus and our motivation is for our attendees to learn all the things I have a teenager and he told me that I had to have a mean so I got learn all the thing apparently if I don't have a meme I'm not I'm not in vogue so Who am I GIC I've written a fairly well-known books yes the second book is bigger because it's better I've already had somebody complained that I've triggered their OCD because it's 7 by 10 instead of 6 by 9 and it

won't fit in their backpack and I'm like well you know I'd like to tell you something there are three kinds of people on the internet and I got this quote from Keith Palmgren who teaches for the SANS Institute there are sheep there are wolves and there are shepherds and I really want to encourage you to be a shepherd and that's what I am you know 25 years on i.t been around I was also an emergency medical technician I found in information security it is an amazingly high percentage of people who've done fire and EMS roughly 11% of sans instructors have fire an EMS background including Stephen Northman himself I'm senior designer and I will also make

an official announcement we do have two more books in the blue team handbook series on the way [Applause] thank you kindly very much I appreciate that and if you have ideas oh by the way I have three giveaways so at about when we're done for about five minutes as I go through this talk on add sim what I'd like is for the first three people to stand up and in Reader's Digest mode share an adversary simulation success story you guys are going to get the prizes so it's it's going to be Jackrabbit time at five minutes till the end so our range is set up with four classrooms arranged in five computer pods and they can fight each other we

have a whole bunch of stuff and what I'm going to go through is how we actually do this the pass on is sure some good lessons for you guys how we actually do adversary simulation at the college and we also focus on one thing we really focus on after-action report I did people around the whiteboard and I make them talk about stuff and you'd be as surprised the amount of learning that you get through there's some lingo that we have to be compliant with first thing Gardner has now produced research within the last five to six months on bas breach and attack simulation and I'll talk about that in a few minutes but we want to think about the red team the

attacker side the red team is trying to penetrate trying to go beneath the radar and then we also have the blue team they are the defenders in an adversary simulation exercise in order to be really successful you have to know how to play both roles or have the right people playing both roles and they really have to have some good guide books because the best situations that I've experienced is when you play one role and then you flip-flop to the next role there's also some other key roles in adversary simulation you have the purple teaming concept becoming in vogue nowadays it's kind of like what threat hunting was to the in 2011 and then there's also white and green the white

role in an adversary simulation is what I do they are the arbitrators they play the role of what's called site personnel when you think about doing an adversary simulation environment and you're trying to do something so you're going to come with an ADD sim event at it besides event or something like that somebody has to be the expert to represent the organization that is being penetrated and that's the site personnel were land that's white depending on who you are you may also have a green role the green role at least in the commercial sector tends to be the person or person is percentage that that word that's plural of person they are the people who are

trying to make sure that you're doing a good job so they are auditing the white team so here if you've never seen a Gartner trough this is a trough of enlighten there so they have a phrase for it breach an attack simulation is very much at the innovation level it is far-left that actually has value to you folks out there who are interested in adversary simulation more vendors are going to catch the term your Google search some use searching is going to be better you're going to be able to differentiate folks there's now an anchor word where we can actually talk about what this phrase means okay so the Gartner definition is tools that allow

enterprises to continuously and essentially simulate the full attack cycle including insider threats a lot of a movement data exfiltration against enterprise infrastructure using software agents virtual machines and other means I'll practice that just for you so the whole idea is there are now products available that weren't on the market a few years ago I mean I think the oldest company in this space is Sage safe breach at something like three or four years compare and contrast that to Microsoft how long have they been in business I don't know thirty thirty years compare that to IBM how long have they been in business before I was born and I'm above 50 I want to say too far

above 50 so a search terms you'll be able to get that for the for the over-40 crowd I had a dan Aykroyd picture I just had to have in this slide adversary simulation regardless of whether you're doing an exercise at your organization where you have red versus blue with white involved has to answer critical questions if you're not answering the questions and you're not going about the process of building an adversary simulation environment or buying a product either way you're essentially wasting your time I think because you really want to know why you're doing this and the reason why you have to know why you're doing this is you're going to devote time energy and money and you really want to be

achieving I'll learn to behavior outcome in the people who are going to participate this is not just going and getting a walker of a von hoening system this is figuring out how we can put something on the network to either make a test or make somebody sharper or better or make yourself sharper or better because we're talking about an organized process so think about why are you even thinking about doing this either building scenarios or spending money on a product what is the actual problem you have to solve I have a list of a number of open source projects out there and I've spent some really good time with a number of the people who

create these open source projects I'm going to show you a few of them that actually do this for you and you can go pretty far down the road without spending anything but a few afternoons they be exciting afternoons because you're gonna learn new tools okay what does go how is this going to improve your internal security posture show of hands how many of you actually think you have a documented internal security posture I see one brave soul and I'm legally blind so I'm only counting him because the rest of you didn't weigh violently you know what are you actually testing and I have to tell you this point right here that I'm highlighting with my cool logitech thing people going

to respond to be tested you can very easily shred people by being a total bad rump and walking all over them and not giving them the chance to detect I hired seven people in my MSSP I made that mistake I want to be a Good Shepherd and I want you to make sure that you guys know that if you're going to do this and you're going to train your internal staff you really want to make sure you're going to build them up then they were going to be the next fleet of peppers and don't make the same mistake I did I actually have chapters and materials and then the next book that really talked about some of these

key points here so you know think about morale and the other thing I'll tell you is that a lot of adversary simulation particularly the commercial products assume compromise and I actually sat down with a guy who wanted to use our range for an exercise and and he used the term we always have a close-in attack posture and I said what's up close and attack boxer oh well we've already compromised the network I said oh wait a minute how do you know I mean I've been doing cyber defense and Incident Response for a lot of years and it is not necessarily trivial to attack a network so I don't think it's a fair assumption that you should assume

compromise there should be an avenue in because when you actually find the badness on the network to have a really good outcome trace it all the way back to figure out how they got in don't just look at cleaning up the environment it's really important to know how the actual compromise occurred in whatever scenario that you devised there are even more questions that you want to answer and you see some highlighted text and I'm going to ask you what is that phrase business that's a command and response business okay and that you're supporting the value okay how many of you wave your hand violently understand what the MBA term value chain is show of hands one

person two people if you really want to be business relevant understand what it is your corporation or your organization or your university depends on and figure out how that component can successfully be attacked or how that component can be adversely affected and trained to defend that component anybody want to guess why I'm telling you this is so important yes sir

because without the context of business impact you cannot prioritize or understand it's an excellent answer and I'll tell you there the 50% of that answer is if you are business relevant you get funding time on the books management cares and why is that because you are protecting what your organization depends on in order to conduct business doing an ad syn project in a corporate space just for the sake of doing it making people sharper is cool doing an ad sim that you train twelve people for in your sock so the next time really badness happens on that that's a technical firm badness happens on the network your sock and immediately go and say we checked all of the order

entry and the shipping systems and there's no adverse effect what can your sock report to management sir critical business systems are up and uncompromised by this attack that must be something else that is a win there is nothing worse because I have been there I have been I may use you I have been in this position what though four-letter words come out very flowery and they're discussing their intensity about the fact that their shipping system is down because that affects their what begins with B and ends with Onis so being business relevant and understanding your value chain is really important because you're in fact defending your company so anything that you do there is going to

really help sharpen your stuff and what you're working on you have to have a plan you want to be training and testing for KSA's ASA's are knowledge skills and abilities this is actually how the FBI interviews they interview based on knowledge skills and abilities all three dimensions at least when I applied there in order to figure out if you're a good candidate for being a special agent or an agent it's a great way of looking at it because it allows you to focus on what you're going to train on do that people have the knowledge that they need in order to successfully attack X do that people have the knowledge they need in order detect trace evidence of an attack

knowledge is one thing taking knowledge and exercising knowledge makes it into an a skill I have a good friend of mine my brother-in-law he's a diver and he said to me Don he said to me Don you can't buy skill and we're looking at all the stuff in the scuba store and I'm thinking man if I got that buoyancy compensator would really help me out you guys gone you can't buy skill I'm gonna tell you the same thing you can learn knowledge you can come here you can go take the guys who did the workshop at 10 o'clock that under the wire guys they've got a really cool online lab you can get all kinds of knowledge there but until

you take that knowledge and apply that a skill you know you're not gonna be able to really progress because knowledge is great skill is what's gonna hold you through to the long run your roots what you really really want you also need to understand when you develop a training program what it actually takes I've done a lot of this this is not easy when you come and you develop a full scope exercise so if there's an organization out there they're the ATV Association for talent and development they will tell you and I look this up recently just to make sure I had the numbers right based on their thousands of members depending on what you're

actually doing it takes between 23 and 143 hours of your labor to develop one hour of premium content professional delivery and that's they a nationally established metric so it's it's not as simple as hey let's go get a County box and see if we can attack the web server it takes time to put these things together it takes time in order to have a really functioning range whatever you actually choose to have a range you have to have a platform virtualization is wonderful I love it you have to have scenarios you've got AWS with VP sees and protected is a really good idea on variety of things you have to have modeled networks you have to think about the networks that

you own and again being back to being business relevant if you understand how porous your dmz is you know anybody does the idea dmz porosity makes sense to the group if you can actually stand up a training environment that is as porous as your production firewall assuming you can actually get the configuration they may not want to give it to you in that case you have to aid and assist them that's the kind of environment you want to have if you have a rock-solid non porous firewall you know it makes it much more difficult for an adversary to get in if you have an extremely porous firewall I really did some consulting for a Corgan ization that had three

thousand and eight rules in their firewall only forty-one percent of them had fired in the last year very difficult to figure that out after action analysis is probably going to be your primary reporting measurement and metric your simulation your scenario has an information flow it does it follows rules you have to set up your initial session in some are usable Manor script it out I'll show you some cool stools to make that are cool tools to make this stuff script you have to figure out how you're going to observe people be aware and don't be blind to the fact that when you do anything and you observe people's performance it will have a material effect on how you how

they perform people who know that they are being observed will behave better generally and more conscientiously and would be paying more attention it is a well-known fact and he pick up any book on Six Sigma and you will find the studies and the references for that so as someone building up adversary simulation and trying to use these tools in an educational learning environment you really want to be aware that people are going to be on their best behavior or best hose are trying really hard that may not be exactly what you get from a job performance perspective it's just something to be aware they have to have record-keeping that you meet your objectives what exactly

are we testing for did could you actually detect these things could you find XYZ and how are you actually going to measure that C T FD phenomenal tool doesn't take long I had an intern with hardly any idea experience I I pointed her in the right direction four hours later of actual sweat equity in her time I have a CTF captured with twenty questions and hints great tool event monolithic but a great tool confirm that your trainee solves the case you as the senior person need to understand for this particular scenario set what actually is the solution is that identifying a root cause is it identifying the artifacts and the run key is it restoring a machine to service

once a particular decision-making threshold has been reached what is your going to be your success criteria if you can't identify your success criteria you don't know how you get supporting objectives in order to get to the people to the success criteria and you also don't know one other thing you don't really know how to guide junior people to help them get there because remember the point I made earlier aren't my goal and the goal that I want you guys to take away from me as being shepherds is to help people grow and become better because we need to be better because they're kicking rum I mean they are think about root cause analysis how many of you have junior

people on your team and if you ask them to do a root cause analysis of why a local file inclusion or web vulnerability was permitted they could figure that out and how many of you know what local lfi is probably half of your at least I would expect but how many of them could go and figure that out from a root costs perspective this is IT in the game job skills so when you're thinking about building these things think about how you're gonna get them to the cross the finish line our network works like this we have 49 50 operating systems I can do all kinds of stuff and I have I've got tons of scenarios we have all

kinds of things modeled in here and when you think about building your own adversary simulation network think about what you actually need I tell you email delivery really helps and it is not the easiest thing to set up gotta have some kind of email delivery got to have workstation networks should have physical network we have industrial control multiple firewalls simulated internet think about all the stuff that actually takes to make the internet work if you want to simulate the internet and your environment you need root nameservers you need static routing you need forwarding name servers you need all that stuff you need some kind of email delivery you need if you want to do email directory and you want to simulate

the network you need MX records email gateways there's a serv resource record type for addy Kim option that maybe the mail server that you're gonna use isn't even going to work if you haven't even set up all that stuff so you could have a lot of setup to do to make email work AWS is the answer it's just the thought I could be wrong artificial domain build tools there are many of them I actually am rather fond of one of them I wonder if you can tell which one it is at Auto lab automated lab Boxter detection lab and if you really want to kick it up a notch and duel and win stuff there's a couple of really nice

ones out there gns3 has over a million or some huge number of millions of users and there's the cisco v IRL detection lab is the one that i use chris long created this I've actually gone back and forth with him over the last three months making little refinements and testing and working with this tool it's pretty darn cool detection lab is a completely scripted builded build process uses packer and vagrant stands up for you a domain controller with a reasonably hardened auditing rule set a windows event forwarding server with actual multiple levels of subscriptions a Windows 10 machine that is highly instrumented with sis Mon and Windows Event collection and Windows image forwarding in detail tracking and get

this a Linux box with Splunk so that you can actually collect all this data and if that isn't enough you get OS query and miters caldera and all it takes six and a half hours of watching the system run it is also fully extendable I've taken Chris's configuration I've added a third server and I've added eight more workstations so for me I can take this tool extend it add more operating systems to it I push a button Saturday morning Saturday afternoon I have a completely fresh environment well instrumented for the kinds of things that we would want it is really a great tool to look into for building an automated Windows domain and you can actually stick your own license

keys in the environment if you want to if you have the volume license keys and you want to make this permanent you can either license it after the fact or you can build it in it uses all native Microsoft techniques you know where you can actually do the boot floppy business and it does the unattended setup it's pretty good stuff so when you think about this how do you actually select tools and we have Gibbs rule number nine always take a knife with you and what that really means is make sure that you have the right kind of tools tools make a lot of difference there are at least there are probably more but these are

the ones that I've actually read about talk to tinkered with actually experimented with in my own life a PC simulator is a batch tool you've got the atomic Red Team tool which is really scriptable really nice very well built maps to the mitre framework i've got some slides for bp3 there's a ton of blue team tools and i want to make a point to you i've put one up in here this one on the whitelist noise generator one of the things you really want to be careful about if you're building up an adversary simulation environment is having other traffic if the only traffic that your security onion box has is your malicious traffic and baseline window stuff it really

helps out the blue team defender because he's got or she's got very little stuff to look at so it's really important for you to figure out how you can actually go get other stuff to make your range look better it's important log n D is available I'm really a fan of this I've actually used long gut log MD on a number of consulting engagements it is a well worth the price tag of free and if you want to go for the the kitten caboodle you spend 300 bucks there's also a great site and I cannot pronounce the guy's last name but his first name is Chris I had lunch with him at the San Ximena recently and he's put

together a whole package of tools called open sock at open socket IO and he's got all the various components listed out that you know he puts together and as a fly away lab environment so in thinking about what kind of tools you can use from an open source perspective just checking out the open sock IO website can give you a pretty rich set of tools to help you along the leg of what you would need the pen test IT folks have actually kept they keep a running list and a comparison of some of these tools what I really want you to take away from this list is not who's read not who's green but look at the criteria on the

left hand side the criteria on the left hand side are saying these are the various mitre attack framework techniques that these tools support that actually guides you in thinking about what does our team really need to understand what how well are we instrumented do we in fact have an ability to detect lateral movement dudes our team even understand what lateral movement is does our team understand what defense evasion is do we think are we can we afford carbon black retain ium do we have one of those premium edr tools if you are a small shop some of these things you can't afford if you're a fortune 500 company then you've got lethargy to work through to know what

tool you have and to get access to it but you still there's all kinds of tools you can be available exfiltration how would you actually detect worm bots that's a term I just made up for you guys warm bots that work for the group how can you detect if a worm bot is actually using Gmail with file attachments to send your private data out the door there's actually malicious code does that apt simulator this is a pretty darn cool tool I would start here it's dirt cheap it's batch it's designed to have you take pieces of part of it you can pull one piece out test and train on one piece rinse wash repeat your virtual

machine snapshot it use a component put it back together use the whole thing this guy will actually do local file collection see two connections see two connections via WMI I will do malware rat it'll actually download me me cats although in deference to Eric on reddit will not download me me dogs that's I'm a dad I have a dad joke I had a teenager me my god have a dad joke I'm persistent there's three or four different ways that it can establish persistence for you for the low low cost of free really neat little tool we actually start in our program with this and I gave you the web link for et3 in crypto IO and I I

cannot pronounce the gentleman's name I wish I could he's got a bunch of a bunch of YouTube videos to tell you how it work you need Kali it runs on Kali so you can drop your your your tax ID system and your and your victim side system out on your network as an as an ISO for free he gives you a number of sample traces a couple you know four or five different c2 mechanisms a few different hash collision mechanisms that look like real malicious software and a number of peak app based executions and his price tag to add more components to the mix is like three to five dollars us it's really inexpensive what he's built and

the guys go in for volume I swear it downloads agents you get the code you are under control you get a piece of Python code where's mark bag and I know he'll be happy it's Python code here these are the ones that are run for free out of the box by the way it took me four hours soup-to-nuts beginning to end to read the 40d page manual fire up the BMS did everything running and actually create these slides four hours is all I took to do this and then I had to go eat lunch or have a snack or something all I did it generate the client give it the profile stand it up and running run it

you're now listening on your River this is designed to look a bit Metasploit like so it's a metaphor we're familiar with on the client-side all you need is the code so that's a Python executable think about what you could do from a training perspective on a Linux box if all you have to do is run some code so let's think about this cron job shout it out cron job modify a file in an it dot b add something to a user's dot bash RC file add something to an hourly cron job what else can you do on a UNIX box if you've got the code audience-participation ssh root key add it to a system utility make your own LS

command to run this in the background I mean you know since you've got the code it's pretty pretty open not only that the code actually generates the signature on the wire so your security onion box I mentioned security onion earlier this is a default rule it's picked up right out of the box by default it's available always works you can pick it up and then you what you wanted you from there is trained people to click the button and go right into cabana at least in the onion platform and go look and see so what I just showed you is I know afternoons worth of work a couple of virtual machines on your network that

you can deploy some execution code under your control your intrusion detection system can pick it up your analysis system I mean anybody in the room is there anybody here who has not downloaded installed security onion anybody good so you guys have all used this right-click integration and I can actually see what's going on this becomes an analysis opportunity for your junior staff poof what does it look like if you let this guy run overnight I wouldn't want to know who did that some guy might have done one don't want to say anything but this could be a great activity you've dropped this off on a cally box somewhere on an end user segment on a switch or a Raspberry Pi

you go ahead and run this activity let it run on an IP address on the network you now they need to investigate DHCP investigate the switch get the MAC address investigate the CaMKII table on the switch assuming you're using Crisco products go find the Machine go find the jack you can actually make this a go hunt down the box exercise for hours and a Raspberry Pi what does it cost nowadays forty bucks and my right it's 40 bucks for raspberry oh you got to buy the little power thingy 48 atonic red team i fat down on the phone with these guys love tool this tool is specifically designed to map up to the miter attack

framework it has codes specifically the numbers against the miter attack techniques to get the attack framework to get eight to get art I asked them can I call it art and they're like you can call it art to get art to run you just invoke the command up there assign it to a variable they suggest that you use the attack technique so it makes your code flow really nicely and run it all of the things of how this tool works are all defined in this really really nice yeah Mel file so it's fairly well documented down here including the command if I have access to the command and I'm doing this what can I do

it's an extensible go ahead what kind of you if I have that easy path of execution right there and the blue stuff on the up the blue stuff in the upper right hand corner and this is actually what it's running down here I got to use the cool thing II right if that's what it takes what can I do I give you I can do anything I want right how about how about for the really sophisticated room go figure how to do some process hollowing that's a challenge I'll see you next year commercial product there's a boatload of commercial products in on the space lots of things are available in the commercial space you can spend tons of

money on this stuff attack IQ safe reach image in this list of stuff there's even some stuff for office 365 they actually have an attack simulator there are also things that you can use that are not necessarily designed to be adversary simulation but they can vary easily be adapted for adversary simulation a cobalt strike I talked with Raphael Mudge on the phone he actually called me and he gave me a number of ideas he actually has four or five recipes over the last couple of years posted on the site you can go use cobalt strike cobalt strike has like a 15 or 20 day trial you can use the sky folks kite this by a grim they're one of

our sponsors here you can come to a region cypher range you can also get the breaking point by Ixia which has hundreds and hundreds of malicious software protocols and I think it's well over 2300 I can't remember the exact number of application protocols there's a number of things that can be used in this space if you're going to invest in a breach and attack simulation tool think about what you actually want to get out of it how you want to be able to measure because they're designed to measure how well your tools work what are you actually going to test do you what do you need to test your email capability some of the systems are very

very sophisticated in what they will do Saima late actually has hundreds of attack oriented payload attachments that they put their model is to send it in to an email address on your domain to one email address that flows through all of your protection mechanisms and they have an opener and if the opener actually sends an email back with the right identifier they know which particular attack pattern got through and they can report on that it's pretty cool stuff how well do you instrument your tools I worked in an environment where that nice feature in Windows where it prompts up and says do you actually want to run this administrative capacity that's not a good idea but that's the kind of thing

you could test so again I asked you earlier about your security posture these tools are really designed to help you assess your security posture and how coarse is your perimeter they take specific approaches most of the commercial tools you know operate in isolation you take a similar environment standing up on your own segment you isolate it you test it you're using your golden image they deploy artificial agents you can also generate traffic on a virtual machine under your control that you dropped off the network so in case something does happen it's a pause button in the VMware interface you can shut that off immediately you can also use what's called black box multi vendor or multi vector where they deploy

a very very thin agent and it really functions just like a bot where you're actually sending command and control from the SAS service also the vast majority of these things are SAS based so in order to use them most of the contemporary bas tools breach and attack simulation tools you have to be connected to the Internet in some way 1500 distinct attacks has a huge active community for support they do not have any c2 capability for these attack IQ folks however they have stellar remediation guidance so if their system actually detects and finds out that we were capable of executing persistence on a Windows machine here are some things that you could do to protect from that

the cyma lead dies these are from there from a foreign country I think they're Israeli or something like that Israeli so they have a number of features I thought it would be kind of cool to give you an idea of what this looks like you pick the tool the technique push the button send it through you as the operator are in control whenever the tool or technique works it sends a positive act back to the cyma late site so you get a way of actually scoring it and getting a reporting mechanism my fav is site by the Grimm guys complex adversary simulation tool scythe actually not safe grim scythe whoever they created this tool to aid and assist in their

penetration testing practices because that is a very well understood technique for them roundest right and they created this really neat tool so they can actually take this tool and do all kinds of different things with it from a Red Team perspective and they use it in engagements and they said aha we can actually use this for more learning environments it supports the attack framework has mobile has guides it has a whole bunch of communication modules now I didn't screenshot the entire screen because when I get that it gets really small and you can't read it I just grabbed part of it so there's a lot more capability modules than the ten that I have here

but the real neat thing about this particular platform is on the next slide where you get to put this stuff together in a sequence in a set of orders and you can save that configuration out and you can rerun it so if you're going to build a tool and you want to roam different people on your team through different levels you make the investment one time with how to build your scenario and then you can actually rerun that scenario later on you as the operator because this tool isn't designed to be like an education training tool it's designed to be an automation and improvement tool you as the operator will have to keep track we ran that's right

we ran salic through the exercise Sally got a B and C right he works for cyborg Ren neat stuff you do need to be aware of range isms as a technical term remember that agents don't behave like people if you've been up against a real adversary an agent will not sleep an agent does not take time off and like and when World of Warcraft gets released in China they don't do that agents just run they generate traffic based on whatever schedule you said agents don't pivot necessarily agents do this one thing they don't figure out and explore and say that looks juicy and that looks juicy and I had this list and I know that's the phone server and if I go to

the fight they don't they don't and they will function like a person they're automated which is good but realize that you have just small limitation in using agents they also don't read your email there's a great story of a number of major corporations don't want to say anybody's name but when the attackers actually get in one of the first things they're doing lately is trying to get ahold and find out who the security team is attached to their credentials and started reading their Outlook mail now that's insidious and I've actually been in the room personally with people that that's happened to agents currently to my knowledge cannot read your mail which is probably a good thing right

the other thing about agencies you have to be careful is that if you don't generate activity with both network activity and some kind of a non system trace event you're only helping your staff learn half of the story so you could do things that have great is this real as the attack traffic rule does this really look like our traveling notebook user really has dry decks on the machine or e Modi or anything like that is that and you can do that and that's pretty good but if you can't do anything on system to find it you're only you're not really training like you fight you're the only training so far so some of these things only carry you so

far the other thing is that you do not want to over instrument your training environment if you don't have 46 88 detail tracking enabled in your global active directory in your company's Network you are not helping your blue team if you enable 46 88 detail tracking in your training environment because you're going to teach them to be dependent on something that they don't have in the real world okay that's also something else I've learned is you really have to be careful and we are really careful when we explain to people that the environments that we trained on in our range have degrees of instrumentation I think it's important to note so if you want to do

this think about the capability that you need let that simmer in your mind to figure out how you're actually going to exercise it figure out what your security posture is what your value chain is what you're actually going to do to build things in right order also you really want to think about this right over here what I'm trying to get across here in this area is ensure that whatever you're doing is repeatable and and the specialized knowledge that it takes for that attack doohickey that's a technical term make sure that that's actually written down because if it depends on you Fred or Amy and Fred and Amy aren't here because Fred Namie went on vacation and nobody else can run the

scenario for your next junior staff so make sure you do your britainís stuff down it's it's important so that you have a repeatable we'll program do a SWOT analysis okay then MBA term strength weakness opportunity and threat think about what you own think about what you don't known if you don't own DLP and you don't own any capability that can examine data that's leaving your network via email FTP or whatever how can you compensate for that do you have a flow auditing capability do you have a capability to look and see for your firewall logs if the packet size is actually reported if the packet size is actually recorded on the flow events in your firewall do you have a way of

actually getting the list of all of the records that went to Gmail from a particular user so you can actually add up all of those packet sizes and make the decision and find out that user emailed 47 gigabytes of mail over the next two days you got to think about all these things when you think about what you have and what you don't have and what you're trying to test and you really have to get creative and think about the data that you have available you will also you're also going to be capable if you take the mitre attack framework and you're seeing what you own at given stages where you can do gap analysis if you can do gap and I'm

really excited about this if you can do gap analysis against the mitre framework and you can clearly say to management that we cannot actually defend these business critical systems because we can't monitor for this kind of stuff you actually have a really good way of thinking about what you're going to budget for and what your limitations are so you can actually improve your posture and then your posture is business relevant because it reflects your value chain that's a win for you because you're talking in the language that your business leaders really want how do you measure success you have to think about your measurement tool you also have to think about your measurement tool that

does not punish the people that you're measuring I actually have a number of things in my book on this topic Carson Zimmerman did a great talk he's the author of the mitre world class sock book it's around 300 pages he did a really good talk at the sock summit in New Orleans you know be on the lookout for when that drops on the sans YouTube channel cuz it's well worth your 25 minutes he talks about toxic metrics and the talk I can't do it justice to what he talks about but it's really really worth it um I have a number of metrics that I've put down with definitions these are things that you want to try to

kind of measure for right what is the mean time to detect and then contain if you want to teach someone who can find an event validate an event and then get them to at least articulate on this event we can do this action to contain it and we think that that's going to work is that thank you is that five minutes ten minutes or eighty minutes because they couldn't even figure out how to research the right thing and the other thing that you want to think about here is how can you have a mean time to detect and contain that does not adversely affect a business process so I've got a story back in my days at Old Dominion is the

ISS Oh public institution I can actually name them we had a machine that was running some kind of ColdFusion app and that app got hammered and it had been online for a year and a half and why it lived for a year and a half is still amazing to us but one day it got hammered and it was hammering the rest of the network and when I say hammering I mean it was saturating a one gig link for the best of its ability so much so it was affecting land traffic okay that's what I was doing when I say hammering the network we shut it off immediately because nobody could find it nobody knew what it was and we try to

talk to all business units we come to find out this machine is running a web app that was created by a consultant many moons ago and that web app is specifically for connecting interns with internships in the summer this event happened in April did I or did I not break a business process for a university when I shut the box off in April how many think yes how many think no for those of you listening at home we had no knows we had lots of yeses that's right the containment action in this case is to determine what is the inbound traffic pattern that that web application uses permit only that at the actual switchport level because that's the

closest to the inbox and block all other outbound traffic until you can figure it out for the next 48 hours while you try to deal with the case that is a much better containment action and the kind of thinking that you want to grow is people figuring that out timeline reconstruction how do you actually measure for timeline and how do you teach timeline analysis what are the metrics that you want to be measuring for the better off you can encourage people to understand metrics and do that the better off you are waagh md is a great tool for this kind of stuff it really helps out in generating analysis sets and helping you do time line and getting information

together it can also collect a lot of other data in addition to the windows login events when you run this tool not only can you run a baseline today and then compare an output set to a baseline tomorrow you can also very quickly see the reports that are zero-length produce no data so that's not an issue the reports that actually have some information has some stuff for me to investigate so using this tool as a simple automation even at the free level can make your people more effective it's well worth the download looked at if you upgrade to the pro version you actually get some really nice add-on capabilities where it really helps to give you an idea of

what's occurring in the process tree so that's something that if you if you try to teach people how to learn this habit this tool happens to help you out and learn that after-action reports you get five people together excuse me six people I like to do team of two team of two of working they're working on a case together you compare notes how do you actually want to conduct after-action do you want to have people print stuff out I have found getting people to talk on the whiteboard works really effectively I'm a real big fan of an incident response report how many of you actually read the 30 some of my pages in my book where I talk about our reports

good stuff I hope so we actually have people write I our reports and I eat my own dog food in this I use my own format I've used this format for 12 years professionally we actually want your executive summary to score a 95 on grammerly that was that pause right because you're all thinking oh my god he wants me to write good grammar yes why is that the image and the quality of writing by your team will end at the end of the executive summary okay I have done IR reports I can't even tell you how many in the professional world I've been searched warranted I've been subpoenaed all that kind of stuff at the

university I was involved an employee investigation for half of my career at fortune 500 people have been hired and fired and sued based on my work and I'm not saying that to be an arrogant pompous jerk I'm telling you that I learned what to do and I learned what makes a difference what makes a difference is when any consumer of your data reads a well-written grammatically correct executive summary they trust you in the beginning of your establishing or relationship with HR and legal counsel they will actually pay attention to your incident timeline if you sit and talk with them because they want to know that you have the chops that if they take a document they do a firing action or they

take that document to court that they have confidence that they are at at worst case scenario not going to be embarrassed but that they will win they want to know that we also make sure that you understand how to do root cause analysis articulate a position even if you make it up about business impact we want to know that you can understand root cause analysis and most importantly well secondly we also want to make sure that you've got the whole thing down there and writing a cap a cap is a corrective action program you want to be writing caps why caps a real world if you're in a publicly traded company and you find a breach and

you find an issue internal audit is going to want to see a cap i'mjust time do we have left we've got ten minutes good I got five more minutes of material and then we're gonna do the stand-up tell me a story to get a book or a an adapter a thingamajig that's best story if I've got some time you don't need to spend a lot of money I used to demo all kinds of stuff by using some really simple techniques you can go to fire hall you can get all kinds of great lists you can go to malware domains pull down some lists just you know get this stuff test it out on your system try to run some of these

things from some of the open source tools build your own scripts drop it off put it in the log on shell of a machine log Anila machine see what it does you can go a long way in understanding what you can and cannot test for what your tools do and do not detect by just using some of the really nice open-source tools I've pointed you guys to earlier also if you do that effort and you say you know we spent a couple of weeks we did some of these things we found our stuff didn't detect this and it detected that and then you want to actually go and buy a BAS tool that's automated you'll be in a much more knowledgeable

position to know if that tool is actually going to help you assess your posture okay think about what you're testing this hopefully you guys can read this this is two different dating oriented websites dating is blocked at my house I hit teenage boys we would much prefer that they did it the old-fashioned way that they actually met the girl in person crazy parents what am I going to say so I tested this note I have two different behaviors for one dating site and another dating site the Palo Alto blocked the one on the lower right the lower it told me it was backwards but the Palo Alto did not block the one on the upper left you look

and you see the behavior here this is all the block stuff you see what's going on different things are blocked some are detected as web browsing happens if you know this is what cool thing it looks like this is all the cool data you can get all this information for that site so Palo Alto pretty much understands it what happens if you do it by another mechanism you use W get W get does not look like a browser you can actually see that my OpenDNS stopped me not my PA box do not assume that your premium amazing tool is going to behave the same way in every instance because here's evidence that says they don't and I happen to be part

of the cult of Palo Alto I actually own one I'm a real fan of a tool but it didn't have the same way I thought this was really relevant for you guys to say even premium tools like this do you behave differently based on how they're triggered many many many ideas you can use for focusing this stuff we're at five minutes right alright so I told you we're gonna you know get some volunteers to stand up for a good red-blue story volunteers first one up on your feet for a alpha if you guys don't want this I want this nobody's volunteering this is a cool tip alright no story stand up and tell us a story good pass up

excellent discovery and you improve your security posture and what happened to you because you did that you became more business relevant and it related to your value chain I'm telling you this makes the difference who's next next volunteer oh the scenario but he was describing is they did a pen test against a publicly facing website and all the users had to factor authentication but they discovered that the admin side did not have two-factor authentication and what was worse they were allowed to get him with seven character passwords so that's the incredibly weak two more volunteers two bar books I know the guy who wrote these they're pretty good

great story I love it well last book volunteer on your feet you have to wave hi because you know I don't suppose it are you guys ballin tearing over there no trying trying who another person seriously one of the for class book alright doing it in general

awesome

[Applause] great so for those of you listening at home the story was that they were trying to attack a website on a firewall product and what they found is that all the normal stuff didn't work it was until they hit the spacebar so that's actually really good lateral thinking ladies and gentlemen I really want to thank you for your focus time and attention and I want to encourage you remember there are three types of people on the Internet sheep wolves and Shepherds via Shepherd