SOC them in the face - Eric Carroll and Andrew Johnston

doing all right all right well we appreciate you guys coming our talk this morning thank you our presentation is suck them in the face building and attacking Sox my name is Eric Carroll and this is Andrew Johnson alright so a little bit about us as I mentioned my name is Eric Carroll currently a strategic consultant at mandiant to fireEye company the formal threat hunter Sauk analyst incident responder so have a wide range of skill sets doing various positions within this industry some hobbies I enjoy and get into I'm a DJ on the side so definitely interested in music and of all genres sports enthusiast car guy preferably German vehicles and then a world traveler and I'm a Pittsburgh native

born and raised and apparently I guess I didn't look at the updated slide deck but being a DJ I guess I tend to play Justin Bieber music I thought the conference needed to know so my name is Andrew Johnston I'm a proactive consultant at mandiant so I spend most of my time doing red team's web application assessments social engineering things like that on the side I also run a research lab at Fordham University where we're using artificial intelligence to fight terrorism online in terms of my hobbies I'm really interested in hacking ATMs picking locks and shooting guns pretty much any hobby that would count as probable cause I'm into this is my first time in Pittsburgh but

it's my third besides presentation and I'm very excited to be here just to note andrew is traveling from New York City to come to besides Pittsburgh today

alright to help us understand why you guys all chose to attend our talk not saying but I think the goal track today is a bit overloaded and and better but just just to understand the audience that we have in front of us today how many of you currently work in a sock or previously previously worked in a sock please raise your hand all right so quite a lot how many of you currently hold a role in the Red Team side of things or previously held a role in the red team okay how about the blue team okay how many of you currently are not holding a role in the cybersecurity industry but do this as a hobby on the

side or are a current student studying a cyber security degree alright some students in the house thanks for joining all right to outline why Andrew and I are here so we have experience with socks I've worked in socks previously and Andrews perspective is more from the the attack side of things from a sock standpoint socks are just another team in the organization except they tend to have better computers than than everyone else in the company and the key goal of this this talk today is we're going to get into some technical contents but we also want to focus you know on non-technical as well doing a pretty even split of both but we really want to

engage really the the human vulnerabilities that exist within within a sock or a security program overall as as us and all of you end-users you know are the most vulnerable so we'll discuss some of our experiences from the front lines being consultants you know what we see out in the field so we're gonna step into three different scenarios with you guys today so the first scenario we're gonna kick off but we'll get to our disclaimer here first before we begin we have to give the the usual legal disclaimer the stuff we're talking about today represents the sum total of our experiences we're not talking about any one specific client but rather overarching or unique problems that

we've seen that we think deserve more attention so yet so the when we reference a client or say the organization were really referring to this case study and not a specific organization we've worked for alright so let's get started first scenario we've titled gods in the sock so a little bit about gods in the sock so this specific organization followed a typical tier 1 to 3 model tier 1 being more of the the isinglass you know the monitoring and detection team members tier 2 being the escalation point for those tier 1 resources and then tier 2 taking taking that up to the tier 3 level which is typically your your forensic analysts whether that's an internal capability

that the organization has or they're leveraging those services externally so for this particular organization they had a combination of both so they had some internal Rancic's capabilities as well as a managed service for forensic support in addition there is just to understand the organization they had more senior folks so folks that have been with the organization for for more than five years that have been in this industry for quite some time so have a wide range of skill sets across the spectrum from both a Blue Team and Red Team side of things and in addition they had some some junior team members that recently came out of college graduated or went through they their internal development program to

try to build up people internally versus you know hire from the outside so there was a combination of you know senior and then junior level folks within this security program so the real breakdown with this situation is you you have full teams within the sock you know your your your monitoring detection team and then they had Intel and then red team operations that were focused on performing red team exercises quite frequently up into a weekly or a biweekly basis so there was a lot going on from trying to process you know alerts triage those in addition to keeping up with the red team's exercises that they were performing so the sock overall was was pretty

overwhelmed wearing you know multiple hats to support you know the current initiatives that were going on to keep the network secure some of the outcomes for this specific scenario are as follows so the suspicious alerts weren't being acted on so as I mentioned there was there was a lot going on between the sake analysts trying to prioritize and keep up you know with alerts that were being generated then the security information and event manager that they had in addition to collaboration communication breakdown between senior and and junior member folks so that was a key point here that the senior folks you know knew the lay of the land they they knew you know the day-to-day

operations that necessarily wasn't being passed down or coached to the junior level folks there so definitely a concern from an organizational standpoint and then just from a project perspective things weren't being prioritized the current staff sock staff members you know weren't aware of you know what to work on first it was you know purely just kind of pick and choose if you will what they wanted to do that day you know this is a problem that we see a lot a lot of the times we'll see socks where especially the junior members but sometimes a large portion sock really has never been in serious security incident or compromised so at the end of the day they're relying on what they see

in the tools and if an attacker is knowledgeable you know maybe those alerts don't look as suspicious as they should you know maybe you file names kind of match up with what they would expect so if there's not a lot of communication going on you end up with Tier one your your people on the ground who might see an alert and it could be indicative of something really bad but it looks just benign enough that it kind of gets passed on so you need that collaboration between the two or stuff gets ignored and really bad things can come all right moving into scenario number two oh no my crown jewels so so this particular scenario had no support

so no buy-in from the executives and in understanding the the security program current initiatives so there's really no no insight into what was going on it's just you know hey we have a security program so we're good right so that was one thing we observe with this particular organization in addition to cybersecurity you know wasn't not everyone was aware of cybersecurity what was going on you know even outside of the executives but in addition the sensitive data or critical assets you know this organization had when were never really listed out or or prioritized then and what we need to secure most so that's an important part that this organization failed to do never define you know what the sensitive data was or

their you know highest revenue assets that they have in addition here they were limited on resourcing so it's something we see quite a bit across organizations is very minimal minimal staff people are often tasked to support and wear multiple hats so that's that's nothing an organization was the same way and they had two environments what barman was the corporate environment that had security controls in place so they had you know AV anti-malware solutions ids/ips had sim so they had a pretty good list of tools but the other environment that was international focused was more dedicated to the the sales folks and it was like the wild wild west there was nothing in place no security

controls at all so no a/v nothing it was you know just kind of fend off yourself if something happens over there it happens but our top priority right now is just focusing on the corporate environment so that was that was a big concern with this organization you know having two environments like that it's you know extremely important you know for you to balance and have you know controls in place on on both sides so some of the outcomes for this particular organization are as follows so as I mentioned there was very poor visibility into the crown jewels one of the first things when you're developing a security program or looking to take you know your

security program to the next step you need to make sure all your sensitive data is defined in addition to having those controls in place on those critical assets once that is defined in addition here there was minimal network segmentation or isolation capabilities so as I mentioned the corporate network you know had some controls in place they were doing some segmentation by VLANs but the the sales network was was nothing nothing in place there so Andrew if you could help gauge the audience with you know what are some current concerns you sing from the Red Team side of things with with having a network that's unsecured like that sure so a lot of organizations obscene they do have

one part of the network that isn't as secure and it's kind of accepted that it is that way for operational reasons or or some other non non technical reason but it it's a really bad situation to get in because when you have one part of your network that just isn't held up to the the security standards you want attackers are gonna find that and they might even find it first and attackers you you know can live in there they can escalate their apologies and they can exploit the fact that visibility is poor work just that you know patches aren't applied as frequently and then they can use the privileges they gain there to move laterally into the more secured

environment now of course nowadays you need to have a tiered defense and you know giving extra security to your crown jewels your most sensitive data and processes that's a great idea but it should never be a question of bad security versus good security it should be a question of good security versus better security so it's a it's a real common trap and it's a very dangerous one to fall into all right so the third bullet here the outcome of that as you guys may have expected was an external service determined that this organization was indeed compromised via a vulnerable web server so in addition this organization had skill skill set gap issues so minimal staff in place

therefore you know people wearing multiple hats to try to support you know everything that was tasks at hand but they just didn't have the skills in place to cover to cover everything so some issues around you know hiring those those critical areas and then focus on priority prioritizing initiatives that's something this organization never did you know it was you know hey I'm gonna come to work and and figure out you know what to do and a lot of that was the individual kind of choosing that versus you know leadership you know handing that down so having good management in place is very important so every once in a while we're asked to do a red team

where the goal is specifically to kind of mess talk and how they operate so to attack not only the members of the sock but also their tooling and show that successful exploitation could actually lead to a scenario where the attackers are able to completely manipulate and control what the sock can see and actually how its able to respond to an incident and so we will explore it a bit in depth so in this particular scenario we had there was a very mature sock most of the members had actually been with the organization for a number of years and this wasn't even their their first position so they've they've been around they've seen a number of things

and they actually had some amazing tooling and some great documentation for their newcomers that would explain not only how they detect things but how they go through the processes and and how they use what they have in order to investigate and remediate incidents the problem they had was kind of external to them and that they were tested way too regularly they were undergoing a full scope red team so a four to six week engagement every single quarter so about 50% of their time they were actually being actively tested and attacked and the problem was in addition to these regular assessments they would have to show at the end of the assessment that they could successfully detect every

single technique that the attackers used and this came all the way down to things like domain names or file names so they had to show that they could cover a hundred percent of what happened and you know this remediation this this testing and patching process it was seen as kind of it was problems with the sock and upper management whereas it was using red teams as a way to kind of evaluate the performance of the sock and anytime a red teamer was successful in any way upper management was doing this as a failure of the sock itself so the outcome of that was was very negative it didn't create a great environment so red team members ended up being viewed as

this guy and when it came time to actually talk with these sock members about what we had done and how we moved through their environment they didn't see it as a learning experience right because they knew what was coming afterwards they saw the red team as combatants as people doing performance reviews of how well they were doing and a lot of the more senior members in the sock were really frustrated because they knew that you know going into all of their systems and you know putting an alerts for the domains that a red team had purchased for an engagement wasn't going to create better security it was just a complete waste of time and they

ended up spending more of their time doing things like that and implementing these checks in order to appease upper management than they were doing things like building out their great documentation or or doing in crea you know additional training alright so we covered three scenarios with you guys and now the next few slides we're gonna cover you know how do we fix some of these issues that we've mentioned in in the three scenarios that we've covered so on this slide some signs of an unlock so many of you mentioned that you currently work in a saw or have previously worked in a sock so these are some of the things in those scenarios that that we have we've observed

corporate politics you know heat huge egos existing so getting into you know the senior versus you know junior team members skill skill set gaps so I think that exists across the board in all three of these scenarios hiring difficulties so based on the locations and and where these security programs were they they had hiring difficulties so not able to get you know the key skills that were required to have a successful security program lack of end-user security awareness so there was no education to end-users us being humans you know being the most vulnerable you know that's extremely important to make sure you are educating your end-users quite frequently and then a communication collaboration break down so a sock is

another team of the organization there's there several teams that comprise a sock so you have your your monitoring detection your eyes on glass your Intel forensics teams data protection teams so it's important that you guys are communicating and collaborating effectively across all those teams whether you're junior member or a senior so our top five recommendations keeping in mind you know we're limited on the slides here so you know we definitely could have outlined more than five recommendations but we at least wanted to to cover kind of our top five out of the three scenarios so politics UG huge egos exist in nearly every organization that I've been a part of or or assess so do your best to work around those it's

it's nothing new every organization experiences that to an extent but ensuring that that that's not a roadblock or a barrier for for you or your security program making sure to do your best to work around those and focus on the end goal which is to protect and secure your your organization next shadowing is the most important tool take advantage of that so I know when I started my career and many of you in this audience I've worked with you know that that's one thing that I always go back to for the success that I've had in this industry is you know leveraging the the senior folks and really learning from them so trying to absorb as much knowledge as I can about

those folks take advantage of that so what you know whether you're studying for a certification or taking classes for a degree you know once you get out into the corporate world and get your first security position you know learn as much as you can shadow shoulder surf do all that you can to to take advantage of that another one of our big takeaways is that red teaming is a great thing and I'm not just saying that because I do it all the time it's it's a great tool and is it's a learning experience but red teaming should be seen as a way to compare how the organization's current security initiatives line up with reality of what

attackers are doing securing a large organization as many of you know is incredibly difficult if not outright impossible to do perfectly and a red teaming is just a way to compare that it's not a reflection on how anyone's doing or whether or not people are doing their jobs and our fourth recommendation is no matter what team you're a part of whether you're in the Sauk focused on IT the infrastructure side or even from the business side of things know your role and responsibilities so I can't emphasize on that enough that you know it's important that you know your swimlanes and stay in those swimlanes and know you know when to reach out for help or know when to hand things off

from an instant responsive perspective that's that's extremely important that when you're put in a situation like that where response times are critical you need to make sure that you know your role and responsibilities and you know your co-workers know the same and our fifth and final recommendation which in my opinion is is the most important is educating your end-users as much as possible I can't stress this enough I value eight organizations on a weekly basis and this is something across the board that often organizations fail to do is not only budget put a budget in place towards this with with training or an education program internally we're leveraging a service externally but making sure to develop you know

bulletins or or have an internal capability for end-users to reach out when they receive you know suspicious emails or maybe it's a social engineering issue but never turned down away and users for sharing that knowledge some of the best Intel is is going to come directly from your end users within your organization so always encourage that to continue all right and that that wraps up our presentation like to take this time to address any questions that you guys may have

absolutely so purple team collaboration is great so it's for some people that might not be familiar with the term a purple team exercise is when a red team works hand-in-hand with a blue team in order to see what's working and what doesn't it's a different set of tactics in terms of I think the the best kind of purple teams aren't they don't have the you know covert quality that a standard red team would have but rather the you know the red teamers can be installed inside the sock and it should just be a matter of seeing what kind of things are alerting do common tactics alert and also just trying to see how effective the alerting is you know a lot of tools

will say that they will stop all instances of me me cats or something like that so seeing you know where the where the faults are but it should definitely be a more collaborative exercise and yeah the the covert aspect should be scaled down in favor of generating alerts when they're meaningful any other questions in the front

you want to I'll take the first half so as we mention in this scenario we often see internal Red Team capabilities that that spend too much time performing Red Team exercises to wear the soft team members just can't keep up with with the findings out of those Red Team exercises so managing day-to-day operations with you know what they're what they're doing within the sock you know keeping the monitoring detection you know analysts in place there's no time dedicated towards you know taking action to those files out of those reports so what I often see is an overload to where the red team's trying to share you know their findings to the sock and the socks just simply can't

manage you know putting the the patches in place you know to cover those gaps from the findings that were observed in addition to maintaining you know keeping the network secure and detecting sure so I think both have their place the you know the benefit and the main problem with a covert red team is that the attackers are only going to use whatever they need in order to complete the objective since an objective based test they're not going to touch anything that isn't directly contributing to accomplishing those objectives so the more gamestyle where you know where team may be aware they have the benefit of attackers don't need to be concerned about being as covert as possible and

they can kind of just test the whole range of things so you know maybe in during the course of a covert red team the attackers wouldn't try to break into or access every single web application that's hosted internally but that may be a concern and those should definitely be tested just like any other part of the internal network so there's definitely benefits in both and I think Eric touched on it's definitely matter of timing them so that way there's enough time to respond because a lot of times the the findings from either one point to a larger systemic issue as a - something that can just be fixed with a patch or some other quick fix so yeah

both have their place and I think over tread teams are great for simulating you know how or testing how well an organization could respond to a very sophisticated attack group that had a lot of time and a lot of a lot of knowledge but yeah both definitely have their place and I don't think an organization would be complete with just doing one versus the other and just add on I think the frequency is important too and ensuring that you know the management from the soft side of things in addition to the management of the red team's out of things communicate that you know ahead of time to establish a frequency so doing that on a weekly

basis you know is overwhelming to your socks so you know agreeing on a time frame that works best I think is uh is key there sure all right if there's no other questions I think that wraps up our presentations we have our contact information here we'll be here throughout the rest of the day if you guys have any further questions we appreciate you guys dedicating your time today to attend our talk thank you very much and enjoy the rest of besides [Applause]