BSidesCharm 2023 - Baby Steps to the Future – Evolving into the Next-Gen SOC - Craig Bowser

foreign [Music] good afternoon good morning welcome to the second talk uh the third talk depending on what you're counting so uh it's awesome thanks for being here I know that this is an interesting time slot because I am effectively between you and lunch and so that's a little challenging so what I'm going to try and do is make sure I am both at least engaging as well as succinct and so I promise not to go over three hours that can we just just be happy with that all right great no seriously um so my name is Craig Bowser and I am talking on baby steps to the next gen sock and I have been Security in security doing security for over 20 years in various roles almost all of it blue team and so that has given me a lot of insight into how security Opera operations has changed over that time and and as a result of just you know being in that field as well as uh talking to people at cons and online I've just seen a lot of how you know that has changed and is changing going forward and so uh that gave me some really thoughts about like hey what's going on with the sock what is going on as we move forward uh both in the past and in the future and so the first thing that usually happens is we have someone pronounced that the sock is dead and usually that has you know it gets some interesting engagement going I mean uh just uh a few months ago there was a whole Twitter thread between Anton shavocking and Richard uh um and I always screw up his name uh latex um so um baitlick Richard baitlick um all on this topic but the reality is that it's sort of like you know the monarchy it's long-lived the sock the sock is dead but really long move the sock and the reason that of this disparity this difference between these two viewpoints is that um in uh what's going on is that the sock isn't really dead but the sock is evolving uh and this this stand microphone this single in place microphone is driving me crazy I usually move around on the stage so hopefully I will not do that and if I do then our good Tech team in the back will scream and yell and tell me to get back to the mic because I can't hear what I'm saying but nevertheless the sock is evolving it's moving and it's changing and really if you think about where where we have come from to today you understand what's going on right back when we first started and for some of you this is before you were started and for a couple of you maybe before you were born which just goes to how old I am uh you know um this it was the defense of the network was simple we were on-prem we had this castle and moat philosophy where we have this hard and crunchy outside and a soft and chewy inside we had AV we had firewalls ids's were just coming into uh just coming into Vogue and so that was that was the protection we had and then we got a few more tools we started adding remote capabilities to our Network I remember I was um you know add an organization and they gave me a 386 box to take home to dial in to do work at home so I'm you know lugging on the subway a monitor a a case with the 386 keyboard you know the whole shebang that was that was my laptop at the time right you know quote it was a whole you know um base but anyway um here I am you know I'm remote we have ips's are just starting ubas or nascent I know when I say Nation it's like the concept and very you know lab like uba and then we moved to where we have more tools and so we start moving from a castle and a single perimeter we have the defensive depth concept starting we have cloud is starting to come out 2010 remembers about when AWS started offering services and capabilities uh threat Intel and uba are just starting up uh and so EDR the concept of DDR is again nascentient oh we moved to the mid-2010s the mid teens on our um and uba and threaten tell are all over the place I remember going to a major conference 50 60 vendors and 80 of those vendors were selling either threat and tell or uba it was like the thing remember just like every new thing is the thing and then it's never the thing but now we're starting to see also sores starting to come out um although most Linux admins will tell you they've been doing soar for years but that's a whole other story um you know microservices are starting to pop up and now in the 2020s uh we're seeing seeing us shift again as defense and depth is now uh moving towards zero trust we have more tools we have xdr we have Cosby we have Ace um a tax service management tools we have all these other tools that are coming in and we're still manage we're managing all of these and I'll talk about that the fact that we're managing all these but we're adding mobile and um and and operational technology into the things that we need to protect and so our socks also have changed a little uh we went you know when it was all on-prem uh you know in the early days that we could put everybody in room with with monitors and screens and Blinky lights and things like that and that was good uh and then you know we started having the ability to remote monitoring and remote analysis and uh covid and the quarantine drove this quite a bit uh and now you know we have a lot of people most people are either fully remote or at best hybrid and their socks now a sock by the way traditionally we think of it as the picture on the left but really what the sock is uh it really kind of balances out is it's a group of people doing security monitoring detection response right so whatever you want to call that your security operations team your incident response your cert whatever you want to call the sock it's a generic term for that group of people in your organization that are doing the security and monitoring for your organization so this is kind of where security as far as the team goes and where we're going with the tools and so but so what we have now is this but we still have a sock architecture that looks like this we have a bunch of uh tools on the left here that are sending data to some kind of central a repository now most of the time that's a Sim but it could be other uh other capabilities other storage and analytical capabilities I'm just using Sim as that generic term because most groups have that but you have this Sim you have a tiered group of people that are uh you know doing analysis and monitoring of the data coming in of rules and such you have some ticket ticketing case management system you maybe have some threat and tell you maybe have a sore and when I say maybe I mean a lot of places do but not every place does and that's how our today's sock is but the reality is that this architecture the way this sock is designed is really geared toward defending monitoring and protecting the defense in-depth capability uh with you know a limited amount or a certain amount of data coming in and so what is the problem with that well that has created some challenges in today's environment as the networks and the Enterprises that we are charged with protecting uh have can have changed faster than our sock architecture first of all the elephant in the room which no one will be surprised is the lack of personnel to do the monitoring protecting I don't really have to talk much about that you guys talk about it all the time we all do we've talked about it for 10 years we'll probably talk about it for the next 10 years getting enough Personnel to do this is hard um we are managing an increasingly complex security stack both which includes on-prem cloud and mobile capabilities The Cisco security outcome study uh stated that uh on average there are 63 security tools and I will come back to that number later in the presentation but there are 63 security tools that uh single organization is responsible for managing administrating monitoring and responding to that's a lot of tools and not all of them work the same way and you for those of you who do multiple things your head explodes trying to figure out which tool you're in at what time you have limited visibility into an increasing attack surface okay uh your attack Surface by the way if you haven't figured out on your own is increasing it's growing as we expand into the cloud as we expand to mobile as we expand around the world into new locations offer more services that is just more places for the attacker to uh to Target and you have to monitor all those that's challenging uh you are not able the socks are less and less able to operate at scale to handle better attacker attackers with increasing volumes of attack um there you know we you know there's just attacks going on all the time from multiple places and by the way at more scale and by the way you have not been cloned yet um and so that's hard of course cloning would solve problem number one but that's another story so we're still fighting this fight this fight probably will be fought forever but it doesn't it still adds to the problem right where you have compliance which we use to get resources so it's not bad but it goes up against best practices which goes up against effective security which goes up against operations in I.T so these are all things that we're trying to fight and balance between the two it's that's hard uh attackers primary targets now are no longer devices they are identity and data and so we have you know so we have to shift how we think uh if you haven't already some there are organizations that have but if you haven't already shipped how you think about what you know we have to do that and not every place has so how did we get to the how did these challenges come about uh first of all operation shifted they moved to the cloud they moved to devops and they did this 10 years ago and most security is not even close to being able to help in this manner we are um so operations is pushing out updates and changes and new things you know two to three to up to eight times a day depending on the industry that you are in and you are woefully behind you're still waiting at the show it up show up in your test lab so you can run through a battery of tests to determine if it should be uh pushed to uh production or not operations because like I they have bypassed you uh in many ways and you know and because they they cannot slow down to wait for your testing or or analysis uh you know in some ways you know there are definitely organizations that have adapted to this process but at the same time it is a challenge to ensure that you know the update that was pushed out 11 A.M is secure when there are already four revisions past that at 3 pm so that is a challenge we have moved to a remote mobile anytime anywhere access that is whether it's your users or your customers right there they there is a desire to hey I need access to the data I need access to the store whether I am at Starbucks at home on the beach on the mountains uh I'm on a laptop or on a phone um on a tablet you know or I'm on some kind of iot device anytime anywhere uh that's hard we don't control most of those endpoints all right this is different than when we controlled almost all the end points almost all the access and the location so we're no longer at that point I do I do have some good news security has become better anyone currently worried about Red Alert or Slammer anyone no those are still out there they're still popping around the internet we're not worried about those things right we're you know so secure security is better than it was 20 years ago I remember who remembers your network shutting down right of an unplugged that Network because of those things yeah we don't care anymore whatever uh but guess what uh the attackers are way past that the script kitties are way past that so um you know the script cookies are like Slammer so that's old stuff right so um so you know so the attackers are better uh but so are we so that's good and bad but all right security tools and methodologies have increased in number and complexity I've already talked about that and so that's hard that's a lot more data from a lot more tools each one of those tools sorry each one of those tools according to the vendor who sold it to you is the most important thing since sliced bread um and most importantly we no longer operate under the assumption that our networks are unhackable okay we uh we used to think that if we do all the things and turn all the knobs and Patch all the uh patch all the devices that are we will be unassailable uh we don't don't think that anymore we think if we ever get to that point then somebody's just going to find some unknown zero day or five of them or you know basically um fish my CEO or CFO or Secretary of the CEO and they're still going to get in so we don't we are no longer under that assumption so we don't operate we don't think about that um you know that maybe that keeps up keeps us up at night or maybe we just go whatever I just go to sleep because it's you know the same scary story uh but that's that's no longer the thing so what's the solution spoiler alert it's in the title so we become next gen but what does that mean right you've probably heard this people are like oh next gen next gen next gen so what does that mean well let me tell you what how I am defining it for the purposes of this talk how I am going to define a next-gen talk uh cover several things first we are shifting our security strategy from defense and depth to zero trust and I know zero trust is a big buzzword but I'm telling you it is coming and it is coming like a very slow moving freight train and I say slow moving because this is not an overnight flip a switch thing but a freight train nevertheless that is coming and it is different for everybody so don't assume like oh I can't do it because I'm too small just you know it adapts so but shifting our thought and our process and our strategy to this concept uh is going is a key part uh routine attacks and alerts are handled by automation right get rid of the crap that you see all the time every day um you know what it's like I think about it like if if your bill pay you just say pay this bill as long as the bills for my cell phone is beneath x amount just automatically pay it actually Banks don't have that setting it would be great if they did just you know a greater less than or equal to pay this amount automatically if greater than this thing give me an alert that would be awesome if you patent that give me credit please but um nevertheless uh just anything in a routine just is just automated just do it don't I don't even want to see it um alerts are enriched and tri-aged so the things that don't get our automatically handled they come in fully enriched most of you hopefully are already doing enrichment for your alerts if you're not get to it add data to your alerts that makes your analysts be able to make decisions faster and more accurately because they don't have to go find out stuff that is related to the alert they're looking at so it should show up with all the information they need to make a 30 second decision right away and then triage as much of it as possible that you know automation doesn't necessarily mean you take care of things without human intervention it means it can mean that but it can also mean that you have done as much as possible with minimal risk to your to your operations to your business and now you put it in front of a human to make final decision or to final decisions right so you are you know tree you know triaging as much as possible if you think about like a medical triage sometimes the uh the the nurse or the the EMT does as much as possible and then gives it to the doctor to go okay do this do that do the other right so that's really where I'm talking about thinking about a triaging your alerts as much as possible before the human gets there oops um so detections and automation automated responses are constantly being built in tune so you have devs SEC Ops you have devops in your security your where your your alerts and your automations are constantly built and tuned and two to three to four to five times a day you're pushing out these updates it's changing uh based on feedback and results so uh so next gen sock is doing this so logs and events then also have to be ingested from a variety of sources you have increased the amount of data you're pulling in and I get that comes at a cost and we'll talk about how to manage that in a slide or two but you're pulling in all of that stuff so that you can have detections automated responses and so you can take care of routine and alerts by automation and then finally you're using ml machine learning and Ai and threat until alongside of automated alerts all of these things are helping you generate High um drawn a blank of the word um low false positive High Fidelity thank you that was it High Fidelity alerts right you're using this you know so where you can create Atomic alerts Atomic alerts or things like if I see this and this give me an alert because those two things are bad that very simple alerts you're like you know yes no on off alerts as opposed to machine learning and AI type of alerts where it requires a lot more processing and thought in large volume analysis so um so uh using all of those to uh uh together to help defend it so that's really what I see as an exchange sock so how do we go from uh what does that look like in reality what does that look like practically uh in in real life so we're going to talk about that picture so keep in mind that how today's sock architecture uh was was pictured a few slides ago and we're going to start migrating we're still evolving that to uh next-gen sock because this is not a overnight Journey you're not going to go back and go okay here are the things we need to do and we're going to have them done by next year no this is like just like zero trust just like anything it's going to take three five ten years of of of change slow change but deliberate planned changed and flexible change because things will be different in five years than they are now but you know we need but that's that's you know but this is how you know this is the things the steps you need to take and by the way also uh you can grocery shop in this right if you are a large multi-billion dollar worldwide organization uh you may be able to do all of these things if you are a 100 million 100 million dollar uh 30 person um you know organization maybe you only do four or five of these things the concept is that you are getting to the point where you are doing uh implementing many of those overall capabilities uh regardless of how many of the actual steps you're able to do you have to find a way to do it with the resources and capabilities at your level that you have so let's jump into it so phase one and by the way I broke this into phases not that you have to follow this order but just be because I try to group things that made sense so phase one what you're going to do is you're going to increase your automation Automation and integration okay that same Cisco outcome study also said that that tools that are well able to be no that are able to be integrated well with each other increase the efficiency and effectiveness of your security team and this makes sense right so I was I've been doing a demo lab I've been trying to build a demo lab that has some automated response capabilities in the demo lab and I will tell you that we found out that some of the tools we're using don't integrate well they don't have full API capability they like one of them says yeah you can create a user using an API you can delete a user but I can't suspend a user okay now I gotta either not use that tool or do something else right I can't suspend a user in that tool so I need better integration capabilities I don't need you to open up your you know vendors I don't need you to open up your secrets I need you to just allow me to to access the things that I can do in the GUI via remote API so I can just have your you know tool a push data to tools