Building a SOC

there we go all right we're good so welcome to our talk thanks for coming thanks to the bsides KC crew for having us um we're really excited to be here today to talk about Bing the stock oh yummy uh so who am I uh my name is Jake Nolton I'm a former security an and engineer I am on the board of directors for vetc we're both representing vetc today it's a 501 C3 for uh military members and then I am currently the principal consultant and team lead for a well-known search engine not that one you're a criminal if you recognize that and then I am also a dad and cat owner and I'm Ben Spencer uh I'm a

former security engineer and instant responder uh I was the differ lead and then MDR director uh currently I'm a Solutions architect so I I have a number of years of ir experience and management experience I'm the one bringing some of the management experience onto this uh and incidentally enough I'm also a cat owner so let's talk about socks now I'm not going to bore everyone here and go into the details of what is a sock I think everyone in this room probably understands what that is but there are a couple things we do want to point out that I think are really important for this talk um the first thing that we do want to point out here is that socks

primarily grew out of a knock environment um the detection response aspect really grew out of the environment where you're looking at a bunch of network logs and then they said hey Network people take look at these security logs and also tell us if we're uh if we're in trouble so you know thinkk a lot of screens thank all the Blinky lights all that sort of thing uh and at the start visibility was the primary worry everyone was concerned hey do I get this log or do I get these logs or can I get this detection out of these logs that's still a concern but we've matured a lot edrs and xdr have come a long way uh we're no longer using Maca

in a lot of places we're using a lot of the edrs and things like that in many environments it's trell now which is it makes our lives a lot easier from a detection response standpoint um but it has also made our lives a heck of a lot more complicated uh as we've matured the challenges have really shifted it is no longer something where we're really worried about the visibility aspect a lot of the challenges that we see in socks are going to be operational challenges um these operational challenges are varied we're going to talk a lot about them a lot about them in this talk but the one thing that we want to point out right at the start is

that any sort of detection response organization their work is going to be unplanned it's stuff that comes up it's a pain you're shoveling it every day and you you can't really plan for it right it's not something that you're going to be able to say hey from 1 to two I'm going to go look at my alert sort of thing that's just not the way this work works so the difficulty is a building an environment that allows stock workers to thrive while also dealing with this unplanned work and that's what we're here to talk about today right so why do we care about building a stock well it's the last line defense for the business

and it's a bit harder than just buying an EDR and turning it on as we all know with a couple analysts so if you don't have people focus on detection and response you're not probably not effectively doing your job with you know there's some exceptions but it's a critical component of it at this point um and with some work put in for detection and response um you know you can get a lot done but without that work how are you supposed to be able to respond to instance would you even know that something's happening and at the end of the day security is really just risk management for the business right and if you have risk at all from a cyber

secur standpoint you do if you work anywhere um it's not realistic to just engineer your way as security as safety you have to do some proactive detection response probably so you know maybe you're a small company um who recently decide to focus more on security maybe your current security procedures are ineffective and you all realize that and you decide you need to work on it maybe you're experiencing the after effects of an incident or breach and decid it's time to restructure to rethink things you know you just got a ton of money and if you're not a technical person or security person attending this talk I would say that you also care about building a St uh cybercity threats

are a threat to the business that's primarily why you have any sort of detection response or so it's one of those things where just about everyone can at least get glean some value from a conversation about this so we have a quote here uh the idea for this quote really is is honestly to talk about the idea that security is not the only driver to this we understand that when you build any detection response org or any Security Org you're subject to constraints um no one has un limited money no one has unlimited Manpower you can't just go hire the best talent in the world and say okay go come in here and do security um really what

it is uh your goal is to build a sock that's going to be sustainable and functional and it's difficult to do that when you have certain constraints so we understand all that um but there are things that you do need to do and if you're unable to accomplish or build those things I I really kind of ask what you're attempting to do when you build a stock if you don't have the resources budgeted to you if you know that you can't accomplish your goals um you kind of have to step back and realize that security does not generally bring value in of itself to a business it's there to mitigate risk and you can't just say

well we have to be secure for whatever reason so the idea here is is we understand that you you don't just have unlimited funds to build the sock but we're trying to give I guess I'd say fund agnostic uh advice here yeah there we go so let's talk about your average stock before we jump into how do you build one that's effective uh Your Average stock is generally four to five analysts most of them are junior all of them have been in the industry less than six years now when we talk about this Average stock by the way and we talk about Junior analysts and things like that this is an average these are things that we have seen at a number of

different organizations I have seen them I've seen this exact sock at an organization with less than 500 people I have seen this exact stock at an organization with 20,000 people protecting vastly different environments but this is kind of the General stock that I talk about so don't don't quote us too much on this but it's we want to set the stage here so two to three Engineers are ALS two to three Engineers or senior analysts are also on there um they've been at the organization for a little while some of them are new generally at least one of them is new generally you have one senior person turning over relatively frequently um at least from what we've seen um this

organization uses a multitude of tools uh some of them are not part of your daily operations than your net witness that you've set up and only look at whenever you're worried about a certain threat um if it even works yeah if it even works uh same thing with Cisco ice it's just a lot of stuff that you have a lot of tools and there's a lot of work for your analyst to go do and there's a lot of things that aren't being used in a very efficient manner um um the other thing to note is you are generally 8 to5 with rotating on call some organizations run 24/7 most of the time you're not most of the time it's going to be an 8

to5 with the rotating on call um everyone who has who is on call says hey I I don't want to get a call uh so you're just hoping that your phone's turned off at that point um most analysts will last one to two years as an analyst this is something that we see throughout the industry repeatedly um it's something that when I talk to people about it it's yeah how long were you an analyst oh a couple years oh three years sort of thing most of the time at an organization you're rotating through anals relatively quickly and then finally there's really no career progression outside of moving up into management you don't have a pent testing

org you don't have an IR org most of most of the entry level people that I talk to when I interview for them uh I asked them what they want to do and they want to be pentesters uh most of the time you're not going to have the ability to be a pentester at your organization unless you work for some of the largest ones out there so you your only career progression is becoming that senior after that one senior that turns over all the time leaves or moving into management yeah so I just want to touch on some things too so we're he talked about you know most analysts only last couple years you know people are

rotating on the company uh this is something we're going to talk a lot about which is analyst burnout um you know we've seen people just leave the career field entirely become developers do appx stuff you know instead which you know he can kind of combine a little bit my last uh um employee went off and did I don't even know something else my wife's boss is a real estate agent now a new one some of us do a very bad thing and go into uh the evil part of the world the sales aspect so uh a lot of people leave this career field all right yeah so why is this a problem well socks are last line defense against the

adversary right your primary line defense would hopefully be good engineering and architecture but we all know how often that happens so um you know at this point you probably have a couple brand new analysts s with some seniors and they're what's keeping your organization from getting breach other than luck so it's also a problem because analyst work is super complex so those of us have been in the industry for a while we know that you know whatever you want to be when they first join a pen tester right but most of us you know end up in some kind of engineering work or something else um and aren't that so the career field is wide and your inist are

probably going to have to do a little bit of everything um but is are they going to be effective right are they just a little bit effective in some areas and more so than others um yeah you may have a team member that's you know can do one thing better than others and if you're a good manager you can learn to play those strengths but you're probably going have gaps somewhere um and that could go un addressed so and security is you know a large investment it's not cheap you know Splunk just got sold to Cisco for $28 billion with a be um so it's a huge amount of money in the industry you you know these tools

trainings and payrolls aren't cheap but they're happening because security is a necessity businesses have decided that security is a necessity to mitigate risk um but not every company is getting their money's worth right so and not every company is using their funds effectively either um so where does that leave us while you're treading water and in reality guys the model that we have we've basically decided on at this point that came out of the KNX uh it it's just not sustainable what I see a lot is I see a lot of analysts going through their day-to-day work is closing alerts they they've got maybe 30 minutes of engineering time maybe some days they have a couple hours of engineering time

but a lot of that time is filled with meetings um and in the end what this leads to is basically an environment where you're never going to be able to mature from a security standpoint you're never going to be able to mature from a business standpoint from a risk statement you don't you're not able to get ahead of things and so you're very very reactive no matter how many threat intelligence tools you put in there no matter how many sores you buy and then try to put into your or organization and say okay that's going to fix it all you cannot this model just does not work because of the turnover because of the large amounts of different threats that

are coming out every day and you're very constrained resources because of the organizational challenges you know we see Team set up security analyst technical security analyst where they're like under the CFO you know there's no CSO or CIO even so you know are your funds going to be allocated correctly you know when you need them if your manager doesn't even understand what you're doing probably not all right you want me to take Tech so yeah there will be definitely be technical challenges so who here has been in a situation where your organization has made major investments into security stack from a tooling perspective right but you don't have the Manpower or the knowhow to fix your

tuning your automation fixing tools when they break uh your log flows are all messed up you may not even know right so you and your analysts are barely staying afloat and now you're bouncing from instant to incident probably um the technical challenges can stack up quickly and compound the organizational challenges from a management perspective I'll tell you that a lot of this a lot of these technical challenges arise out of management dysfunction uh how many of you have actually been in a place where they buy a tool and then no one's actually been assigned to actually operationalize that yeah it's pretty common it's something that as a manager I did this to my team uh and it was not

purposeful it's just one of those things where it's like I've got a purchase sort of coming down the pipeline and I I don't have anyone that can actually do it stuff came up I've got an incident what am I going to do yeah people get busy stuff Falls by the wayside but if you don't go back to it then it's never going to get done so before we jump into kind of the rest of our talk here I do want to bring up something that I think is part of the structural issue here and that's the incentives um the organization has a security team to prevent breaches you on the security team do want to prevent

breaches most of the time I'd say you know if you're unhappy that's that's kind of what happens but most of the time I would expect most of us want to prevent breaches from happening functionally this is where our incentives diverge your business really what they care about is preventing breaches and being able to prevent to mitigate risk that's what they care about you as a security practitioner you care about getting better at security growing your career uh you also probably care about making your security team in your workplace better that is something that I think is reasonable and as someone who works in security I do care about that but I know that the business

if they could basically pay less to have the same security team and as long as all their risks are mitigated they don't want to do that so it's one of the things where if you're an internal security team you're heavily divor divorced from business objectives in a lot of cases yeah so quick story so where Ben and I met the first place we worked together um was basically a Payment Processing Company they did some stuff but um they had really great technical Talent right um the problem was was that all the technical Talent had entered the organization within the past like two years I think pretty much all of them um and the company had been

around since the 80s or so you know so they've been doing the best they could with you know as little resources going to security as they could and as a result there were a lot of problems that were endemic to the cult culture of the company and those problems are really hard to root out once they're in place you know this architecture wasn't built correctly um the leadership didn't really understand unless the CIO was in their ear you know telling them how they should be doing things and they would just say why you know business decide did not understand but what that mean for us as analysts meant a lot of really late nights running incidents as in like

IR type incidents as Brin new security on us trying to figure out what the business side is asking of us because we're on bridge calls with like the board basically um you know as somebody who just ENT entered the industry like the year prior and then you know we're also trying to decide what we need to do as security practitioners we're trying to do our best and we want to protect and you know make our team proud um but this is in addition to both of us already doing engineering projects Sears School everything you know that we do as analysts to try and get better at our jobs um needless to say we got kind of

burned out for a while and I will say that this organization is fairly good at Security even to this day it it doesn't sound like that that Jake kind of talked a little bit about it and makes them sound not very good uh they invest really heavily into security so it's it's not just a thing where it's you know a business doesn't care about security the structural issues are rampant throughout the industry no matter how much money or resources you pour into it so obviously this is all fine uh you guys can leave you can go take away that hey we don't have to change anything this just works um and in a lot of cases

that's what businesses have done uh none of the stuff that we're talking talking about right now is honestly groundbreaking at least it's really easy to point out these challenges so we're here to kind of talk about how maybe you should look into fixing them um which hopefully has been covered a little less uh and we're not just poking at hey this is how poorly organizations are run uh no one really likes just someone who complains about things um so hopefully we're going to be able to provide a couple Solutions so the first thing that I really encourage everyone in this room and anyone who is part of any security organization to do is honestly determine

what your objectives are um this is easier said than done in a lot of PL in a lot of cases and one of the things about determining what your objectives are is also determining your constraints so is your stock there to just detect and respond on threats are you just there to go through and shuffle through alerts or are you also an engineering org to um are you also a risk org where you have to do Audits and things like that what is your actual jobs in your security organiz organization um the other aspect to that is then understanding your actual business risks uh most organizations do not need to defend against Russian APS as much as we

like to think that we're part of the organization that does most of us do not I currently do not work at an organization that probably has to worry about that quite frankly um it's just one of those things where you need to understand what is your threat model and where are you at in that threat model and what does that mean from a security standpoint for you um the other aspects there is once you determine those objectives get alignment from the business with those objectives all too often I hear a CEO that says oh yeah my security teams they've got all this they're doing all the stuff they're protecting everyone and then what I hear

the security team say is we're just barely getting by we we don't know what's going on here we don't really have a great understanding of things and so what you get into this point where there's just a breakdown of communication around the security team can't do their jobs the executive team thinks they are doing their jobs just great and a lot of times it's just no one's really talked about it um the only other thing that I will bring up here is if you are at an organization with a sock determine are you running 247 or not um this vastly changes the resourcing aspect of your job it vastly changes what management needs to happen

um in a lot of cases I'll be honest it's really hard to run 247 so if you think you're going to get by with 8 to5 and then on call um I just hope you don't have an incident that happens at 1:00 in the morning uh I've had to respond to those incidents thankfully not very many internally but from an external standpoint uh it's really hard for a lot of analysts to to go out and wake up and start doing investigations yeah it's really want to foot stomp something you said too about threat modeling this is the ideal time to threat model in the beginning stages understand the you know who's going after your business um you

know the historical impacts of stuff like that um you know how you would defend against that now is the time to threat Mell during an instant is not the time to threat Mell please do it early all right so what are some immediate actions um from an analyst perspective we need to communicate we need to let our leadership know you know they don't know what they don't know so we see a lot of analysts who are suffering in silence because there was a breakdown in communication between you know what was happening on the ground and what the ceso or CIO knows um so if you're the manager who's only filtering up Rosy information to your chain of command

because everyone is somehow getting by um what are you going to say when something finally happens somebody finally gets through and you're now you're on a bridge call at the CEO explaining that you didn't report the team has been having challenges with the tools that the company spent an enormous amount of money on um to get in first place um to pre prevent this very thing right it's going to be really awkward so uh talk to them the best time to break bad news is right now waiting doesn't solve anything and it'll only further burn out your analyst while you're waiting for a bomb to go off um document everything you know put documentation in

your org's Confluence if you have one write PDFs put them in share Drive do what you have to do write it down everybody hates documentation but you have to do it um use what you've learned in past life I was a systems engineer in the Air Force um you know Ben was a network administrator so you know everybody has these diverse experiences maybe you worked in like Communications or something like that um and now we need to talk with leadership about the incident that's going on so maybe you have a diverse experience that can fit in there have you taken a class you know youve done gcfa maybe that could come in handy so speak up make suggestions talk

to your leadership don't be a silo and if you are the management manager who is going up there and presenting a Rosy situation when things aren't all great stop doing that it it doesn't really help you in the long run um the other thing that I will say is we're not saying when we say communicate to leadership go ask for a bunch of resources or things like that honestly situation reports are incredibly effective telling telling them hey this is what's going on on a day-to-day basis having those meetings and taking the time to communicate and tell people directly this is what we are prepared against this is what we aren't prepared against and being very straightforward

with that is going to help you a lot most of the time what I see time and time again is security Security leaders will go up and ask for resources instead of actually just describing what the situation is there and letting the leadership of the organization kind of decide hey security might need more resources or we need to think about doing this in a different manner yeah in terms of you know purchasing resources make sure that you or one of your senior members are involved in the purchasing process we see a lot of organizations where it's like the CIO you know driving purchases Alone um and they may not get the best stuff so make

sure that you try to involve yourself or your manager is involved in those decisions all right so let's say that you're not just treading wire more your maturity level has increased a little bit so what do you do you got to start automating stuff um so if you know you know some python some Powershell bash try to automate your repeating task may take a while to automate things especially the first couple times but one it's great learning experience right and two future you will definitely thank yourself um learn how apis work work with your network team to safely allow the traffic that you're trying to automate um and integrate these automations into your processes so you

know you can ask a friend or a Dev for help your company you can use bar or chat GPT for boilerplate code actually you can just put whatever you want in bar or chat g chat GPT it it works perfectly fine don't worry about it for the love of God do not put your company and client data in chat GPT and Trust open Ai and then future automation is going to be easier right a lot of the apis are similar it's probably like Swagger or something um comment your code learn from it mature your processes um and then use the documentation to your advantage so what I really like doing is using jupyter notebooks because I can

put my documentation and the code right there so then if I need to train new analysts who come in it's really easy this is how we execute things and then if they need to go back and change things in the future it's all right there super easy I I will say this it is incredibly important to try to do some sort of automation at least for a lot of the repeatable tasks there it it's not something that requires the whole org to buy on to it if it's just one or two things that you really hate doing try to find a way to automate it it's also a great skill to learn and for me

personally I learn best when I have a problem right in front of me to solve um I expect a lot of people are like that so it's it's a good opportunity to practice some of that skill yeah can you the code you gave me through an error can you fix it it worked fine when I tried it thanks chat GPT so Jake talked a lot about a lot there and one of the things he did say is create a uh create repeatable processes and create documentation so that it's easier to train new members uh and that's really very important um you're probably not going to be able to remove the aspect of analyst turnover it's part of the industry um there is

maybe not a skill shortage but there's a lot of people who want to move up very quickly uh and that happens so you need to have some sort of idea of how do you forge a proper team and how do you build the right team um one of the things I will bring up just immediately is realize that people need rest if your organization is an organization where all your analysts are running 100% And they have zero time to themselves um you're going to create a lot of alert fatigue it's just the way it works and I don't me you could have the best tuning in the world there are still going to be false positives and your analysts are

going to be far too tired to actually investigate things effectively um from an instant response standpoint I have been I cannot tell you how many incidents I have ran where there are people who have seen the actual bad and then what happens is they don't investigate further they go and deal with the immediate remediation and they don't don't dig in further you see it all the time b64 Powershell uh ran on a server and no one goes and looks for the rest rest of it and it's just because honestly the analysts are overworked they're incentivized to not do that so what I tried to do was for my teams I try to give them about 20% slack it's

hard to do that it's hard to get enough resourcing to do that but honestly sometimes as a manager it's your job to tell people hey uh we cannot take on any more initiatives this year we can't operationalize another tool we have to do what we have to basically level up what we're doing now and then next year or the next quarter we can start taking things on that is part of your job as a manager yeah two things to add on to that too so you know uh first I thought you're more of a teams guy not a slot guy I am a teams guy thank you um but yeah so the stockwork being unplanned work and that

people need rest um also please don't beat up your analysts when they're learning if they make mistakes cuz that's like one of the reasons why I left my first company so they're learning you know let them learn let them give them an environment where they feel comfortable making mistakes right like don't put them straight in a prod and let them break things but you know let them learn let them you know develop their skills in a place that's safe for them I guess and to combine on to basically jump off of that technical ability when you're interviewing is heavily overrated in this space and I'm probably not going to make a lot of friends saying that um but I can't tell

you how many interviews that I've been in as someone who's ran very large IR uh where I've been asked questions like hey what port is pingin on uh can you go do this command line for me and then run this command line stuff that it might be good judges you know a good way to judge technical ability but in general brings nothing to your organization uh from basically a skill set standpoint what I will tell people is when you are building a team understand what your team actually needs what the business needs and what your team does um honestly sock work is so generalized that you can train a lot of people to do a lot of this work it in a very quick

amount of time um I've hire hired analysts who have had zero technical uh background and zero it background and they come from an intelligence Organization for example and within about four to five months they were operating at a very good level they were operating at basically an L2 level and what I would say um and all it took was some of those repeatable processes and your culture being open to analyst making mistakes and people asking questions um the other aspect that I will say for anytime you're hiring is figure out what opportunities do you have for the people that you're bringing into your organization and I don't just mean hey there's a promotion uh Mo like

I said most teams do not have red teams most teams don't have IR teams that you can filter people into but what you do have is you do have time to run tabletops you have time to say hey we're going to do a CTF um you have time to send people to conferences figure out those opportunities for people to learn and if you do have the training budget send people to go get some Sears because in reality what's going to happen is they're going to go take all the stuff that they've learned from all these different exercises and bring it back um they'll also leave your organization in a year or so but that's okay that's

completely normal you you want to take that advantage of that experience while you can and then when they leave figure out hey how do I hire the next right person I mean they don't even have to be expensive conferences right like we're at bides casc right now it's not expensive there's bides all over the country there's places that you can give your you know ample opportunity to learn without having to send them to like RSA or something like that finally after you've done all these things after you've kind of figured out this basic structural stuff that's when you start enabling your team and by enabling your team I mean getting them the right tools and spending your budget

appropriately um so I I I've kind of switched to the bad side of uh I guess the evil side of some of our um security realm where I I do a lot of sales now um and one of the things that I have to talk to Cent cents about Trad yeah yes very much a Trader one of the things I have to talk to clients about repeatedly is hey do you really need two years of hot log storage are you Microsoft no okay your compliance requirements are probably going to be things that decide a lot of this stuff from a purchasing standpoint really understand what are your analysts going to get use out of um

it's one of those things where I I point out edrs are really effective for analysts because they're a graphical interface for people Sims are much harder for a lot of people to learn from so you're RS are going to be really effective for basically hiring on newer people whereas if you have run everything off of a Sim it's a lot harder for people to come up to speed so understand what your team is understand those objectives and then buy the right tools and spend your budget appropriately I also don't recommend going out there and buying the most expensive tool on the market Splunk is great Splunk is also really expensive if you do not have a specific need for

Splunk I know it looks really cool but be honest with yourself are you going to be able to operationalize all the really cool features that they have or are you going to go with this something that's cheaper and it's going to be a little bit paired down but you have to mature anyway to get to any use out of that tool um the other thing that I will say is whenever you do buy a tool please allow time for someone to go and operationalize it you're just spending budget that could be spent on staff salaries or other resources when in reality you you need to have someone who's in charge of bringing it onto the

team and so that they can actually use that tool itself they need to you cannot say hey we're going to buy net witness and then the team's just going to set it up by themselves that's not going to work you need to have a full project plan for it um yeah I mean if you have the budget too you know use the Consulting side of for notet witness RSA so you know have them come in set up for your analyst so that they're not spending time you know struggling and then they'll have a working tool that they're able to get trained up on and use there's a lot of different ways you can use your budget effectively other

than just buying tools and hiring people and that's really kind of what we want to say on this slide at least is understand what your budget is and understand what your organization needs oh go back for a second I want to touch on one thing so no other the way wrong way so engineering is different than analysts work too they are not the same thing please treat your engineers like Engineers they're going to be working with Engineers from other teams they're going to be working with developers they're going to be working with network Engineers with system Engineers they're trying to build stuff so please don't treat them like normal analysts don't make them triage alerts

let your analyst do analyst work um just treat Engineers as Engineers please or else they will leave you got to insulate them from the unplanned work because it really destroys any sort of long-term Project work okay now the slide where we're going to make people mad unless you work at an mssp or like an MDR service then you're going to be really happy so are you sure you really need your own inhouse stock not every organization can run an effective stock um ask your business why they want to run one in-house you know why have we gone to this it's not just the business it's the people in the stock too they're going to create a lot of work you're going to

need Insider threat there might be poor work results from the team um just the lack of skill or mistakes can cost the business a lot of budget as well you know are you able to effectively secure the company with the budget that you're given and the constraints being imposed by the business so you don't always need you know an internal stock maybe if you're um if a tiny company right you're just making apps with like nine people you don't need a stock you have a team of Engineers just build it securely or if you're you know large team you can you know Farm it out really and in reality and it's not just hey if you're

a smaller team a smaller company you don't need a sock if you're a larger company there are some ways that if you Outsource that L1 L2 triage and then have a smaller dedicated team doing engineering work and doing high level instant response you can get a lot of the same things now now I'm not saying you have to Outsource your stock or things like that but be very honest with yourself do you have the budget are your objectives aligned with the constraints that you have and then go back to your leadership and say hey are we going to be able to accomplish this Mission or are we going to need outside help and it's just one of those things where I

see a lot of companies struggle I've had to respond to a lot of companies that struggle uh and they have their own internal stock and I just talk to the analyst I talk to the engineers I talked to the managers and they're all just sitting there saying we're trying to fix things we just can't we don't have the money we don't have the resources or the the threats are just too advanced and so it it's be honest with yourself take a look at where you're at and then and then say hey maybe we need to purchase some outside help yeah I mean there's a lot of you know experience that you know an out outside help can bring too a lot

of resources that you know they've already scaled for and they may be able to bring in cheaper than it would be if you did try to doing that internally so um you know calculate your expenditures see how much you have um you know is your business even willing to consider transferring the risk to an mssp though it's a conversation you have to have with leadership and and this is something that you do have to have with leadership not just with the sees things like that but this is something that management needs to have with higher level leadership the other thing I will point out is treat it kind of like pentest teams and IR teams most

organizations don't have pentest teams they farm that out and it's the exact same reason why it's the exact same reasons why you might want to look into having someone do some of your triage work yeah just think the same thing you know we're listening to a lot of these red team you know reports today and uh all the red team presentations there's no way that a lot of like smaller you know teams can handle that amount of you know investigative work they're going through so consider it at least all right so some are our final thoughts and we're going to set it on the slide for a minute you know so we talked a lot about socks and why they're

difficult to build um we have a couple final thoughts but in reality you know the main thing we want people to take away from this is that building detection response organization is really difficult and it's expensive business and security incentives aren't always aligned and this creates a number of difficulties that aren't technical at this point but are like entirely operational right so the thing to takeway is that you know maybe they're we hear a lot about like a cyber SEC cyber security skill shortage um is it at this point or is it just a poorly designed you know operational environment that you're having to deal with um you know how many people are here that can't find a gig in cyber

security right now there's a lot of really you know there's a lot of people at the entry level especially um that are competing for gigs and I think a lot of that problem um when companies say they can't find entry-level people is just that they don't know what they're doing um most of the difficulty comes from having poorly designed operational environments that are ran by people who may not understand how to interview or hire even um they don't know how to meet those challenges um they don't know what the business needs they're just kind of flying by see their pants and you know lastly honestly um a lot of businesses probably just shouldn't run their own

detection response organization unless they can handle it um there's a lot of complexity and difficulties involved involved um you know so do the math right look at reputable msps MDR Services if you're financially able to even offload like the tier one tier 2 triaging that's going to be huge for your analysts they can concentrate in the more fun things um on the engineering work it's a big win um but again you know that involves talking to the business you there it is there we go involves talking to the business getting align with accept acceptable risk mitigation strategies um but you know hopefully at the end of day you can keep your analyst from getting burned out and

and I will will just say I want to point out one of the things the there's a lot of entry-level analysts who have a really hard time getting jobs um if you're a manager that gets promoted into a team and your only thing is just keep it running you don't want to improve things that's fine honestly that is but a lot of times I that's what I see that's the reason why I see a lot of these entry-level analysts be unable to get hired when they're clearly qualified it's just hey we lost one guy he was an expert at network security we need to bring someone in who's a network who's an expert at network security we're

going to pay him $70,000 a year and well I can't find any Talent this it's no one wants to work anymore sort of thing right so it's think about are you trying to improve things what is the structural issues you're trying to address when you're a manager don't just say I got to keep things running the exact same way that they're at right now yeah I mean a lot of what we see too since Co is companies that traditionally hired you know from their own city or town um they were pay anals a lot less then they went full remote now they're competing nationally um with salaries and their are just leaving because there's a ton

of opportunity elsewhere and then they say oh nobody wants to work we can't find anybody well you're paying $30,000 a year for a security analist so so thank you for coming to our talk

are