Building a world-class soc starts with building world-class people

all right so the talk is about to start um The Talk this talk is going to be um about building s so with les Ferguson he leads um the Automation and software engineering efforts on a cyber security incident response team and he's had over 12 years of experience in software engineering and cyber security I'll give it have thank you so I plan on this being a relatively informal talk um I will warn you I didn't have not practiced or timed it so hopefully it's around the right amount of time if not ask me questions and we'll get there um yeah and go ahead and get started y'all are late um for reference I will say I have

slides there's going to be any information it's just the title of the section um I said it's informal let's just talk through things why should you listen to me um as he did mention I've been in cyber security um cyber security technically for eight years um I've done software development for a total of like 12ish um and I've worked through the entire sock life cycle um eight years ago I started as an associate analyst and was working I'm going talk the topics so starting out people people are your most valuable resource right um a lot of people think it's other things but it's the people the people who make the company without people you don't have a

company so you should treat them like that you should treat them important make sure they know they're valued you shouldn't be investing in tools with the Assumption you're replacing people um that's a thing cyber security has a really bad habit of is going well I can just buy this tool turn it on and now I don't need to hire any body to work it it's not going to do anything for you if there's not someone to tune it to work through it to investigate it it's just waste money whereas if you spend a little bit more time on people you can actually get more value and then they could take that tool and give you more value for

it um uh kind of going into that right your people don't just know everything invest in your people you make they need to grow um and that needs to be both Technical and soft skills um we do a lot especially in in response of communications we're talking to people um calling out Emily's talk from this morning on her EMS thing soft skills are very very important in the instant response life cycle you've got to be able to Comm communicate you've got to have empathy okay and so you need to work with your people you need to give them chances um one of the fun things that we have done that is we kind of take over and share who does Incident

Management at the time who's your Incident Commander and it's not always one person um let's rotate it through so everybody gets a chance everybody gets to talk to everybody and this one is the next one is really really important and often overlooked and that's why you lose people as often as you do compensation your people are growing their value is growing if you don't match that you will lose them they will go somewhere else and all this investment all this time you have spent in growing this person is now essentially wasted you have just thrown away your all of your investment so keep investing do more you grow they grow everybody

grows and as you're going empower the employee empower the business okay your people that you have spent all this time you've trained them you've taught them um you've invested money they've sent them off to conferences and who all that well if you don't let them do their job do you're still losing your investment um almost everything in here is going to be related to people it's why it's the first slide it's how do you keep your people how do you treat them right and one of those is giving them B giving them power um one of our team motos and goals that we work towards that my team is focused on is empowering the responder is a thing that we put on our

metrics and everything what did we work forward to empower the responder to empower this give them the power to make the decisions um even if they are a tier one analyst just doing your first triage line of defense they still should have responsibilities they should be able to feel like they're adding value to your company so set them up for it um we'll get to a bit more of my things but one of those is like tuning a lot of your tier one analysts are going through finding false positives day after day after day and for a lot of companies and socks out there you don't do anything you close a ticket and it's gone and

they never see it again or they find a true positive and they pass it off to an instant response team and never see it again that doesn't give them any value they don't feel important they don't feel like what they did actually matters because they never see the completion they don't see the final product keep them involved even if they're not the one running the incident to completion let them Shadow let them come in and let them do things and make sure they know that their decisions helped let them tune let them give suggestions they''re send the same alert with the same s IP and the same IP address or same hash whatever 15 times in the past

month and they're like why am I seeing this we tell them let's here tell me write up if we don't have it or give them the actual tools to do it to go in and change and add to exclusions let them know that they are making things better they are fixing the things they don't like yeah lost my notes there we go this one is one of my favorite things to talk about and why I fell in love with the team I work on currently is collaboration not competition um a lot of companies a lot of teams are very competitive it is how can I make myself better how can I get ahead how can I go to the next

level IR especially that doesn't work because you can't know everything there there's no way that you can go into a major incident and handle it yourself there are 15 different things there are 15 different breadcrumb trails that you have to follow to find the true source and at the same time someone has to be talking to your senior leadership someone has to be adding things into your block list you can't do it by yourself and if you are constantly trying to be ahead of and be better than everybody else no one's going to work with you this goes back to that treating people with empathy those soft skills be a collaborative be a team um one of the things that I've set out

to do and it's one of my favorite things to do as I've gone into leadership ship is to say my job is to make it where I don't have to do my job anymore um I have loved being able to see people that I was an engineer and they were an associate I trained them I taught them the processes and I answered their questions and now two years later I hear over the cuq ball them get asked a question by one of our new Associates and me knowing I don't have to answer that because they can it's really really nice to see this collaboration and this team and seeing everybody grow every once one one of my phrases would be you you

win if your team wins not the other way around if one person is the one doing the best the rest of your team won't catch up and your team will fail if you have four people only one person is a superstar those three people are still doing work they're going to miss something they're going to mess up somehow but if one person who was a superstar is still working with them and talking to them encouraging them to come ask questions when they don't know they'll go ask and everybody's better you have more growth potential when you're also growing those around you and same idea it's collaboration right one of the best ways to teach in a

lot of senior classes one of my favorite classes and least favorite classes involved the teacher getting up handing out a syllabus and saying okay next week those three students are the ones who are teaching the class and he didn't teach a single class for the rest of it all he did was grade but everybody who went through that class came out and did the next set of levels classes so much better than people that took the classes that were easier because when you're teaching you have to know more you learn more so when you're growing and you're bringing other people with you you're going to grow faster they're going to grow faster and now they can actually

add in and give you more value and more time to spend growing more everybody raises up faster share the load what happens if you're overwhelmed you're burnt out you're doing this thing and you're really hit you've hit three four hours in one incident having someone there to help you hey I need to go take a lunch I need to go get something out here's where I am here's the data share the load we're all as a team when the incident hits 24 hours that first person isn't working anymore if they are they're not like you're useless so things like making a schedule in an incident um that's a really important thing of saying hey I've got a

team of six inent responders but it's looking like this incident is going to be more than a few hours set up a timeline say hey first two hours it's these two people then these two people and you trade or you go eight hours 12 hours whatever your thing works trade share the load so that nobody burns out and then my last one which is why I have this picture here is buses and everybody's probably heard it or seen it somewhere what happens if the bus runs you over on the way to work right it's the bus problem if you you are the only person who can do a thing and you don't show up to work and the company or the

business fails or can't proceed that is a problem because it means you can't take vacation you can't go turn your phone off and relax you are always on call you are always working don't make a bus situation don't have any situation where only one person on your team can do a thing one person can be better but at least two people need to be able to do it you need to be able to take a break or else you will burn out that's a big thing in ir and insecurity is burn out this is another phrase that I've kind of come up with and I've been using a lot which is build Hills not silos we in it as a whole and especially

in cyber have also been very guilty of building silos you find that thing you're very passionate about and you keep going and you keep going and you keep going and you never share it and so no one else grows with you no one else does anything with you silos stifle growth in the long run and create unnecessary politics and friction when you have a whole bunch of SOS and each Silo only does one thing what happens when you need to collaborate you're not going to because it's my thing or you need help figuring out how your tool integrates with their tool well that's your job not my job silos create friction they create politics they slow

things down and they make people not happy um I've had to deal with more than a few teams that have done that approach and I hate my job whenever I do that so don't build silos build what I call Hills Hills allow you to grow in a specialization um don't not specialize right find the thing you're passionate about grow build your Hill but know that your Hill can be climbed someone next to you can follow you up that hill they might not go as high but they can follow you up and you can share down and then you have five people who built five Hills well if you have five Towers you have you have five Towers they're just

next to each other if you have five Hills you made a plateau same thing as I talked about earlier right your entire team is grown because you were willing to share the load you were willing to work with each other build a hill not a silo and things that happen in that one Hill um so Silo versus Hill right if something happens in a silo it stays in The Silo um that really gets important instant response when someone decides oh you know what we don't don't need those EDR logs because y we never use them so we'll just stop sending them to the Sim well for I that was very important but they weren't in our Silo they never

asked us because we never worked together whereas if you have a hill even if they start that process you'll be having a meeting and they'll go oh by the way our change for this week was we're disabling this what what you're doing what um this is a thing that's happened that I've had to deal with where someone in their little was like well I never use these I'll just get them turned off and didn't realize the IR team had 10 alerts that all relied on those things that just stopped working one day and we had to figure it out and we had to get it fixed whereas if we talked um and we've built really good relationships in our company

where we work with our engineering teams we work with te and we have weekly meetings that everybody goes hey I built these things here's the changes I've made just for informative nature just so everybody knows what's going on even within my own team I've had this where I have had a team member do a really cool thing but not talk to anyone about it and then another team member build another really cool thing and not talk to anyone about it and then we realize they both built the same thing we could have saved some effort there so just talk communicate we're all together we're working we're building together this is my last slide um we are

going relatively quick um I will be open to any kind of questions if we want any questions I said I want this to be collaborative and that includes this if yall have questions want examples I more than happy to share this one is honestly the most important one to me um because as I said I started as a sock analyst um these traffic lights are what a lot of socks have um there was a blog or a thing on dark reading literally three weeks ago that had the metrics that I think on the average sock has 3,200 alerts created a day and like 10 people to work those you're not doing that I can tell

you that's not getting done um there was the target breach way back in the day where in the postmortem they found that there had been a security alert that triggered on the initial access that never got looked at right if you have too much work you can't do anything and so we reduce we automate we enrich um and that applies to both the Technologies and to the people um it's a really good thing right let's reduce let's focus on that for now in alerting in inal response bace in the sock we need to reduce false positives false positives suck they take they drain you they make your hate your life they're one of the things that burn

you out and they make you make mistakes you don't look at things critically anymore because you've looked at the wrong thing a thousand times 10, times 100, times so we need to reduce those we need to figure out and this feeds back to that empower the employee way ear in my side right the people working the alerts who are seeing the thing should be able to feed back into that pipeline Loop this is all a loop everything goes back and forth and so they should have the ability to say hey either I personally or whoever isn't responsible I need a direct connection let's talk through how we can get rid of this and do that early on when you're already

getting 10,000 alerts a day you can't it's hard right like there's too much and you risk missing things out so you're too scared to do it it's better if you start early but whenever is better than nowhere or whenever is better than never right now is better than you tomorrow get it done get started that's automate in alerts automating is true positives right it's a similar idea true positives are cool they're the thing we're looking for but if you're seeing the same true positive over and over and over why are you still having to see that shouldn't you have done something this is going to be prevention this is building out rules in your EDR and your

antiviruses this is building out automations to react I'm building out a tool that says hey this IPS alert is known bad we know it never F triggers a false positive it's always a malicious scanner we're not vulnerable to it but can we just feed that into our firewall as a block and say they're obviously doing something bad I don't want to see anything else from them because that means now for that one thing you just automated you also blocked every other thing they about to try and so you could have saved you more true positives and it sacks focus on just getting rid um of that noise just lowering the counts and then enrich there's obviously not everything

is an obvious true positive not everything is an obvious false positive there is a pretty large amount of things we investigate because we don't know a way to do it there's no way to do it this is the living off the land alerts which I hate with the firing passion but we have to deal with them right it's the things using things that are legitimate for bad things and they're hard to do you can't just automate it or get rid of all the false positives because the bad guy looks the exact same as a good guy but can we enrich it can we add data before you even look at the tool or look at the

alert so that that we have the full picture going into it we don't have to log into 15 other tools and extract data and pull event logs and do this can we just have a script that goes hey solve this it had this IP address associated with its threat and tell feeds with this there is this hash that results from virus total and any run and all this are these there was this URL and this URL leads to this with a picture of this all in front of you that alert that could have taken 45 minutes or an hour now it takes 5 seconds because you can just go oh yeah obvious now escalate or cancel

it's rich and all of those each piece of that honestly can be done with automation tools there's no reason for a lot of this to be manual and that kind of goes to this next phase which is not in addition to learning let's do the same approach to the people let's apply it to our people in our soft skills let's reduce let's automate let's enrich with a little bit different perspective reduce the current work this feeds in this is the automation these are the things that you are doing the exact ex same set of mouse clicks every day 15 20 30 times a day each time you do this it's a two three minutes of your

day but you've done this 15 times today that's almost an hour worth of work that was done doing a thing that is the exact same process over and over let's automate it let's reduce that work that isn't actually adding to you and it can add up surprisingly quick um one of my favorite tools and things that I've done is I automated taking malicious indicators from fishing emails and adding them to tools they just go out throughout the environment and just take hey this email's bad added the tool this URL is bad added the tool it seems simple right we're just saving a few clicks here and there of logging into our web Gateway our email Gateway

uploading it to a list hitting save but you look and each time you do this you take five minutes well we're investigating 50 to 100 fishing emails a day that are true positives and malicious that 5 10 minutes times 100 is more than a full-time hour or full-time employees thing I have just saved a full-time employee and the best part is you can tell that to your Senior Management and you just saved money for the company and it's not often cyber saves money so same idea with reduced groundw work is automate the repetition these all feed in the same idea of getting rid of it yeah it is actually a custom application so we built it from the ground up um

it's written in Python um yeah so it is relatively modular modular so we can actually feed in tools and get rid of them as we migrate um and then it just maintains the list um can't do too much detail but yeah

um that's where you're going to really need to be on the front line with your people right what are they doing that they see the same thing over and over um we do a few different sets of reports and things um we have monthly meetings where we actually do a report that says this alert has this true positive rate this false false positive rate these numbers and we look at the one that's the biggest if it has 100% false positive and it triggered 100 times this month assign it to someone this is their thing they take an hour a day figure it

out depends on the the tool some are friendlier than others but yes usually pull down all different the which has

top

at's yeah exact yeah so there's a few ways to accomplish that depending on your tool for us is relatively nice in the fact that our case management tool does a lot of that automated correlation and so it becomes very easy to go oh hey this indicator is the exact same indicator in 100 other alerts and so you can just really quickly reference it um or just whatever your sim tool most of them allow you to aggregate by something um and that would be that part of that investigation that's saying hey this is obviously a noisy alert and the person who's going in is going to dig into it they're going to say okay let's look at

everything that triggered in the past month two months whatever and just put it at a table um

your for few days your a little bit of both um it depends on the tool um at the company I'm at I started when we were literally just building out the cyber security program so I got lucky that it was one tool at a time and we could just tune it as we went um if I had to go back and we had everything um yeah you pick one that you need to focus on um look for things that vendors have as their High Fidelity ones most vendors have those where these are the alerts they know are usually better you start there um but be quick umy there it's usually embedded in some things so it depends on the vendor um a

few of the IPS vendors will also it won't be to straight up hey this is obvious F positive but the severity will match with also like an impact metric or some other metric they have and you can start correlating them um what we what we have done is take a relatively big Baseline of the set of alerts which are like Mau alerts or C2 alerts we just assume those are true up front and turn them on in full block um it has caused issues before but it's also prevented issues but then we keep up we keep looking at it um um and you're not going to be able to just take it and get it

right the first time um our alert Fidelity is very much a sine wave it's up and down up and down because what you do is you notice you're doing more false positive work than you would like and so you started signing out those rules let's look at the noisy ones let's get them down well now we've gotten rid of our false positives but we have more time every day we've got more time to do things let's go build more well those new ones are also adding more alerts and those are false positives and so your alert goes up um it's hard if you are coming in and you're already getting 10,000 alerts it like if you just going straight into it

and it's not tuned at all you just chip away at it um if you're at a point where you've already got it relatively stable you can do it in those batches you can go in get this new tool let it run for a month or two probably don't create cases with the data up front if you can um but then do do the same metrics you're talking about right like let's get this list let's aggregate by the signature ID and also get all the data points and look for patterns um however you want to do that some people like doing it with the maps your various threat maps and Intel Maps where you have everything and things I hate those

uh I do really good by just putting it all in a table and relying on my eyes go hey those look similar um but however it works

have yeah I personally am really quick to throw stuff in the python other people within the inst response team are not as quick to do that yes I am talking about you Zach whatever man I'll do either first before yeah I mean it's going to take it's going to be work it's going to be finding what works for you and your team and what skills you have everybody has different skills um a diverse set of skills is really useful on instant response because we touch everything but that usually means people do things very differently um the way I accomplish things um I can personally say I don't work many alerts anymore and I think everybody else in is response team is

happy with that because my documentation is not the greatest it is every important point is documented but it doesn't read very well um you're

Goode yeah not not I am in a really really lucky spot of having a team of software developers in response most companies don't have that um yeah and the way I got it though was by being instant response and just being able to code python um but we just realized enough value over time that we dedicated more people to it uh and the way we got more people in more time was by doing these use cases of that one with blocking indicators being able to say hey I spent two or three weeks doing this piece of code and that's saving us a full employees worth of time every week when you do that that math it's

like oh yeah that's obviously a good deal right like that's a significant amount amount of money improvements um but those are usually accomplished in small pieces it's small pieces that over and over and over just build up um but also having time dedicated if your people people are literally working alerts from the day they come in the moment they come in the moment they come out check out you're doing it wrong um there's too much what

those yeah it's my boss by the way yeah yeah that is actually my favorite thing is when vendors come in with their automation tool like hey we know most people are looking at 4,000 alerts a day we can get you down to 300 we're at 50 a day with automated responses for the remaining 100,000 but okay sounds good um and it's it is good it's good to focus on that

we do both um we have in actually almost all of our case documentation all of our um playbooks case templats whatever you call them in your tool is actually a final step of tuning which is do you see an obvious way to tune this um and you don't always but if you've done it that's the opportunity to go hey no I think I can get this done in two hours Let's Do It um but then there's also a lot that you get where you come to the end of the month and we take set it back down and go okay bigger picture you didn't get any quick wins but I do see this one is a 98% false positive and

it's triggered 40 times that's what like 36 35 false positive alerts what was false positives in those and can we do that filter so we do both um there is the day-to-day case management and that's part of that empowering the employee letting the people investigate alerts have the ability to do that tuning up front um but it's actually even in the weekly um our team most of the alerting is built by the insurance Response Team um we build and maintain our own tooling and our own alerting so we're already in the tools we're already doing it um which gives us that extra bit though right we've investigated we've done that extra enrichment piece of I have this alert that I know is

always false positive and I always go to this tool to figure out why well we know what to look for we know which enrichment piece we can get and now automate the tool um yeah it's both it it really kind of does it um in the day-to-day your false posit rate kind of flickers um but then you look at the bigger picture it's just whatever we're focusing on um it's usually like a two or three month thing where we'll spend two or three months uh really tuning it getting it nailed down before we now have that extra time in the the day to go build the new stuffff and then we fill up our queue again and

then we work through it um yeah and it it's flexible go Z

[Music]

um yeah so I mean it Stacks into it like it it is a very important part because it does feedback um it's going to depend on the way your team is built and how much pamper you have um with us we're special because r r team is the detection engineering team so it does feed into that part um we do keep a pretty Dynamic approach of we're always everybody in our team is doing a research we're doing things constantly we're seeing as the worst one I hate because it was a very long weekend log for Shell coming out like we see that bubbling up through all the communities going I now know what my weekend is

dedicated to um but when you have those really high visibility ones that's where you drop two three people out of the day-to-day that is almost always not a big deal and say hey you've got to figure it out um in that example it took us four hours to have a d a smart alert that was actually extracting the C2 and callback domains de office gating them adding them to a lookout and then checking other logs to see if we had a successful back to the IPS but that was very dedicated work where I wasn't doing anything else um more day-to-day stuff just you keep up you see what it is um I personally like everybody every engineer to have at

least one forward-looking project and that could be doing more alerting it could be they really like to create new detections they're doing research they're looking at oh OCT is popping up more and more in the um incident area right we're seeing more more SSO type attacks let's dedicate and two or three hours a day at most but usually like an hour or two a day um or a day that you spend four or five hours on it a week let's dedicate some time let's just build it um other teams have full straight up detection engineering teams I don't like having those separated from IR personally because you build things that aren't useful to the IR

team um been in that boat where people keep building things saying hey you really need to look at this and then instant response looks at it and goes why it's absolutely useless and has no data in it that I can use it's nice to be there it's nice to touch it um my team of software developers I make them work alerts I make them work cases because to make things better you've got to understand what you're doing um but if you do have the ability and the manpower to separate it out then you get people that are dedicated full-time to it um we have a few people on like our threat intelligence team that that is what they essentially do

all day every day um but there's also a lot of that where it's us going to them saying hey your thing sucks please fix it so you know it goes both ways um yeah I mean that kind of question

um yep uh uh yeah we're actually we have a manual review process that where you go through and approve it um we actually have all of our Logics stored in a git repo that is about to be production automated to deploy to our Sim um it's been a long process of a lot of going through those silos we've talked about um yeah uh one of those steps does require approval because before that we have been in situations where someone tuned an alert and left an extra bracket and it wasn't until two months later that we're like hey why haven't we seen this Alert in a while um it happens um so it's easier to have that extra approval step of that we

don't make it like very hard to do though right like it's not going to be you have to wait till a formal meeting um we just make sure at least one or two other people reviewed it before you deploy it um with a little bit of flexibility for people like on call who get alerts where they get like a thousand alerts that are a random tune that happen in the environment you could tune that out real quick but you have to follow it up with the official process um yeah I definitely advocate for peer review um that goes back to that collaboration step I've done a lot of cool things I've done a lot of stupid things um I've

definitely prevented the business from running before um but you learned but I learned I didn't do that same thing twice um but no I as as a lead I've got very a wide range of people under me from people who are literally fresh out of college to people that have been doing it for 30 years I still have them peer review my code they have to approve my code even though I'm the lead even though I'm the one who's been around forever because there have been more than a few times that those people even the people that have only been working for two months and just got out of college be like but why did you do it

that way I just wasn't really thinking but maybe you're right maybe it would be better a different way um peer rview is important but don't make it to road block your red tape um keep it collaborative keep it friendly you're doing things to build um pretty much my entire deck has been everything is the same thing it's all about working together building each other up and that's the same thing piew is not a way of catching people who made mistakes it is a way of making sure everybody builds better

most yeah it's how many times M people here how many times I've been in Emily or Zach Cube over here because we're all working on something going well what if we do it this way no no no we don't have those specific logs but we do have these logs over here okay yeah yeah and we could add this together um yeah all of our coolest stuff has been built as a team um my team

Zach's pointing at me currently mind thanks for listening to me yeah that part about me being very happy that I don't have to actually answer questions anymore I've kind of become that guy now where they'll come to be like hey so I'm looking at this thing and I'm really working through this part and this is what I'm saying oh you walk like you're welcome I worked really hard on that one um I realized I for missed one of my bullet points um one of my big things and we've kind of talked about here is false positive true positive um a really dangerous place to be um is focused on true positives and false negatives over false positives and what

that means is you are insistent that you have to see every bit of bad traffic you have to catch everybody and everything what that does is that opens you up to that false positive slute because everything you catch is also catching two or three other things or 10 things or 100 things or a thousand things and sometimes it is actually better and more secure to allow false negatives to allow you to actually miss a thing if that comes at the cost or at the at the benefit of getting rid of 15,000 alerts right because guess what if you have 15,000 alerts in your queue that one true positive is not getting investigated you're not going to see

it so sometimes you do sometimes you overtune your alert and then you need to come in but then you do this defense in depth thing that's everybody talks about right let's have two or three different tools different alerts that stack and that one that slipped through will get c three other places yeah so

I like the idea of risk based alerting I haven't seen a lot of people that have truly done it to the level that it could do um for reference I'm actually a statistics guy that was my focus in school with Statistics I love data science I love doing those numbers and it's been really good for tuning alerts and being able to see patterns risk-based alerting is really really hard because not everything is always the same risk um one activity done one specific way is one thing but doing it a different way is actually a different risk and adding to the ghetto is hard um it's also a whole lot more tuning you have to have people that are

dedicated to making sure your scoring is always floating and maneuvering and it's going to change this false positive Alert in Q a really high false positive should have a lower risk um but when you tune that alert well now it should be a higher risk because we just Don it but it doesn't usually get tuned if you can do it and you can dedicate the time to it it's really useful I like to do two approaches a lot of really high fidelity alerts and then low Fidelity alerts that you can build into a bigger picture cor perspec

RIS yeah you've got to start tying a bunch of things together to get the value out of it umats build

hard yeah yeah I mean that's if you get into it and having like a a really good data science team is going to be really good um a lot of those things can be done with relatively simple alerts by just tuning your alerts up front um so there are a lot of those alerts that become really valuable there that I've seen become really valuable just by figuring out what's always triggering it to be bad and excluding that very specific piece and some of those alerts that would have been noisy and would make really good risk space actually become decent alerts but we do also have our own uba tool right we've got that too um

I'll be honest that it doesn't get used nearly as much as a lot of other alerting because we usually catch things with other alerts before the risk-based score becomes an issue um but we have a lot people that have been dedicated to building out the things over the

years it's a little bit of both um it does it does risk scoring but then theom anomaly detection also creates the dynamic

risks prob yeah it I said I'm a data science guy I've seen really good and I've also seen them not get done quite correctly um but it does take a lot of time in investment and we just found that focusing more on that day-to-day alerting and tuning and adding and creating new detections for St stuff has given us more value than trying to tune these big data models for 90% of the stuff out there and most attacks or commodity right there if you look at pure statistics the chances are you will never be attacked by AP unless you work at a very very important location who you're defending against are those low hanging fruits um I'm done so I got to wrap up

but yeah it's you're hanging against commodity you're hanging against the people that are knocking on your door seeing if your door knobs are open right those are who you're defending against nine times out of 10 um so focus on those first but I'm done if anybody has questions I'll be around