The No BS SOC Story by Mark Simos

[Music] thank you so now I get to add bsides to that list so yay um so this is the nobs sock um for some reason people seem to like the uh sort of edgy kind of push the limits uh slightly rude uh titles my I have a talk at RSA in about a month that is called you're doing it wrong about security antipatterns and they took that one I'm like okay I'm starting to see the pattern here I have to get a little bit more rude so here we go let's have some fun so I'll start out with a little bit of who is this dude you heard a little bit in the in the intro thank you um and

where does this content come from just to kind of give you some context of where I pulled this from um which doesn't sound good when I go to the next line of where does the sock BS come from um so we'll talk about that and sort of some of the Challen alling uh information confusing information sort of what are the sources of that and why do you see that and then get into the anti-patterns we see so these are the common mistakes we see that security operations folks are making often times without knowing it um one of the things that we highlight very strongly in that RSA talk is be kind to yourself and to

others this is not a place where there was a magic book that everybody should have read in ninth grade that explained how to do security right anybody get that book nobody got that book um we are all figuring this out as we go it's hard it's complex it's constantly changing so be kind to each other be kind to yourselves it is very very hard so these anti- patterns are there to help you understand the common mistakes they it's part of that we saw the OD to cyber security before we started it's part of this really being a love letter in terms of hey this is what we are doing wrong um and then what does good look like

let's flip it to the solution side talk about the uh the challenges that we face um being very honest and direct about those but kind and then how is AI changing SE Ops CU everybody asks it so I always put in my decks and then the story of a sock and I will do my very best not to actually sing that song I even came up with alternate parody lyrics um I'm got to stop myself right now and then a call to action uh to to stay focused on what matters and for reference um because I will be talking uh the Microsoft seoc or Security operation Center um is the primary source for a lot of this

information it's a very highly mature organization um there have been some recent incidents against Microsoft um because of the way um regulations are written and being enforced today um I cannot tell you anything even if I knew it and and I'm not saying I do or don't um that is not already public um because we have to do that fairly with all investors so just so you know I cannot answer questions on that um because there's no point you can just go read the stuff yourself so just as a heads up on that so let's get started um I do Microsoft work I do Open Standards work for all to read that is completely vendor neutral abstract the

Zer trust work that we do at the open group is focused on um defining what good looks like for security sort of that missing book that we wish we had in ninth grade kind of thing I mentioned earlier um starting with capabilities everybody in business knows that you have to be able to take money and you have to build a product you have to ship a product they Define those capabilities and it changes over time do you ship it by sea by land Etc do you manufacture it here do you Outsource it those capabilities remain the same even if how you do it does different security needs the same thing and so that's really what

we're focusing on very heavily is how do you uh what are those capabilities which are defined in the reference model you know what are the rules for doing zero trust and good security well the zero trust Commandments and then um what are the architecture building blocks Etc that map to and so that's the work that we're doing right now so that's that's what I do there and then because I was really kind of bored and I needed something else to do um I decided to go ahead and write a Playbook which became a Playbook series and so um when you uh when you take the fact that there's about 65 or 70 roles in security and

then you multiply that by we found out it takes about a chapter not a section to describe each one um it got long and we weren't going to give you a 7,000 page book so it's a series the first one's out so those are the the the the the things that I draw from those different perspectives on the very role-based the capability based and Technology architecture based views and uh for those that haven't seen it marks list I post all the public stuff that I do as well as other good stuff I refer to people all the time so security industry has a few buzzwords any of these sound familiar and Microsoft of course contributes there in the top left corner

with our own terminology um which hopefully will stay stable I think it will and we won't have to do anymore product name changes it's I think we're in a sustainable system I'm hoping we actually sustain it I am not in marketing so I can't make a guarantee um but ultimately a lot of this material is drawn from the security adoption framework or saf um which one of the missions is to put all this in context and organize it and make it make sense so that is some of the Microsoft work that I do is that security adoption framework um and this is something that is actually deliverable through Microsoft unified you all know what that

is you know kind of replace Microsoft Premiere um that's the organization I work in at Microsoft we built end to end strategy end to end technical architecture and they're the process of building all of the the initiatives and how to modernize them the plans the architectures everything to get people to modernize in a very regular measurable way and the material today is drawn from the security operations or sck one we have both a short and long version of that a couple hours and a couple days um depending on how deep folks want to go in and then we're working on building that for the rest of them we have most of the short stuff done and we're working on the long

stuff after that so that's the context of where these slides come from let's talk about the BS so do a lot of research spend a lot of time paying attention to what's going out in Industry what are the recommendations Etc because good ideas can come from anywhere the thing that we see in most of the guidance is it falls into one of these four buckets sometimes it's too high level it's great it's accurate it's correct but it's not actionable you can't do anything with it it's a high level principle sometimes it's too lowlevel this is exactly what this particular nation state did and here's the ioc's hashes the IPS of this that the behaviors but okay how does

that fit into an overall security operations program sometimes we see vendor biased I don't think I have to explain that one um and sometimes we really see uh Bine secretions um sorry that's the BS thing outdated or just plain wrong and so we see a mix of that oftentimes in there though and this is sort of the mixed bag is there are gems in there there are nuggets of wisdom and beautiful insights that people have that just get caught up in that broken context and so we do our best to pull all that together now why do we end up with this two things we tend to see Silver Bullet thinking how many folks ever heard a vendor say if you do this

it will stop these attacks okay how many of you have seen vendors that are smart enough to not say that exactly but imply it very strongly also seeing hands um what we see this thing has really gone underground so the Silver Bullet thinking hasn't gone away but we we do see a lot of sort of implicitly believing in absolute claims obviously most people can spot the BS on an on an obvious one but it's starting to get into things where hey I'm not going to do that because it won't block all the attacks well you're sitting at 0% Effectiveness why are you complaining about 80% so we do see this sneak into people's thinking and so it's something

you have to keep an eye on um RSA talk goes deeper into that but um and lack of life cycle thinking we also see one and done we're going to solve it once we put it in the script and we never have to look at it again and this is sort of the lack of governance and and those things so it does rear its head in a couple places other one that's very very prevalent in security operations is technology Centric thinking because these attacks happen on the technical environment we assume that we need to block them technically attacks are held or are um are conducted by people people who have a goal and a plan and a way to profit

from it whether it's a raise from their boss or making money directly as sort of a solo contractor our adversaries have a plan the question is do you and that's the real question we have to answer here and so we've got to look beyond the technology and look at the people understand there's burnout understand people need to collaborate across teams for it to work understand that people need training if all you've ever learned is network and IP addresses and subnets and all that stuff and the attacker is using pass the hash or some other identity attack that experience and knowledge isn't going to carry you to where you need to go so we have to

remember that this is a people Centric thing and you'll see that show up in all the diagrams that we're going to be seeing um and of course you can't solve a people process problem with tools if you have a problem where someone calls up and says hey I'm the CFO and they have this nice fake you know AI generated voice and I need you to approve this thing is there a technology solution to that problem it's a people and process problem that you need to go ahead and have an outof band check and all these things you need a process solution for things like business email compromise and um executive whaling you cannot expect tools to solve every

problem they will help they automate processes they make it so that they can guide people with less training to do a task in a simpler way but they don't magically solve problems these are people and process problems that technology automates and of course money money accelerates both of these factors which accelerates all of the pile make sense all right let's move on start talking solution side what is the mission of Security operation very simply it's about reducing risk what is risk risk is bad things happening and doing your best to keep them from happening or managing it well if and when they do happen the same is true of hurricanes Financial crises everything all the bad

stuff is a risk discipline security is sometimes an enabler and in the mindset of having an enablement hey how do I help the business succeed is very important but at the end of the day it's our job to keep the bad stuff from happening and so the way we do that in security Operations Security operations is right of bang the bad thing has already happened even threat hunting which is a proactive version of security operations is all about finding a bad thing that already happened that we missed so everything in security operations is Right a bang and by the way there is a missing sister or cousin or brother whatever you want to call it

operational function left a bank before things happen called posture management often times it's just vul management it's it's governance team some people are stepping in but there is a need for a corresponding operational function on the left side if your sock is doing vul management they're doing two different operational functions at once two very different focuses two very different Dynamics working with some of the same teams but in a very different way to make sure patches and configurations and everything are right versus dealing with an active adversary two very different skill sets make sense so the mission on the right of bang side is time if I happen to know my adversaries if I happen to have great tracking

across the 10 or 12 or 30 adversaries that could be art targeting our organization then I can absolutely get more specific and more detailed with my sock goals if I don't know that if I don't have that level of threat intelligence if that's two five 10 20 years in my organization's future what I can focus on is time in two and a half minutes an adversary can do a lot less damage than two and a half hours two and a half days weeks two and a half months or even two and a half years that they sometimes get do not give them time to operate time to hide time to get to their objectives time to burrow time to

resell it to somebody else burn their time down find them kick them out now that sounds like speed is the thing right accur accuracy impact and speed you ever seen a kid try to do something fast that doesn't quite know what they're doing the outcome isn't always great if it's just about speed there's you want to make sure you got accuracy impact and speed you're actually aiming at the right thing you're doing it quickly and you're doing it effectively make sense so that is the mission dwell time mttr so let's go to the the metrics uh I'll bring them all up the metrics mean time to re mediate that's the important r that is the adversary no longer has

access the clock starts ticking as soon as they get access to your assets and of course they will grow it and the clock stops when they no longer have access that is the time period that is most important there are a lot of other metrics a lot of other things that are interesting and useful that help tell you that you're doing something healthy in a good way but the amount of time they have access to your assets is something you can can talk to and explain to a business leader and they will understand because they understand hey we have a thief in the building we're going to try and knock them out in as few minutes as possible as few seconds

as possible CEO can get that and you don't have to explain all the rest of the stuff hey this is complicated stuff we do if you're interested I'll tell you more but you can communicate that up and it relates to them and it makes sense and it actually reduces risk makees sense one of the other key uh measurements is mtta meantime to acknowledge when does an analyst start working on it how long has a little blinky light been blinking before someone does work because that can tell you you're overstaffed you're generating a lot of false positives it can tell you a lot of different things it depends on your sock and what's broken but the

longer that thing is sitting there blinking the more time the adversary has to play because no no one's even trying to chase them make sense so these tend to be the two most important ones we'll talk about more in just a moment culture eats strategy for breakfast you have to have a strong culture that has these elements Mission aligned generally people don't go and look at a bunch of technical alerts unless they give a sorry unless they care um so that one's pretty much almost automatic in most security operations roles it is not something people do for the fun of it or because it's an easy job um generally they have a caring for keeping the organization safe that's

almost a given insecurity but but the important ones are continuous learning what you know today is not going to be enough for tomorrow you have to be continuously learning whether it's identity Network learning about attacks learning about better ways to do things that is critical and then teamwork collaboration matters if the sock is trying to be the solo hero or there is a person in your security operations that is trying to be the hero and doesn't share what they know and doesn't talk about how they do it and doesn't want to write it down for someone else to do it that person is poison you have to be sharing and collaborating not only within security operations because this

is literally changing by the day by the minute the adversaries if they do the same thing that they did yesterday and cheap and easy blockers stop them they're not going to be good adversaries and they're going to go find a different living they have to change they have to find a way around that is the only way for the adversaries to succeed and so there is always continuous changes you have to learn and you have to work together as a team and share what you know with your teammates and learn from them that is the only way this works and that is not restricted to the sock or security operations that is a cross to it and Architects and Engineers

do you want to investigate the same exact incident as yesterday because they didn't block it anyone ever complain that you're investigating the same incident as yesterday because nobody blocked it yeah I'm seeing some hands okay got some honesty there got to communicate with those folks and help them understand listen we're seeing this this is a report this is what we did this is our investigation share with them buy them Pizza have the managers buy them Pizza is even better but go ahead and work with them and help them understand informally and then later you can formalize it as a thread intelligence discipline thread intelligence is not just data it is a discipline of sharing information make

sense all right so what matters for success as we said it's about the time minutes matter that is the overriding thing but you cannot do that in isolation people are the ones who do it and express their human judgment this is a people Centric thing this is red versus blue this is spy versus spy this is human mind versus human mind they're no longer fighting with their fists and their bodies or their Spears or their tanks or their planes they're fighting in a technical environment to defeat each other this is the conflict this is just Conflict Management you know a board game is also a domain of human conflict so the people matter the most but they need to be

doing it in a process because if the people don't write down what they do so they can learn and get better at it the next day they're going to quickly find themselves losing technology matters because it automates those processes enables those things and allows you to see data and insights that you wouldn't otherwise see accelerating those processes taking the annoying burnout manual ones off of your plate and intelligence matters and we're not talking about IQ here we're just talking about data and context on what are the adversaries doing what have we seen and learned from our attacks and what have others seen and learned from around the world and our peers and our industry competitors Etc how are we

bringing that into this and as I said before teamwork matters because none of us is as smart as all of us and that is extremely true in the sock where we simply do not have time to keep up we have to be learning from each other leaning on each other and the zero trust Playbook as we're going through it one of the things I'm driving for is it doesn't matter that one person can't possibly know an entire technical estate of containers and malware and this that and the other you have to have that in aggregate in your team so somebody knows containers somebody knows the server somebody knows Linux somebody knows Windows somebody knows endpoints somebody knows identity

somebody knows email and work as a team on it because that's the only way you're going to keep up with the massively complex technical Estates we have make sense all right so let's have a little fun with anti-patterns shall we so we see some of these same anti- patterns playing out over and over and this is almost and that song is coming out again the story of a sock um I'll probably end up singing it by the end um a couple of common mistakes that we see over and over again we all know that we need data we need visibility to be able to see the attackers right if we're blind we get we don't see the attackers because we don't

have have that data coming in but often times we stop at the first stage which is we need the data and it's like great so you have all of the data and you have all of this information what detections have you built off of it and how how high of a quality are they well we have the data that doesn't mean you actually know that the attacker's in there you haven't found those 12 or 15 or 200 events out of the 10 million that are actually relevant and brought that to someone's attention start an investigation and then kick them out we see a lot of this and go back going back to money are Sim

vendors paid on how much risk you have or are Sim vendors paid on how much data you're storing you think their sales people are going to worry about it if you're storing all the data or they're going to move on to the next customer buy their BMW buy their BMW that's a pretty good choice yeah but ultimately you have to think about this in an outcome Centric way we also see a lot of security folks nothing wrong with this started as networking folks what was some of the First Security technology we saw firewalls and what team did they assign it to when there wasn't a security team yet the networking team and when and they where did they add the

IDS I IPS when those came out networking and probably the dlps as well and they might have fought it over if they actually have a data team right but that's the kind of stuff that happens we have that history right and that history influenc the culture of security and we have a lot of folks with those deep skills the attackers have gone far beyond scan and exploit over the network they have gotten deep into end points they've gotten deep into identity applications you see some of the nation state stuff today it's all cloud-based stuff based on identities they're very sophisticated they've learned the ins and outs of those systems we have to get past that original Foundation it's

necessary to understand networking but not sufficient to do what we do today so network is the only source of truth is a common anti-pattern not invented here we didn't have great tools again we went from collections not detection we had all this stuff here and how do we get something out of it well we had to write our own queries we had to do our own stuff and when you create this thing and you spend hours and days and months and years of your career doing it do you like it or hate it usually both but you have this sort of hey we did all this stuff and then you know all of a sudden there's a market for it and

vendors are putting out good tools and detections and we see a lot of resistance to using an off-the-shelf thing because it can't be as good as this custom thing that we created um we see a little bit of obsession on the shiny object syndrome of going after the cool sexy top end stuff without doing hygiene which is just common across security um usually after the first breach people finally get the budget they've been asking for they get the EDR the xdr the Sim whatever tool that they wanted and they try to solve every problem with that one tool that they got budgeted for right sound familiar and so you have this one tool to rule them all

one to rule the allall kind of thing going on that was my best attempt at I have saw in there um and then uh and then often times following breaches sometimes or more budget comes along you get a whole bunch of tools um and you end up with a tool of paloa um the one not I haven't cracked the one problem I haven't been able to solve is when people are sort of getting in this point where they finally get the budget for all this and they think that they can solve it by throwing even more awesome cool tools at it that aren't built to work with each other um listen you really want the tools that work together

um yeah that's the one thing I haven't solved because everybody wants all the cool tools and the the far right of the magic quadrant the far right of the Forester weave have all that and like listen you know it it kind of sucks to have to make all these things work together getting people to imagine that scenario when they've ever had tools I still haven't solved that I have not been able to get people to time travel into that like this is it really hurts but anyway these are the things that we usually see go wrong all of these are symptomatic of a technical Centric bottomup approach that I mentioned earlier the solution for this start at the top

what is our mission what are the metrics what are we trying to achieve what teams and functions and Specialties and expertise do I need what are the processes that those teams need to work together then you get to the tools and technology and architecture to support and enable those and then to the training and skills and enablement to make sure people are successful on those processes and tools to achieve the mission and then Automation and data constantly automating St stuff and making things better and managing the data that you have to make sure that those aren't getting in your way top down make sense so here are the metrics these are directly from Microsoft internal

security operations team this is the um we're in the process of merging a bunch of them together but this is from the ones that look most like our customers this is our essentially it environment um that uh that particular security operations Department mttr mtta are top of it and by the way this is also what we recommend to our customers so it's both a case study and a recommendation um we haven't seen any reason not to use this uh at most organizations uh case loads we want to keep an eye on how much volume is actually flowing through um so we know what each team is doing and specifically we want to understand how many of those

are being automated as well so we want to understand how well our automation is working um there's always this interesting tension of you want to have a human kind of double check the automation but you also don't want to have them burn down so there's always this interesting tension of your especially triage your tier one folks on how much you want them to touch those automated cases versus not so there there's no magic solution there that we found so far um but you um uh detection Fidelity this one's super important if you are doing custom detections if you're just taking them off the xdr um and what's recommended and working the top of the stack down like a like a

Defender xdr would provide for you you don't have to worry about this as much but if you're doing custom detections either in your xdr in your sim or some other uh detection you want to make sure that you're not putting a bunch of false positives in front of people that are burning them down that's a very bad bad bad thing because you disconnect them from the mission you burn out the few people that you can get and you don't actually reduce risk you're just basically wasting time and quite frankly people which is you know not a good thing at all um if you do have your own custom tools um watching the uptime is important and then keeping an eye on how

are we resolving it so we understand how many true positives how many false positives we're dealing with in Aggregate and uh this is actually the ratio and proportion uh from uh that Microsoft sock we actually end up with a lot of benign positives the technical detection was correct but it wasn't actually malicious we're a security company we have a lot of salespeople we have a lot of field support people we have a lot of researchers and sometimes they click on mimic cats on the wrong machine stuff happens so we end up having a lot of these benign positives probably more than most organizations do but we do see that the most then we see true positives false positives and a

whole bunch of duplicates merged and all those kind of things kind of bring up the rear can I ask you yes this is alled on instrument andly deployed and all the sources are reporting in properly how do you validate that so the question was uh or the comment was um this is all predicated on all the information and data being um uh being correct and the systems being correct and accurate and that that is an absolute correct statement um I in terms of like how do you measure whether the quality is good or not that all that all sources are reporting in you're getting all the log sources yes you're supposed to yeah that's that's so the you know

that all the log sources are being reporting in ETC yes there is a governance element to it in addition to this this is more of the outcome and the risk management elements so you are absolutely correct on that thank you good point and of course you always want to watch Trends you know what are the adversaries doing where where the Investments that we're doing are they having an effect you know we put this in in May is it lowering or increasing anything are we seeing um because SEC Ops the other thing is is um you see this red box shows up anytime there's uh metrics there very very important thing about metrics SE Ops is not in control of its

Destiny it is a response Force it deals with whatever the adversaries are throwing at us this day there may be more in a given month there may be less in a given month there maybe better tooling that just the adversaries got to um for example in 2008 to1 time frame we saw the spike that has led to this room being filled with cyber security folks today because what used to be um a very small number of nation states with a big R&D budget that could do credential theft and pass the hash became an open source well an open openly downloadable tool then later open source mimic cats that allows anyone to do it and we saw

second and third tier nation states and criminals eventually pick it up with a ransomware gangs and so what's happening in the world is affected and what the sock sees is affected by what's happening in the world so it's really important not to punish people if the metrics go up or down because we control some of it but we not control all of it we have to recognize that we are just dealing with whatever the world is throwing at us so kind of along that line of the the history thing you do have to ruthlessly prioritize there's a little bit of History um I should have just gone forward to the next slide sorry about that

um in the old days sort of pre 2008 you had commodity attacks that everybody kind of got used to and AV mostly handled and then you had antivirus antimalware and then uh Advanced attacks that were the the domain of a very few uh nation states that were well funded as those tools and others started to become open source and more widely available to attackers we started to see more and more advanced attacks coming from more and more countries more and more attack groups um and so we did see that um bump up in priority for more organizations and then the ransomware thing came in two different waves one was the first basic one at a time one uh

one endpoint at a time almost nuisance ransomware that we were able to sort of get under control fairly quickly um very consumer oriented but then shifted to the Enterprise because they realized there was money there the big big big big shift was after the not Peta or Peta depending on how you want to name it um uh destructive operation by a nation state that masqueraded as a Mass ransomware Attack shortly after the wac Crypt that sort of gave rise and I Pro and I think probably inspired the ransomware groups to start using those credential theft techniques in their tax and using human operators as opposed to a just automated script and run and see what comes back and that changed the

game and that's where we got into big game rant some where where they started targeting organizations and using some of those pass the hash pass the ticket Etc attacks that allowed them to take an entire Enterprise down and most socks should be focusing on ransomware extortion as their top priority because of that if they aren't already and then there there was an interesting thing that happened on the way to the play which is commodi gangs realized they could resell the access they had to Any Given organization to those ransomware gangs commodity gangs don't want to take the huge law enforcement risks they're just trying to stay under the radar so they get 10 grand 25 Grand whatever it is to

sell access to any given company it varies um uh it's very much a online market and the prices go up and down um and then the ransom more gangs try to turn those tens of thousands into millions and so you saw this collaboration cooperation among the attackers that all the stuff that you could ignore for the entire history of cyber security as low grade all of a sudden bot Nets and whatnot became a very urgent priority often times more important than the nation states make sense so you got to keep in mind the world changes on you so people there's a old uh Jimmy Buffett quote uh uh relationships we all want them we all got them what do we do with them

sorry I was just in Florida it's a Jimmy Buffett thing anyway so the passion that people have especially in security operations to go and Chase the bad guys in real time can also be the very thing that causes us to burn out and we have to watch this very carefully and it doesn't just affect analyst it affects managers as well we have to take care of ourselves we have to take care of our people because the last thing we want is little gray pile of Ash instead of an actual full person that is not doing anybody any favors that person the company anybody it does not it's not a good thing that let people burn

out why do people burn out a couple different things this is basically a psychology slide with a couple of cybercity words thrown on it just so you know this is this is General burnout but their mission and what they connect to which is protecting the organization if taken too far can lead to exhaustion and can also be not recognized um and which leads to the burnout and the lack of connection to the mission that originally started it you've got to make sure people rest even if they don't want to that they are learning they're doing self-care you know send them to conferences whatever make sure that people have time to breathe and think and document and celebrate the wins make

sure they get recognition for the hard work they put in not every time not every attack you're going to get a W sometimes you get an L right and you got to recognize that you got to support people whether they win or not that they're putting that effort in and make sure you're rewarding that learning and all those other kind of good things that they're doing doing other people's job sucks anyone like doing other people's jobs and not getting paid or recognized for it I don't see any hands odd you have to make sure that the sock isn't doing a whole bunch of jobs like you know implementing maintaining tools when you're already a security analyst that's

rough you know writing up a report analyzing your architecture managing and coordinating incidents that's different than investig them researching the attacks and other questions like as you have time and resources to be able to do it let people specialize in these things so the analyst can do what they are best at and the one that really really sucks is vulnerability management because that's a very different discipline it's a very it operations oriented um and infrastructure operations oriented thing and that is the posture management thing that a lot of us have been missing but the sock gets slapped with that because hey you're operational so you do this so so that's that's one of the big uh

mistakes we see wasting effort on false positives and repetitive tasks is also a source of burnout this is where vendors can and should be trying to help you with technology getting rid of some of those lowquality detections um which can require hunting we'll talk about that in the story of a sock um automated Advanced analytics helps and then integrating thread intelligence so you don't have to do manual copy paste and looking this up to see if this is well known all of those things help that's where technology does help you know not much technology in the rest of this though and even in this space you can't do everything you have to prioritize what are the cleanest

alerts what are the most urgent alerts what are the assets that we have to worry about the most spend your time on that and don't make people expect that they're going to get through a thousand alerts in a queue in a day that is not the right way to do it you have to prioritize nobody want to do that to themselves why would you do it to someone else make sense so AI can't have a talk about anything without AI nowadays right so role of artificial intelligence already being used been being used y'all heard of machine learning mathematical probability algorithms being applied to a lot of data one of two ways it's either I have an expert I had them look

at a sample data set and it told the machine what to do it tagged it and then we run it over a whole bunch of data sets and data streams from there on out that is supervised machine learning other way of doing it is here's a raw data set we don't have anything to do with it but let's throw an ml algorithm over it see what it finds for cap uh clusters and patterns and those kind of things and then give it to an expert to look at and see if those are interesting or if they're just a coincidence make sense that's what ml is that's what it's been doing for a long time it's been

baked into xdr is kind of the foundation of some of the Innovations in that space and it's being applied to sim and some others now so AI that form of it has already been in use for a long time and it is very very very useful um think of it as the opposite of that uh that old um really terrible but good sounding advice oh you just need to look at the logs anyone ever get that advice anyone likeed the person that gave them that advice didn't think so so this is uh helping get us out of that pain but still get the insight and the value that was desired so dive into this in a little bit more

detail in just a moment but it changes how security operations works and learns it changes the interface the way that people interact with technology and I'll zoom into that slide in just a moment it fundamentally changes that and makes it more natural language we're still in the v1s of it but the ability for that to generally understand our intent and then be able to do something about that is very very powerful it's very dangerous but it's also very powerful on the good side like any tool you know blades nuclear weapons Etc not nuclear weapons nuclear power which could be nuclear weapons or you sorry that just that went wrong but anyway it's a double-edged sword we'll stop there and shut up all

right um and then uh we'll talk about the scenarios in security operations where we are seeing value as Microsoft and that we're investing in because these are the ones that are the most ready first there's a lot of things you can do we've done a lot of learning with our security co-pilot um through the Early Access program and some other things that has given us a lot of insights into what it is really good at and what is going to take a little bit longer and this is what we found it is really good at at first and so we'll talk about those in just a moment so interfaces I'm not going to do the Jimmy

Buffett joke again anyone ever anyone here in the era of Punch Cards I'm just kind of curious wow okay um I'm not the oldest person in the room clearly so I never did Punch Cards I started back at an IBM PC was my first computer in the early days the only way you could make a computer do anything useful was you had to program it you know through Punch Cards and all that and it's gotten better over time right you got um from direct binary into um assembly into compilers and and then into um kind of the uh not the object orian language what's the like the Java and all that the the easier to write

languages whatever those are but program I started that was absolutely speaking the computer's language and then we progressively made it a little bit easier but at the end of the day you're telling the computer exactly what to do in its language we kind of got tired of writing programs do the same thing over and over again so we created a bunch of commands that could do the same thing over and over again consistently hence the command prompt was born dos um the command shells and Linux Unix predecessors Etc and that was a great way of repeating it but then we got kind of tired of having to memorize all the flip and commands and there was a lot of

people that couldn't memorize all the commands and so we got hey why don't we have them click on those commands and we now have a guey that explains stuff and that tells you the options that you can do to click and of course each of these then progressed and got better over time and then there's been this dream for a very long time I know Bill Gates was promoting a90s it probably predates that of natural language interface where the human language works and the computer understands what we're doing and we've had chat Bots for a while we've had like the little the series and all that kind of stuff um but Siri honestly feels like

dos but verbally to be honest with you because if you don't say it exactly the right way it's like nope command not found or I'm sorry I didn't understand that or something but we've had this dream for a long time and this is starting to become real because we can now speak to it in our language and it will generally be able to figure out our intent maybe not perfectly but it's able ble to start doing that in a way that we have not experienced before and so that is one of the interesting things about this that is transforming a lot of things including security operations and the trend that we see over this time is that what it takes the

amount of skills you have to have and learn and train and go to school for is getting smaller and smaller to be able to be effective and then what you can do with it and again it takes time to develop these things and build the apps and all that kind of stuff allows much more to be done without having to go through and do as much work make sense and so that's kind of the progression we're on it is by no means magic but it is moving us to the right on that curve so the scenarios that Microsoft is focusing on because um they work and they work well um not perfectly but well is incident response kind of guiding

people along that journey and so that's one of the the priority these are the four priority scenario for uh co-pilot for security used to be called security co-pilot now it's co-pilot for security thank you marketing um impact analysis which is what did this incident result in so that we can then do business analysis and all the other things and actually produce a report that our Business Leaders uh can understand and then a technical incident summarization so we can understand what happened and we can bring someone on and have them you know help out with it you know supplement it take over the the case Etc um and you know send a report to the boss and get

them off my back so I can get back to the incident and then reverse engineering of scripts we found it's really good because it understands languages well and scripting is indeed a language right it's a logical flow so those are the scenarios that we found AI works well for so let's talk about capabilities and then I'll do the story of ack and then I think that's it so very very very people Centric as I Mission it's as I mentioned it's very Mission oriented un aligned and everything started with a Sim in most cases grabbing a whole bunch of data and then hopefully turning into a detection but not always xdr was a game Cher and this has

changed how we think about sock how I would build a sock from scratch today if I did today which is instead of requiring you to do something with the data it comes with vendor built vendor maintained often times Cloud hosted detections and all you have to do is learn what to do with those detections you don't have to figure out all the detection stuff you just figure out what to do with them because they're handed to you on a silver platter now you can still get Advanced you can go into hun hunting and use some other stuff for that but these things know and the thing is they're better than the classic Sim approach because endpoint for example

here when you talk about endpoint logs you're stuck with whatever that endpoint and the apps on it decide excited to share with you as an event they wanted to write when you put an agent or some other sensor on there that looks at memory looks at process trees and looks at uh those activities as well all of a sudden you now can look and see things that could not be seen just by looking at the logs regardless how how good you are at kql or Splunk query language that asset specific Focus plus the somebody else doing it for you on the on the alerts and the response procedures and automations often completely changes what you would

do in a sock I would never start a sock with a Sim today I would do a Sim last I would do a Sim after I put xdr out there on the things I cared about most endpoint identity email that is the traversal um the traversal pass of the attackers today network is interesting but we haven't had in the Microsoft sock um that I was mentioning earlier we haven't had a primary detection from Network thing uh from a network uh data since like 2016 or 17 I think it's been a long time we still use that it helps us with the investigation it helps us with threat hunting but the initial detection of an adversary almost always

comes from endpoint email and identity and then there's some app specific stuff rounding it out as well automation is key last thing you want people to do is repeated the same thing over and over and over again soar is fantastic the EXT our stuff that has sore built in is awesome and then you want to have custom SAR to do your own scenarios connect your tools and orchestrate them together threat intelligence that context of have we seen this before that helps you prioritize deprioritize it link and correlate that this is a part of that attacker that we learned about last month Etc having that context is critical it's not magic but it is critically important and then you don't have to do

it alone you can bring in experts and other folks to do it and Outsource it as well um so that's that one and then just cuz I uh and then the generative AI as I mentioned decided to put this as an extension of the human because that's really where it fits um there's a few new capabilities like the script analysis that it adds but that's not significant enough compared to a new analyst being able to ask what should I do next you know can you look at this that allows people in the first couple of years of their career to be a lot more productive than they would be otherwise very very powerful for bringing people up it does also help the

experts but it's very very powerful for folks that are Junior and early in career and then just because I could I used a morph transition to here's Microsoft's technology that does that um this is uh public as part of the MC so you're welcome to grab the actual original slide you're welcome to take a picture um but uh that one is part of the Microsoft cyber security reference architecture um which is a free download and includes all the slides SL notes for how it builds and all that it's probably about a page and a half two pages of notes which is the hardest part of getting those new ones out by the way doing the slides isn't as hard as

actually writing all the notes and making sure they're a so let's talk um skills and careers like as you build a sock um we'll do this through the lens of the the Personnel first and then we'll do it through the sort of stages of the team generally the first thing that happens is a couple people that are looking part-time and then maybe later full-time at alerts that are coming in sometimes often penetration testing as an outsourced function usually also at the same time as volume grows organization grows team grows you start splitting hey I'm going to have some folks working on the front line and doing the more repetitive tasks and the higher volume stuff and

then some stuff they're going to do more the high complexity deeper investigations that are likely a human attack operator so that's the first natural split that we see um sometimes you know Outsourcing and getting digital forensics and reverse engineering and whatnot from other sources or tools or what have you and then we see sort of the tier three as it were being built um and by the way we use the names of these not the numbers because we're not stack ranking people um we are absolutely not stack ranking people what we are doing is recognizing that there are different skill sets um there in fact when we do a large investigation at Microsoft um there's a lot of occasions

where these these guys are really great at emailing the things that they cover all the time there's no sense in having these folks waste their time on something they don't do very often so we often have those folks task out and do some of the email cleanup and the endpoint cleanup while these other folks focus on the servers containers and other elements of the attack and so we do see a lot of that teamwork going on there so it's not you know a one two and a three and you graduate up you never touch that stuff again that is not the way to do this you want to be looking at this as teams and functions that are

working well together and then sort of those you know classic tier three as it were you got threat hunting proactively looking for the adversaries that slip through detection engineering which is a very closely related capability um that uh basically builds clean detections that can then be used customer detections for your investigation and your triage and then purple and red teaming exercises as budget Etc allows so that tends to be there and of course the managers come in somewhere around the two to three uh time frame and then you may see some um additional functions as well where instead of the sock manager doing all the uh Incident Management coordination if you get large enough you uh may want

to have some dedicated folks doing that um often times the technical coordination gets pulled from the technical folks that have investigation experience and the business stuff talking to Legal comm's business leadership um e either on the manager or Incident Management and then a dedicated thread intelligence team sometimes comes around that same time this is not a guaranteed recipe that you have absolutely should use and follow this is generally an observation of what we typically see people invest in make sense so let's take a look at this through four different time stages part-time small medium and large very important not all organizations need or can afford a large team so don't sweat it if you don't have to get there it's

just uh to show that journey and it's critical critical critical regardless of your size to partner with all the other teams and I'm just about out of town so I'm going to go very very quick Qui LLY on these so triage investigation sorry I was calculating towards the end of the hour so I'm going to go very quickly you know kind of a part-time thing xdr first even though Sim was the way we used to do it I would never recommend it today getting into hey it's a full team with a full-time manager boom and add these capabilities and like I said these are all the security option framework I mentioned earlier you got the full-time

teams then you have the inter team problems and you need to coordinate between these teams um so very important you end up having to have more tools more capabilities more processes to connect and support them and then as you go to the global all of the things that were optional tend to be mandatory so that's kind of that Journey that we see there so just wrapping it up very quickly here as we mentioned minutes matter people matter process matters technology matters intelligence matters and teamwork matters and you know buy the book If you like and check out Mark's list if you want thank you [Applause] [Music]

[Music]