The 'Thinking' Analyst: Unlocking Core SOC Analysis Skills - Michael Lamb

ready hi everyone um I'll just point out straight away it ISS to join SL don't join slider I even installed the plugin it's not going to work I built a presentation on another laptop so I'm very well prepared um so my name is Michael uh this this Talk's effectively going to be about uh softcore analysis skills as it says um so it's not really going to be about uh technical skills um it's it's really my observations over the last four years work in different sock teams uh but predominantly over the last year um have had the pleasure of um working in Spire Technology Solutions um security operations center so without the sales spel where just based down on

the key side uh I work with Scotti here who's uh one of our team leads actually um and that team I've seen kind of grow from uh well a few people up to about 15 17 people um before that uh I actually started as a a cyber security Apprentice with the UK gol um you can imagine how that went it's public sector um and then I worked with n all for a brief period of time and then doed around uh a few of the government departments before then joining us by they kind of came along said Do you want to build a sock I said why the hell not um and I think I've lost him here in the process uh to be

honest but uh it's a great team now I think the last talks kind of set us up very well um I am a manager as well um so we can't really avoid opening up presentations and stuff without stats um so I'm just wondering you know these stats I've put on I'm just wondering if anybody agrees with them but um as it says 8 and a half out of 10 was generally the average rate of importance when it come to the businesses being surveyed um for complimentary soft skills in the in the sector um 32% of those businesses as well over the last 12 months they said that um they've seen applicant for cybercity related role

lock in uh complimentary or soft skills um so really what we're saying here is not necessarily the ability to use a tool uh or use technology or understand uh Frameworks but really that that's sof bit writing reports uh engaging with stakeholders being able to take uh information in and generally make meaning of this uh all of those softare skills uh seems to be locking in in the market with uh with regards to people who apply for roles 24% of all the businesses that were surveyed as well uh they're not very confident in carrying out a range of tasks that require a mix of those Technical and complimentary skills as well I found that quite surprising be so high uh you know

quarter in general is is quite low but um across all the businesses when I looked at those in that survey uh I would have said that's quite a high figure to be honest I'm going to introduce something as a as a term um has anybody seen the film you don't get the guess because I think you've seen the slides it's Forest Gump but um I've done a very brief stint with the British army as a reservist and at the time I actually asked um a friend of mine who's uh he's he's in the Special Forces as a reservist and I said well what's the difference part from being special um and he actually said well for

me it's it's the thinking Soldier um I said well you're going to have to elaborate a bit more than that all of USS we can all think and for him he said that there there's a general ideology at least in the forces where um the difference or the demarcation between the typical Soldier and the thinking Soldier is somebody who can project manage someone can manage their own time they don't need to be given an instruction that follow through with something uh they're able to analyze risk take that in and generally just see the bigger picture um and for me I feel like that really uh kind of mirrors well within a security Operation Center or

mirrors well within um just secut operations in general so pent test teams uh infos teams Etc there is that quite fine demarcation in terms of skill level in terms of being able to apply a framework or just use a piece of technology to them uh obviously then that softer skill as well now apologies so this is supposed to be the bit where we're all supposed to use slider um um so what this was going to be was effectively a quick exercise to just get people's brains um thinking and also just to get a gauge from the room on you know what would have been your question to ask of these things uh we actually done this exercise with with our team

not too long ago and it was interesting there isn't really a wrong answer um it was more about you know the questions that we ask when this type of alert comes in as a security Operation Center analist the questions that you initially asked change depending on well the time of the day it is you know so if you're a night shift B based analyst you shift buddies at fish the night before so he's off sick and you're on your own you might Panic a little bit and you might not ask the right questions immediately and you don't come to the right answers in a time that you would expect to be uh juning a day shift you know nor you can

turn somebody or ask but generally speaking you know there are some uh interesting points just to to flush over from this uh from this alert name so that's things such as intelligence you know so it's no malicious uh we could have a look at that for example and really dig into uh has the intelligence kind of aged over time how often do we uh rule our intelligence feeds for example we've got a characteristic there as well it's a binary it's an executable file um so if we're talking in the sense of a Windows system could that really tell us uh a bit more about what the file is could that tell us um again when

we look at the intelligence and kind of tie those two things together um does that tell us a bit about what the kill chain is what's the the motivations of uh the threat actor that we're looking at uh Behavior as well is important uh that kind of you know play things very nicely in terms of what do we need to do about this when this alert comes in so if we've set them an alert to detect a nor malicious binary that executes versus a non malicious binary that's uh written to the disk they're very much two different uh things in the sense of as a analyst I need to do two different things either if it's written to the

disk then obviously I just need to think about deleting that file and then investigate what led up to that however if it's executed maybe am I dealing with fishing for example am I with user based execution am I dealing with a driveby download from a fishing site or am I dealing with a threat that's already been in the environment for weeks months perhaps uh we do see it quite regular um and they actually try to execute their own binaries as well uh which in that case depends how we analyze this so there's a lot of challenges and then the last bit being kind of an environmental Factor so Mike's laptop um you know we can separate those two things so we've

got Mike as a person or account on the domain of the network we've got a laptop which is ultimately not a server so we then have to think about um you know what is M role in the business does he have privileged access to information does he have uh Roots into customers or partner networks as well where his laptop could be abused for that again thinking about uh the length of the breach as well in terms of timeline if somebody has been Stu on that laptop for quite a while have they been able to cphon emails or keep an eye on emails to the point where they can use that knowledge now to move on to a server

with within the network as well so really that that whole slid exercise that I've for fudged was going to ask you guys about what would be your questions and some of the examples I think that came out of our kind of tabletop if you will a couple of months ago was um things about um this binary itself so what can the binary do what is its characteristics um you know we do see it in so analyst interviews you know uh this there's normally a typical exam question of here's a malicious file what you do with the first world Chuck virus tool so you know that could be one of the first things to do the execution

element as well has been executed on any of the machines you know you want to know how Prevail in it is both locally and and kind of globally as well if you're a multitalent environment that could tell us as well a bit more about the intentions of the the banner file and again Mike's laptop there's a lot to ask about that as well has there been any other alerts um who was the last user that was authenticated to that machine and then we can work back from that as well and say well who actually authenticators might be Mike who's authenticated as Mike and so there's a bit of non-repudiation will apply that as well so changing our way of thinking as

well um it's kind of where things start as a as a sock analyst um I think we we all start in in that role and smashing things in a virus total and using shown looking at IP addresses and always constantly using the same searches in Splunk or whatever SC you us and or constantly clicking the same buttons in the EDR gooey because no one's ever updated nobody's ever showed you you could do something a little bit differently over here um so really it's it it's all centered around our way of thinking we are habitual people they're humans we do like hubs um and it is natural to just go and a comes in I'm going to do the same thing every time uh

there is that is you all say I think of doing the same thing and expecting different outcomes and all that um but I think there is Method Madness sometimes of doing the same thing every time um but it is mostly about just changing the way that you think being aware of that so starting about um start to talk about that self-awareness really of changing your way of thinking first thing bias we see it all the time in stock analysts uh but we probably see it everywhere else in in setups or cyber security um so there's a few different biases that we see uh confirmation bias being one uh really when you've kind of already decided uh what the outcome is going to

be when you're analyzing some evidence and you really chase that down to try and confirm that we see that all the time to be honest in s analy um guilty of it myself as well uh during incidents um evidence bias is really important as well you've got availability bias which is not on the slide as well so ultimately your evidence buyers being the fact that if you always investigate the alert that's just been on that slide there and you always go to the firew war logs you know you've got to bias towards the evidence Without Really recognizing that there could be a more valuable source of evidence could you be looking in the point lary such as cismon or

could you be looking at the EDR lary could you be looking at the web Prof logs as well could you be looking at identities such as active directory depending on the question you're trying to answer um you might actually arrive at a conclusion a little bit quicker a little bit more of an efficient manner the bottom line advice for the bias uh stuff is not really to kind of Crash it out and say well can't just run with a bias just to be aware of it so one thing I do like to do is when I'm writing up reports for example after an incident I do like to put some comments in and highlight bits of texts that I've

written said I've got a bias here and this is the bias so when somebody else proof reads that before the report is finalized they go well do I agree or not with that bias and that's fine a have again I suppose we are humans so there's two different thinking systems I suppose in our brain um not psychologist by the way but um I do like to think of it as Thing One and Thing Two for anyone that's seen C heart um but it's it's more formally known as system one and system 2 so system one is your intuition it's experience it's that quickfire make decision now I've decided on you know what I want system two is a little bit more

deliberate um you could think of that to be used in a sof dur maybe a post instant review uh possibly in kind of a retrospective or if you're doing threatment for example that can be a bit more deliberate because you've got a lot more time you're looking for something you're not expecting to find something sometimes maybe you are but you're not expect a make decision immediately it's uh well it's more deliberate again there's different biases in those so if you know if you were to split out system one and system two you can imagine they are going to be different biases in in each one but you ultimately can't have one without the other um as it says there you know um

our judgment as well um we actually judge people in about a tenth per second of seeing them so before the conscious mind of system two can actually decide what we really think of them system one's already decided that that board sealed and that's very much the same when an alert comes in um you know fatigue is a real thing workloads a real thing especially if you work in a 24/7 sock um that we operate in as well um you know these alerts come in all the time so system one's going to be doing a lot of the work there it just wants to get it out there's going to be slas that are going to be set it's proudy too well

I've got three p1's over here I've got this turning red over here uh the manager standing over the back of me at the moment asking if I can finalize whatever it was last week so system one does a lot of the work and a suck and again it's just to be aware um of uh of system one it can sometimes be a bit of a pain in the neck um but sometimes good to go back over an alert and use a bit of system too and there's a few different principles few different uh kind of Frameworks out there that can help contextualize uh system too but really this statement uh is actually in uh our

skills framework actually uh within our soft analyst should be deliberate and clear on actions to take during investigations um so really um the way to do that is to apply some principles so for example you've got the the Alexi ible uh so that is it's effectively a set of four four questions and ironically the first question that it asks is what question are you trying to answer uh yeah the second question is what data do you need to to answer that question how do you then EXT extract that data how do you get that data rout and what does that data tell you um and I think the principle's being covered in quite a few different DFI or blogs

recently few instant response blogs and articles it's very useful when you you know an alert comes in just start what question are you actually just trying to answer in the first place um and that tends to be a conversation I think both me and Scott do have with our analysts as well um we see alerts you know as they start a triage it and they go straight to One log source and that's not a personal thing with that person it's it seems to be a habitual systemic problem across different socks that I've seen over five years of experience now that they go straight to a certain log source and I said well what question are

you trying to answer you know you seen that claim being made by a tool you know that malicious Banner is executed on this laptop you've went to firewall logs why you doing that and you can really demonstrate that through typically in an itm until you know you've got a ticket to work with you you put comments on or you'll scratch notes down but it's always good to stick with that principle um it helps you know from an order perspective as well when you go back over the ticket or your your piece of work whatever it is your investigation record and to then demonstrate using that principle you know this is the question I try to answer here's maybe

the bias um here's where I looked for that data um because when we try and answer that second question such as where do I need to go to answer that question how do I extract that data there could be two three four different places depending on the size of the the organization that you operate in as well so if somebody looks back over that and says well youve looked in the firewall logs but you might have got a little bit more context because we had full packet capture over here we could have seen packet level information for example um and that's again where the evidence bias kicks in um it might just be because you

used to use in firewall and you know you don't really want to start using full P capture because you're going have to start learn youri model you know have to start learn how to use wi Shar for example and a lot of other technical challenges so it's all about using that framework to recognize uh some of the barriers really from from a skills perspective but also if you need to share your work with somebody else that they can understand what the method of the madness really was um the last one sorry actually is is handland Razor uh so that is just about never attributing to to molus to what can be explained by stupidity um again

talking about being humans uh humans do stupid things we click links um I was actually I think Contracting at the time in in UK go as a senior analy and uh I got fished um she was really silly and I effectively got put into the work jail attend a fish course and everybody goes around and they're like so we'll do we'll do a round table it's almost as if um if anyone's been called speed you got speeding awareness thing and you kind of do a round table of inter in yourself and I was like yeah see anal from the security oh yeah so you'll be delivering this it um so yeah I mean people do it um but

really what this is about from from our standpoint sock analysts is just to take a neutral position uh try and consider the probable causes as well um because sometimes you know systems do things just because systems do things you know they're you Computing things they're just given instructions sometimes by humans um so that's kind of what that's all about handling information is is kind of a big area for so analysts again and probably pentests to be honest um you know we take a lot of information in we're expected to take that in understand it make meaning from it and kind of come out the other end with either a decision an option or an instruction just do

something with that really um so there is uh quite a cool little uh loop called UDA um don't check it I think it came from the Royal Air Force um but it was mainly used in dog fights between fighter jets and the whole point was that the pilot was supposed to uh take information in this for to analyze their environment who their enemy is ETC decide knocked on that and the whole concept behind the ud Loop was whoever closed that Loop the fastest in theory should always come out the other end on scared and I've seen this again quite a few different times now that I respectively look over uh work that I've actually seen this um in different sock

teams so a few years ago uh we had um I think it was a fishing ticket and we had things different observable such as a malicious link a user account involved or an email an email sender there was a lot of different things there to analyze and it was very much analysis paralysis we were just stuck there's just too many things to analyze so there was a lot of observing and orientation going on but there wasn't a lot of deciding up so throughout that ticket there was a lot of a going on so it's say well this Ling malicious and it's got these characteristics it relates to this file there was a lot of virus to and going on

as I like to call it but then there wasn't off the back of that going well okay it's delivered in the mailbox somebody's clicked it because we've seen the web proxy logs or whatever and it's downloaded file they're observing that but then they weren't going right I need to do something about that or I need to give an instruction to someone or at least say this is what we need to do about that so to be honest in soft teams again talk a bridge and that skills Gap we see in uh kind of junior stock analyst we see a lot of observation a lot of orientation and it might be a confidence thing it could be uh it could

depend on the culture on the team as well if there's a lot of blame culture as well well no one's going to make a decision could let alone take out an action as well um so it needs to be uh the right environment for the analyst to operate in really and so there's there's a lot of stuff on the ud Loop to be honest online on how you can map that back to a work culture and how you can really reinforce that second half of that Loop but uh generally speaking the F you close that Loop um you know the F you're going to beat the threat actor and there's a lot starts online uh about

threat actors being faster and break out time increasing and stuff like that so I think it's a very important piece a lot of what we do is tell them a story uh people like to be told a story again the humans uh we like to hear it whether or not it's the truth it's all about the confirmation bias again um I speak to quite regularly Financial directors it directors head of security sees Sals who you can kind of tell they want to hear a story they don't want to hear the truth and they really want want to hear that actually it was this guy over here that clicked a link and it's all their fault not the Russian person

who sent the link that it's their fault s them um and they want to hear that in the report and they will try and guide you down that route so you know bottom line is it's very important to understand your audience understand your stakeholders and understand what's their requirements what do they want to get out of this if somebody's asked you to write a report and they want to be seven 8 nine 10 pages make sure they're not wasting your time but make sure that what you're putting your name against in terms of advice and what actions are being taken marry up with their requirements some of the tips of written communication um again stuff that we um

we see across different socks is generally report of quite poor quality and I think that steps back to the original survey the stats there because of the I would say the types of people that become sock analysts at the moment there's a massive drive to attract people from uh different areas kind of different Industries as well and some of those people don't come from uh previous roles for example where they've had to write reports or they've had to uh explain to stakeholder for example or they've had to really communicate the risk and and quantify that for example so there's a few tips there um Scot know I love keep Simple stupid or kiss actually got that from the British army

unfortunately should probably drop that it's not a good idea but I do like to stick to that to be honest uh that's probably a bit of a system system one principle um it's a bit of a raise a thing you know you can normally just explain through stupidity um and just keep it keep it simple don't overthink it don't over complicate it you know for binary comes in a nor malicious binary do we really need to understand what Windows API calls it makes do we really need to understand the strings that within that compil binary probably not at this stage what we need to understand is how do they get there in the first place what's executing it what's the

parent process of it what's the command line and ultimately a lot of what we deal with as so analyst is claims being near by security tools um and those claims are well there's a claim being made that something executed well let's look at the rule logic how do we know that it's being executed is it is it a certain flag is it a command line parameter um you know is it is it a certain PID that's generated on the machine so there's a lot of validation to do as well with all these disperate security tools as well some of the other principles are actually taken from uh the College of pleason um so uh we

actually found that I think online a few months ago um um and it's I think looking at the website it looks to be a publicly available library of trainer resources for kind of train police officers and there's there's a really good article I'll put the link at the end of the slide deck but it's all about delivering effective analysis and some of those principles uh bottom line of front or Bluff kind of just get straight at the point really 433 which is all about you know uh there shouldn't be any more than four paragraphs and each paragraph shouldn't be any any any longer than three lines for example um and that can keep things quite sent it

keep it quite concise um you've got ABC or accuracy uh brevity and clarity as well so just being clear and being accurate um especially in the police suppos and law enforcement I suppose they have to but again in socks you know you might have to take quite a forensic approach you might have to prove um that you know what you've seen and how you've analyzed it is exactly what it is so such as if we see that something's executed on a system we might have to use a couple of different sources we might have to point back to our experience as an analyst and say well my experience of XY Z tells me that this is

genuinely malicious for example we have to really be clear on that and then there's different techniques as well depend on what you want to get out of you know whatever you're writing and again this these can all be applied to either comments and tickets it can be full report it could just be a single slide within an overall deck if you're summarizing an incident um for pen testers as well I think you know that's going to be quite important and you know there's a lot of findings that have to be communicated so if you're writing for impact if you're writing for risk or you're writing for Action Can Depend who you're talking to really um but rting

for risk you know do we need to quantify that in some way do we need to link that back to financials in terms of impact of the business if that risk isn't um remediated or um at least Quantified in the right way um do we need to maybe visualize it as well rather than just having bullet points or you know text there's so many different ways to do WR communication for verbal communication I'm actually not going to give good tips uh to be honest what I'm going to talk about is actually what impacts um verbal communication what makes it bad I think is is easier with verbal communication the logo in the bottom right of the

slide um is Sky um I think Lisa fellows done a very good article in interview on this recently uh in the inet community community sorry where she introduced this um this particular resource and when I went to dig into it I realized that actually this marries up very well Security operation so what skyber is is it's an online resource of um causal factors of accidents in the aviation industry and when we think about the aviation industry it's a very human Le um industry there's not a lot of automation or at least we had to go with that a couple years ago didn't go very well if you've seen the news um but it's a very good resource and there's quite a

few different kind of categories of factors and one of those is verbal communication I believe causes about 70% of accidents in the aviation industry so things like fatigue you know if you want back toback flights very much the same working on night shifts or shift work um you know for example our team works at 414 off and they switch between days and nights quite often as well so that you know can really tie out and sometimes you just don't even know what day of the week it is there's technical issues as well obviously postco this is accelerated quite a lot so typically in remote teams you know you've got Microsoft teams but in F in house teams

like ourselves you know if one person's working from home uh sometimes we just forget to tell people into to meets as well when they w at home and that cause a break down communication as well that can then cause issues later on the day later on the week when certain people will not calls workload as well as massive um you know we talk about that a lot in cyber security so I'm probably not going to lab that point but you know alert fatigue everyone bangs about um manner of speech as well is very important um it depends really on your environment so you know have you just walked straight up to somebody and you know use in quite an aggressive tone uh

or are you just you know talking in in the work kitchen and breakout space in quite a casual manner it depends again on your role so you know if you're a manager and you you know you're giving out an instruction in quite a casual manner uh that's not really in a wordplace environment such as the the kitchen area does that person take that as a form of instruction or not probably not and if it's quite an important instruction that relates to um let's say an incident you know maybe you didn't get the point across and then you follow later on go how you get on up they're like what do you mean the same again for so analysts as well

when you know we're communicating between each other if we're handing over between shits for example as well it's very important to be quite uh objective to be quite factual um to make sure that that person can take over from you uh on that next Shi for the next 8 or 12 hours or whatever it might be subjectivity it does impact verbal communication but it's important you know again we are human we need to feed our opinions in we need to use that system one our intuition what does our experience tell us as well so it's it's quite important to be subjective as well um that does matter rather than just kind of being a robot here here's the

fact here's the data from the internet put in ch apt and this is what that's telling me uh just use your experience as well it's important you know um we've seen all of us who've got even a year of experience but they've got a ton of exposure uh to certain areas within Tech um that they can make a decision very quickly based on that incident reports as well um so this picture I don't know if anybody's if anybody knows what this is but uh it's the term rubber duing it's not an innuendo um the whole point of it is actually in software development houses where the concept is that if you can explain your code to a rubber duck you

should be able to uh understand how it works you should be able to explain it to an exac or a stakeholder um and I don't think we do a lot of that to be honest in in cyber security all in security operations you know we're not developing things but there's a lot of outputs from our work we do a lot of great work we we write up those reports we fill in a lot of tickets uh we set a lot of calls with with different people so it's important to kind of rubber dug your work just explain it yourself go through it with you in your head or just on a oneon-one basis with the person next to you at

your desk or over A team's call just go through it step by step and that sometimes helps me from a rep perspective um you know I kind of go through it and go doesn't make any sense at all but avoid assumptions um again as it says no matter how technical or good you are on those tools uh the success of that report is really how you tell the story um that's what people are reading it for um if they don't get the narrative or they don't get what you're getting up they don't know what you're trying to get out with them they will just close that and put in the bottom of the but of their look in box know your

audience and there's a very good resource down there which is Storyteller tactics by pip decks um pip decks do kind of like tarot cards so tips as long different ones but the Storyteller topics one's really good and I'll just finish off with some tips to cost I appreciate probably just throwing a lot of principles a lot of ideas they're going well what the hell do we actually do with this um if you're you know a sock manager or you're aspiring to lead a sock or getting into that kind of capacity within a security operations team I would encourage you to look at n nice it's a framework of skills knowledge Etc and and it's not

just technical by Nature um it has a lot of other soft skills in there but but within the this nice framework it also groups those by role as well within uh security operations so if you needed just to get started you can just take a cyber incident responder and it gives you all the skills and knowledge already needed and then you can start to assess either yourself a against that or your team provide feedback to each other so that UD loop as well is important you know need to constantly have that feedback loop uh you need to be constantly saying to each you know yeah that's great that's not so great Etc uh I'm not going to give you advice on how

to get feedback uh such as the sandwich um But ultimately it depends on you as a team how you work with the peers to how you prefer to do feedback quality assurance is important qer there's a very good article by uh expel uh John from XEL for certain him but xl.com there they're a big MDR hosts suppos in the US uh but they do a brilliant article on how perform quality assurance for uh Security operation Center Performing uh post instant reviews lesson learn sessions as well uh they great to ident F where's the bias um you know where was my system one a little bit wrong here um could we have told the story a little bit better how

is it that we gather the evidence where we asking the right questions from the Geto for example tabletops ctfs and Labs um it's actually really good to see black hills are here so back doors and breaches they are giving away those um we do them every now and then we've got the decks in the office but it's got to do tabletop exercise got to do a labs ctfs and stuff and really do that in in a a nor pressure environment um but sometimes again you know doing that in production can sometimes draw the real reactions when it is is live I don't avoid that either and that's it really just some some links and references um so that the

skyber stuff H Razer uh the police college and then there's a boook called thinking fast and slow that's it any questions yes um how do you alter or train people to Rite for tickets so that you can actually understand what they've looked at previously rather than writing a report or something like that see a good ticket tells you what you need to know rather than one that doesn't yeah yeah you on um it's more of a cultural thing to be honest um it's really about changing the mindsets and and really selling it why rather than it just needs to be done because a manager somewhere needs to do something it needs to be sold on the basis that it's it's all for

Improvement helps them understand kind of the bigger picture of of analysis um one thing that I'm seeing teams having some good success with is using chat GPT um so I think uh our service desk team are actually using AI to generate uh the summaries of their analysis or their work uh on tickets that a customer face effectively um so AI can really help uh but some general guidelines and guard rails as well can help uh so say you know uh some of the techniques that we've covered in those slides as well um so those principles such as bottom line up front um you can write examples that the analyst can then read from and then

learn from that as well think that's all the te we have