Cyber EMTs: Emergency Malware Technicians

today we have Callum who's joining us to talk about cyber EMS uh emergency malware technicians uh so yeah welcome Callum and take it

away all right hello everyone welcome to cyber EMTs or better How to Be an Effective sock analyst in incident response first I'd like to thank bid camber for having me here today and for all of you for coming along so who am I I'm a senior sock security Analyst at tessen working in the MSP sock that provides a manage detection response service to wide variety of customers as part of this I've had the great privilege of you call it to be involved in a number of instant response events previously I was in Army for six years before jumping over to reserves where I predominantly support cyber operations and there're just some of the qualifications I

have so what we'll be going over today mod is a bit tongue and cheek but plays homage to a real world equivalent uh to our role put simply as a sock analyst our place in the incident is the first responder lot of this presentation will be covering what should be understood and practice by sock analysts before and incident and to do so I'll cover what we need to understand about ourselves the adversaries and the tools available to us then we'll go over some skills and techniques I practice and find extremely useful when involved in instant response um so I've aimed this talk at newcomers and experienced analysts alike seeking to impart the fundamental skills required to Be an Effective sock analyst

in and around incident response all right so some context before we dive in as I just alluded to as sock analysts we're the First Responders to an incident we're the first ones to find something that is wrong and as such we're best positioned to assess the initial impact and stabilize the situation before instant responders fully assess the event and move to eradication and Recovery we need to rapidly triage the event and understand the extent of it to be uh to better sorry to better support the dedicated IR team and we achieve that best by our knowledge of the environment to identify what is normal what warrants further investigation and potential targets and high value information all right so we'll start off

with uh the understanding phase while we're far from being in a war the defensive offensive nature of cyber security still aligns with this lesson from sunu we need to know what we are protecting the most likely attacks and the tools we have available to protect ourselves the understanding phase is easily where we spend the most time by a long mile gaining awareness of the environments we're protecting is not a process completed overnight and depending on the complexity of your environments this is a large undertaking but well worth it for not only IR but day-to-day day operations as well so as Sun Su says we need to know ourselves and the enemy let's start with ourselves understanding ourselves

underpins all three areas we contribute to as First Responders to an IR investigation knowing what is normal warrants further investigation and potential targets his phase of understanding is really just asking a lot of questions with each answer bringing us closer to understand the environments we're dealing with broadly speaking we can segment what we're defending by four groups networks endpoints users and administration with within each of these groups we should ask and answer the following to complete our perception and understanding of the environment in the interest of time I've kept the descriptions to a high level these initial questions posed will Cascade into into more granular questions based on your environments so starting with networks we need to understand the composition is

the network small and flat or does it span multiple locations so on premise cloud or hybrid of both how do the different environments connect to each other if they exist and are they segmented and what devices and Technology enable the above Network functionality additionally we should look at how users connect are they via VPN or are they in uh specific offices do they use MFA or single factor for example so after networks we need to know what devices our environments consists of their purpose and how they're used configured controlled and protected so when we're identifying our endpoints we shouldn't just consider laptops and servers and desktop PCS we should also consider our equipment operational technology and Internet of

Things uh devices if applicable we should look at how user device devices are managed is it uh managed by the organization or do they bring in their own devices and how many do each user have additionally we should prioritize the assets we have and understand what is vital to the to the business and security of the organization within that we should also account for the operating systems and software in our environment we should understand what purpose is serving and how their uh logs are presented to us we should start to recognize typical system events in logs uh so we can start to uh filter out known benign events knowing what equipment software and operating systems are present is

essential to know especially when wide reaching vulnerabilities in software and devices are discovered for example the Myriad issues with the sharing software Network Edge devices we've seen in the last couple of years so moving on to users uh so we need to understand the types of users we have in our environment are they typical office suite and business line application users or do we have power users such as developers who have what I call varied and unpredictable actions within that we should also identify who our executive staff are while they typically have a standard user access they can often have uh an increased influence to alter processes making them Prime targets for for threats like business email

compromise should also understand uh the user Mobility as in where do they sign in from do they work from home or do they work from multiple offices are they expected to travel and are there any uh access restrictions in place and finally how are the users activities controlled is it through acceptable use policies as in we just tell them pretty please don't do this or is it enforc through um things like inuring or group policy answering these questions helps establish what typical user Behavior looks like uh aiding in identifying true and false positives so moving on to Administration knowing how an environment is administered is crucial for us as First Responders admin processes are often abused by threat

actors and the level of access of admins have to an environment requires vast response when suspicious activity around them occurs so we should look at what their tools and processes are are they commercial tools or is it just a bunch of scripts have hobbled together themselves where are these tools stored and who has access we should also look at how admin accounts are managed are they separate from their day-to-day accounts and if so are they named admin accounts as in one per admin or do they just have one that they share that everyone uses and related to the are service accounts these are accounts that enable automation like automated log on to specific services so often they'll have

very specific but highly privileged permissions so we should understand what their usage patterns are and where they're expected to connect to so we can start to identify um deviations from that expected Behavior so we also should understand the organization so it's crucial to understand our place in the world so we need to know what industry we operate in what products and services we offer what are the goals of the organization who are our peers or competitors and what countries do we operate in these insights help identify the functions we support typical third party interactions and potential exploitation points understanding the industry context allows us to spot trends that could impact us and adapt defenses finally we've got all the

questions but where do we get the answers so we can look at documentation like Sops Network Maps policies Etc but it's often better to just talk to the admins Network Engineers Etc to understand how they work and administer the system this is especially useful in establishing these relationships early because these are the people you need to talk to during an incident response so having that relationship already there is essential so next we need to know how these uh known events appear in logs or if we can see them at all finally we need to document all of these insights in a shared knowledge base this helps the entire team to to distinguish the false from True

positives during investigations however we should regularly review this information to keep it accurate and relevant as environments change over time and working from old information could be more detrimental than no information at all

all right so we know ourselves now how do we find out about the adversaries so most cannot defend against every threat actor so we need to prioritize based on what types are likely to Target us so we started looking at the highest level at the threat actor types I usually like to look at Cyber criminals and insiders uh regardless of the industry and and that sort of thing we're in a these often like cyber criminals often Target indiscriminately often focusing on exploiting the latest greatest vulnerability while inside a threats can be the result of simple mistakes inadvertently being manipulated or maliciousness to further consolidate this list we should consider the goals of these threat actor types and how they

relate to us as a company industry and Country from here we can prioritize that list based on most to least likely to Target us so next is uh understanding how they achieve their goals against us while threat active techniques vary the overall attack blueprint remains fairly consistent typically adversaries aim to gain access move within the environment gather sensitive information establish persistence escalate Privileges and then achieve their own goals not all attacks will follow this exact blueprint however it serves as a a good initial guideline of how to think about um how attackers will try and move through your network so finally in the understanding we need to understand our tools so we need to consider which logs

are being ingested and what events we can and can't see within our Network endpoints Cloud for example should also understand what security tools we have that are preventative measures and what they can and cannot do identifying these gaps early allows us to streamline investigations and prompt fixes where needed again this should be documented and regularly reviewed all right so now we'll move into some skills that are uh critical in instent response so starting with uh creative thinking to identify potential scenarios that we're seeing in logs we need to think outside the box using our knowledge of the environment potential threats available tools and the of suspicious event we've identified we can start to form hypotheses about its cause

essentially we come up with a narrative that describes the events we are seeing based on our understanding of ourselves and the adversaries a way to approach this is to think like an adversary so we adopt the attacker's mindset so when we see an event we should consider what would I aim for next we can use Frameworks like the attack Matrix to anticipate adversar tactics and find ways they might achieve their goals from here with each of these um hypotheses we ask what events are benign what events are malicious and where we can find evidence to confirm or refute each hypothesis next is communication collaboration so we should never work with in isolation we should collaborate with our

other security analysts the IT team Network Engineers Etc to cover all aspects and promptly address questions we should also understand our place in the incident response plan and what we need to do at each stage we should also have a firm communication strategy especially when an incident is identified we need to establish a communication plan and adhere to it even if there are no updates should also understand what channels we use for internal and external Communications as appropriate finally we need to communicate with empathy Your Role oh sorry our role is to gather and present facts to a varying audience with varying technical knowledge everyone will be feeling stressed and looking for answers so we

should keep that in mind when communicating all right so the last section is just some uh techniques I use during incident response to try and make things a bit faster and a bit more efficient so starting with miter attack as a map so for those aren't aware miter attack is a framework of the threat actors ttps or tactics techniques and procedures uh presented a matrix structure so tactics oist sit across the top left to right in a loose order of progression in an intrusion techniques are listed underneath uh and clicking on a technique it within the Matrix provides descriptions and procedures used by attackers that are seen in the wild so not every attack involves all

tactics and techniques the tactics used depend on the attacker specific goals and they may not follow a strict left to right progression so how do we use this as a map say for example we found an alert looks like a persistence event if we look to the left at that we can see that uh initial access and um uh sorry execution came before it so when we're looking through the logs we should be looking for events that line up with that from there we can look forward and anticipate things like privilege escalation lateral movement exfiltration and impact for example this is especially useful in dividing tasks among analy one can look forward while the other one looks back overall it

provides a structured map to guide the investigation and shape initial assumptions and hypotheses so next is the structured analytic techniques stolen this from CTI but I feel like they can share so the first one being the key assumption to check assumptions are great in that they can guide our initial analysis especially based on previous experience but we shouldn't hold on to these if they cont contradict evidence as they appear so when you start an investigation note down all the assumptions you're going down are going into it with and regularly review that to see if it still matches and if it needs to be thrown away essentially next is the analysis of of competing hypotheses this is where we

list all the theories we have of what the event could be and as we go along and find evidence we map that against each of these hypotheses as to say whether it supports refutes or has no bearing on and as time goes on as we continue this practice the most likely um explanation will start to emerge as it has the most evidence supporting it a caveat to this is use this to structure your thinking and notes but avoid over complicating the process it's very easy to go into a rabbit hole with getting these formatted all nice and everything but it's just a tool there to put like a left and right on your on your thinking

process so next is log analysis this is something we do every day but sometimes you'll find yourself using a tool you don't use very often during an incident and that is the worst possible time to be trying to refresh s on how to use it so we should regularly practice um using all the tools that we have available to us and understanding how they present the logs to us how to filter those logs effectively especially for those benign events we've identified finally you need to understand the data each log contains and the limitations of it some only contain metad data about events While others contain a lot of information so learning how to find what you need to

find within those is essential especially ahead of time so next is notetaking our investigation is incomplete if we can't collaborate and share our findings in an effective way the best way to support this without losing too much time is good but simple note taking in a shared resource this can act as a single source of Truth for tracking the event and it should have accountability immutability and versioning essentially we need to know how the notes have changed um over time and who made those changes when conducting an investigation you usually want to have a timeline communication plan assumptions and hypotheses and space for collected evidence for that investigation that each analyst is contributing to it

simplifies ongoing communication and document generation especially for PS po incident reviews and after Action reviews and finally poison rationality like First Responders we need to stay calm Under Pressure triaging intrusion can be stressful however it's crucial to stay level-headed and maintain analytical rigor the task may seem daunting but applying the skills knowledge and techniques provided can offer a clear road map of where are we now what do we need to uncover next and how can this intrusion be contained so we should like I've already touched on this but we need to collaborate work with other analysts and split up the tasks remember to take breaks these are marathons not Sprints to lean on that cliche uh make sure to step away when

you can to recharge and finally make sure you review and improve conduct an after action review with the entire team to identify successes gaps and areas for improvements and as great Panic at the Disco said it's much better to face these things with a sense of poise and rationality all right so just wrapping up we covered a lot today in a very quick succession we established the essential understanding skills and techniques First Responders to security incidents need understanding started with ourselves namely our networks endpoints users and administration before knowing our adversaries and their techn we Capp that off with knowing how the tools available to us work we covered essential skills in creative thinking especially thinking

like an adversary as well as how we should communicate and collaborate internally and externally with empathy finally we covered some techniques that are incredibly useful when conducting instant response investigations like miter attack Maps structure analytic techniques log analysis and remaining calm under pressure as my parting advice one of the most important parts of being a first responder is to look after yourself and your teammates this is a stressful but rewarding job work together and most importantly have fun when you can thank [Applause] you excellent thank you very much Callum um we've got some time for questions oh Jesus I'll oh hi Callum um great talk so I was really interested to know whether you do

structured debriefs after your um incidence yes that's what I was touching on with the um after Action reviews and that sort of thing I I also try to include people that weren't in the investigation as well in that because they come in with a completely fresh perspective and can see the forest from the tree sort of thing so that's definitely something we try to do after everyone um come pass that back um just on the topic of not taking and sharing resources what would your recommended way to have a clean shared resource or application for taking notes as a team so yeah I deliberately didn't um talk about specific tools CU every shop has their own um tools they use so

you know if you got office use one note um if you stuck in something else try and Leverage What You've Got cuz getting new tools is often very difficult so I sort of just wanted to give the I guess the purpose of it and then to try and let everyone work out how they can fit what they've got into that kind of uh

format how do you handle uh if your sock is leveraged across multiple different clients do you handle your analyst having to know uh have knowledge of each of those differ environments yeah so I'd say uh as an MSP we definitely experience that um sort of as I said at the start it's like a it's a very long ongoing process and that first phase isn't just for IR it's for sock operations in general so we try and have analysts do a deep dive into clients present that back to the other analysts share that we're not expecting each and every analysts to do the same level of research for every single client we have it's a shared

responsibility okay thank

you uh so it seems like a very high stress job like so how like what advice do you have for people trying to get into it like help managing stress under that situation where you're kind of under attack or managing an incident um I think it's something you definitely learn over time it's hard to practice it going in but I think what I often do is I think of how I've handled stressful situations in the past not necessarily in a soft context and sort of the techniques that you use to deal with stressful situations in general and keep those front of mind when you start to go into these times pressure situations

thanks and we've got one more up the front hi so uh in relation to your communication back to well other team members but also the end client um do you have any advice on how often you communicate with them and uh on every assumption that you find something new what's the how do you deal with that usually uh so we'd like to try and agree upon a schedule if it's an external client try to agree upon a schedule and rather than Pepper them with every little thing we find we'll give them say it's every 30 minutes or every hour depending on what's agreed upon we give them an update and that's still the same if in that hour we haven't found any new

information we still go still investigating you know same as what we had before excellent thank you very much Callum here's a gift from B CRA oh thank for presenting [Applause]