Exploring Threat Hunting Practices And Challenges In Cyber Defense

hello all uh my name is prianka uh I'm doing PhD from University of Bristol and today I'm going to present uh my research uh it's called unwell the hunter gatherers exploring threatened in practices and challenges in cyber defense this is the research paper which I have recently published at the US security uh this work I have done with the support of my supervisors Marvin elanora and aice so quick introduction what is threat hunting so basically threat hunting is an approach where cyber analyst aim to identify potential cyber threats which may have evaded your organizational security systems or it can be done in the two ways uh proactive approach and the reactive way so why we did this study what is the

motivation behind it there is a growing need to understand security workers uh there has been work done into the bug bounty hunters cesos security analyst and pentesters but uh we want to focus on the threat Hunters because there is a limited understanding of the threat hunters in their practices till the date there has been no work focused on specifically on the threat Hunters what they feel what are the challenges they face so we we want to answer two research questions basically two broad research questions who performs thre hunting and how do they do it the second what are the challenges they face and how do they overcome them so in order to do so we conducted a

semi-structure qualitative study where we wanted to hire a threat Hunters who have minimum of 2 and a half years of experience in this field uh and then we use the snowball and then we recruited threat Hunters uh through different channels at the end we have total of 22 participants from across the globe majority were there from the US and the UK in our sample we have the fenters uh working in the M mssp inhouse plus mssp and the in-house so we have overall really balanced sample so you know let's go to the findings who performs threatening we want to understand who perform exactly so we found that there these are the people who have the diverse

qualification they are part of the dedicated team or they are not they have the main job as different they have different responsibility and job and then they do threat hunting as a part-time and these threat hunting teams are formed randomly for example one of our participants mentioned that I have a team of eight people most of them actually started on our foreign team and I have kind approach people from the forensic team pentest team to form threat hunting team I had to to cross Trin the forensic guys and then pentesters so the takeaway from that is the threat ending is basically carried out by analyst with diverse qualification who may be part of dedicated team or have other

organizational responsibilities since there are no specific quality qualific requirement for the threat hunting organizational organization can spend the time and resources to train them now once we know who performs thre hunting we want to understand how do they do it so we found that basically they do it in three ways use case base hunting Intel based hunting random based hunting in the use case they basically use the predefined scenarios they build the hypothetical cases to find the suspicious activities within the network in the Intel Bas hunting they leverage the right intelligence uh such as the indicator of compromise indicator of attacks which is also known as ioc or I way to guide through T hunting process

and in the random hunting it's I don't think this specific technique is mentioned anywhere is so Random because our partisan mentioned they so Random they they do it without any prior knowledge and any without any specific ioc or any without any predefined plan so this is really interesting to know that they are doing that way as well the one thing to note here is that also there are three different methods they are not limit themselves to only one sometimes they also use the mix approach so in the second part of the threat hunting as I mentioned there are three methods however there is also something that they follow in a systematic way which I'll explain next

slides so first our thread Hunters mention that they start with the planning which they call the preh hunt plan in that they gather all the informations for what they want to look for what kind of the tactiques they will need what kind of resources they will allocate for the specific hunt as mentioned as our Pary mentioned that we try to plan all the stuff out in the beginning we plan getting together in the beginning with everyone involved in it and then just let the system run and then if something comes up we deal with as a team so the takeaway is like more most threat ending activities start with a dedicated plan where threat Anders

Define the scope of the investigation ass say the threat intelligent generate hypothesis uh use cases and identify relevant data sources uh for the basically choosen approach if they have the uh if they have the use case based hunting they plan according to the use case based hunting if they have the Intel Bas hunting that they have the plan according to the interfase hunting so once they have the preh hunt plan they move towards the data collection and preparation so we found that they use the various tools to collect the data uh the main thing is like the hunting approach determines what kind of the data they will need and also the data as much as data they collect the quality of

the hunt would be better the takeaway is like the threat Hunter stor their data collection to align with the threat hunting goals and collect as much as as are possible to maximize their threat hunting Effectiveness next is like hunting and validation once they have all the data and all the prepared everything they move towards the hunting hunting and validation is two different processes however they are also connected together for example in the case of random hunts we found that uh also they are unplanned and are often initiated by hunch threat Hunter observe something that is malicious so they have to go back to the data collection they have to refine that query until and unless and then they

find actual what they're looking for so identification and validation are interconnected processes threat Hunters employs various tools and manually search and validate ioc's and also validation is a team process because if you need to validate you also need to discuss with the team that it's actually the proper hand it's not the false positive once they have the validated they have to take the remediation action and start reporting these are also to uh connect interconnected processes they take the remediation actions based on the severity if the severity is high they take the immediation immediate action uh lesson learn are also critical part as mentioned by the Earth Hunters they mention that reporting will really enhance them in the future hunt if they

reported properly it will really help them to identify new threats and then will really help in the future threat hunting process the takea is like validation remediation and reporting are also interconnected and vital stages of threat hunting they are crucial for ensuring that threats are effectively address lesson learn are incorporated into future hunting processes and necessary actions are taken to enhance uh organization security so now once we know how do they do it we also found what are the challenges we ask them what are the challenges you face while doing the threat hunting they have mentioned lots of challeng es but we have categorized them into three broad categories um method related challenges data related

challenges organization and people related challenges I will not explain what the false positive and everything is but I will explain what our threat ERS feels about this challenge how they hinders the process in the threat hunting uh method related challenges so false what so our participants P9 mentioned that yes there are many false parti positives especially on the powerwell side we buil a use case to block poers shell in the network and it sometimes collected when we ask about we receive responses like no we are not using powers shell but it turn out there is one admin who is very proficient with power shell and who used it we thought oh what's happening here and alali says

oh no one is is in po shell but there is a senior guy who was using it there are lots of cases like foral POS someone is using something in in the organization and there are not informing the security team and that flags up and that are really really uh making alerts and that makes a really challenge for the right Hunters building use cases are complex gtps our participant 8 mentioned the biggest challenge we have is uh going through all the tactiques and identifying what we can look for in particular client environment because the My3 attack metric is very broad there is so much data that that you have to go through and understand what data

sources has needed for particular techniques to focus on evolving techniques and techniques of course it's like cyber teex are emerging so as so does the adversaries so participant five mentioned the biggest challenge would be uh is the changing ttps when you have groups that change the way they operate you can't go in knowing that Conti does this or APD 28 does this you can't really be used that much because they could change tactiques groups change the tactiques all the time so that's probably the biggest thing in involving techniques if some of you are working in this space you can relate with some of the challenges if you are facing yeah system and tools falling our

participant to mention because we do correlate we some re uh searches takes more time in CPUs than others so we need optimize so that one research does not Crush all the system we try to do that but the issue is the lab we only have few rules and only running when we want them to run because when you have the brick production you have thousands of rule running all parallel together so when we create a rule we have to make sure that it will fit into the big bigger production without crashing everything so yeah it's a really big problem so take away from the method challenges like building relevant use cases or hypothesis is a big challenge

because attacks constantly evolve deling false positive whether positives or negative pose many challenges including times and team efforts while cyber security tools plays a really crucial role in tread hunting they sometimes fall shorts uh next uh is the data related challenges complex data it's very challenging to keep up with the tech technology and available options logs often change their format the way they appear the way they pass so what that mean is that the detection that we run is based on what the logs look like a year back they are not looking what it is today incomplete and low quality data our participant 18 mention another thing that becomes apparent is when you expect the data to be there but it's not

you should have 30 days of logs but when you check there are only seven what happened there I think there are the basic of obtaining the data and some people would call this visibility I need this visibility into this thing in order to do that log is a really big challenge sometimes you have less logs and sometimes you have a lots of log which is like data overload to make sense of the data we have a lot of it and to get something interesting from it is a big challenge you end up with L of results that are useless in the data set it's hard to filter out new threats it's very challenging and it takes time to adjust

the query to get the right data once you have the data also there is limited data storage of course it's a really big problem because although you have data you might not store that data like B mentioned that the problem I have seen that people don't think about what happen when you need the forensic what happens when there is compromise you need data for 30 days 60 days or even one and 180 days takeaways like threat hunting mostly relies extend extensively on the availability usability and the quality of the data however this is crucial aspect present in it's a really significant Challenge and this difficulty arises from obtaining the required data for the investigation due to PO logging poor

visibility and lack of the data storage in the organization practices the complexity and overwhelming of the nature of the data making it challenges uh for the analyst to uh whatever the knowledge they have for the tools organization and people related challenges skill set communication budget constraint skill set our threat Hunter 8 mentioned our main challenge is that not a lot of ID people we encounter are well vers in security and Incident Management so we have to give them stepbystep instruction on what to do and it takes time and even if you do so sometimes they don't understand or what is happening why do we need this we have to let them understand otherwise they are going to make

mistakes in in terms of the communication they are really Keen they are different types of the communication challenges either they have the lack of the communication within the team either they have the lack of the communication within the tools for example our participant 3 mentioned there is no point of having threat in D if you can't communicate it to someone in a proper way so someone who is very skilled at reporting who can create a loss of information communication is quite also useful to the people that are working on the broad things like that we don't understand cyber and presenting the threats to them in a way that they can understand and they can use it in a way

so if you can't communicate well it's not going to work work well with the threat hunting team budget constraint and lack of resources cost is one of the biggest issue I think every organization have this issue because the technology that compromises needs are expensive this participant say that I have been working in threat ending for almost 20 years one from another as well as exploited all kind of stuff we have to do the best job possible with lack of information and lack of Technology at the client so threatening uh Effectiveness is integrally tied to the skills and experience of the threat Hunters however finding and retaining quality staff in the field is present is is a really uh

Big Challenge the threatening is also under are funded in most of the organizations so once we ask them they also mention the best practice uh strategies to overcome this uh uh to overcome this challenges so we they have mentioned different uh Challen uh strategies but we have categorized them into the three main uh strategies uh reanalyze retuning and collaborating so in order for example in order to address the false alert our participants mentioned that uh you they have to do the conduct Toro analysis uh they have to go to the historic data identify patterns and common characters of the previous attack in order to reduce the false positive or the false negatives they also explain some

constant fine-tuning on their approach refine the detection uh rules use case and algorithm in order to do uh in in order to hunt for the best they also make uh importance on the collaboration and sharing and learning study the je is automative repetitive task they mention that uh it will really help to help them to scale the search and improve its Effectiveness uh automation also helps them to ensure the consistency accuracy around the data collection and Analysis and also help help them to reduce the burden uh because if they automate certain tasks they say that they will have some times to Define more strategy and then they perform the more better threat hunting practices refine data collection

strategies the mention that identifying collecting only information that is needed because if you have so much data you can't just find everything in that so they mention that you can collect only what is needed and also use the officient method for collecting and storing data they also encourage the best practices around the data collection while other also mentioned that having centralized data in the place they really help them because if they they they don't have to go to the firewall to collect the data they don't have to go to the EDR they have to have the centralized data collection platform strategy for asking for better budget allocation because organization do not think that threat hunting requires lots of uh

budget but they say that acquiring right tool organization can conduct trainings uh participant mentioned that having adequate budget collection is also crucial that will really help them and if they if organization can spend in resources and tool that will really help them to detect and mitigate threats keeping up with the current threats staying ined because cyber attacks are emerging you have to have stay informed about all the latest threats and uh regarding that they also mention the continuous learning approach to keep them updated on the current threats and how to identify threats to response them they could be achieved through trainings and research effort they also mentioned that documenting and Reporting new threats is part of their

new learning and keeping up with the new threats being flexible open-minded a partisan mentioned that one way of making threatening easier is not following rigid and repeative process as threats are constantly evolve one needs to be creative and find new ways of hunting for threats being strict in approach or not adapting the process could lead to missing new threats or emerging threats yeah thank you so much uh this is our paper link so please read our paper we have so much details in our paper so please connect with me and linkon thank you so

much minutes for Q&A one of your first SES you had snowball up there yeah what's snowball so snowball for example in the research you need to in the recruitment for example I know you and you know someone but but I don't know them so I know them via you so that's called the snowball techniques so I I I I I talk to you and there's okay this is the study I'm doing and I need people who are in the threatening and if you know someone you pass my study details to that person and then I connect with that person via you so that's called the

snow really good research did you get any timings on different t time is

ining oh that's really good question really the partisan mention is based on what for if they are doing the mssp so if they have the contract so the threat handing is not as defined if you have the incident response for example you have if incident is trigger in your system you have some sort of the time based on the civility if the civility is high you take ex immediately if it's you know one 2 three so if it's low you can take a week or so so also in the case of the threat ending but they have not specifically mentioned that how long it will take for them to actual hunt yeah are specific tools used only by

hunters or just simar tools like analyst but so in order to tools for example in the use case h okay so they are building the hypothetical scenarios so they can use the open source framewor for example Mitra framework so they are utilizing that platform to build the hypothetical cases and then if that cases is triggered then they will start analysis and even in the analysis they sometimes for the log collection also they have The Firs they have the edrs all the logs from that all the defensive devices they use the remediation of course they will block whatever the tools and Technology they have that is based on the which organizations they are working on