Threat Intelligence In Numbers - Nir Yosha

yeah I like you so good morning everyone my name is Mitch taki I'm on the Planning Committee for b-sides Boston Hope thanks me so I'm just here to introduce near he's going to talk about statistics threat intelligence threat hunting he's from threat coach motion so he's near he's also at the end of the talk these lovely laser me toss them out t-shirts and stuff and stickers so be on the lookout all right thank you guys thank you hi everyone how is everyone doing as well so I took this photo you see up here while on vacation in the Caribbean specifically in Sint Maarten and I remember taking that photo because at the same day I received the email from

b-side informing me that my talk was accepted so I remember taking the photo sitting in a balcony watching this beautiful sunset feeling the Caribbean breathe and thinking about what might be sliced off right so here's what we've got to talk about today the first thing I want to suggest is that threat intelligence datasets are growing and continue to grow I will use some statistics and some numbers in order to prove my point and then since we are dealing with big data sets I'll suggest data mining techniques first classification then scoring then I'll speak a little bit about threat intelligence investigation and I'll end up with threat intelligence effectiveness if you guys have any questions at all you can raise your hand

or we can have some Q&A time at the end so a little bit about myself you probably guessed I'm not originally from the US I was born in Israel I served in the Israeli intelligence courts specifically working with signal processing and intelligence I graduated as a computer engineer and I moved to the u.s. 15 years ago working for multiple vendors specifically in the network security in user behavior analytics and obviously insert intelligence so what is threatening diligence in my mind so as a threat intelligence engineer I read a lot of reports coming in after every incident and I like all the reports whether is the pandas or the Bears the infrastructures or the Internet of

Things the Middle East or the America it's all very interesting however not everything that is interesting to me necessarily relevant to my customers on the other hand the customers thinks on imminent threats and risks does not necessarily match the actual threats landscapes so threat intelligence to me is all about this middle ground it's all about the overlap between the organization goals the organization organization needs from one side and threat intelligence landscape from the other what impact your critical asset right what impact your vulnerabilities and your reputation and since this is what threat intelligence cycle looks like in my mind it all begins with objectives we need to understand what are the requirements of the organization's are we looking at

tactics strategic or operational threat intelligence do we want to work with the cyber kill chain model or do we want to work with the diamond model once instead of the requirements we look at collecting threat intelligent information and threat intelligence comes from both internal and external sources

the one second

okay so collection involves indicators of compromise now those could be coming in from inside or from outside and those indicators eventually will be due to placated aggregated normalized the next step is taking the information into processing what is processing processing is the face where you are actually trying to make sense out of the data right and so this is the one I'm going to focus on mostly today and eventually dissemination threat intelligence should be actionable we want to be able to send alert into the IR team to send information to the security operation center send reports to management and so on so getting back to my first point wise rated data sets will continue to

grow starting at this chart it shows you the predicting number of connected devices and people to the internet by 2020 now we currently around 28 here billion devices already connected to the internet thanks to the Internet of Things and this is going to grow up to 50 billion devices by 2020 I suppose people we have around two billion people connected today and that number is going to grow as well to around six billion by 2020 now what if that means that means that both the network attack surfaces and the human attack surfaces will continue to grow that means that they're going to be more surfaces for adversary to launch their attacks more social engineering more spearfishing and so on

on top of that there is a cyber security hiring crisis we already there they are missing people from this cyber security workplace and it is expected to get into one and a half million people missing from the security workplace by 20:19 what that means is that they're going to be less defenders going to be less people watching our networks and less threat intelligence engineers and so as a result we will continue to have incidents coming in all the time here's an incident and when we're getting we're getting those incidents the companies will serve information there's going to be more and more data sets or threating to information and therefore data sets will continue to grow now getting back

into my vacation how do you think the weather was in st. Maarten so me and my wife took the average of the temperature calculated they standard deviation and decided that 95% is going to be a good weather right no we're not that crazy we know weather in st. Maarten will be always nice right however the physics is neither useful weather prediction and can be used in threat intelligence so this is an example of a chart done by palapa theory from the hack mageddon website and it looks for the past three years of reported incidents and the average is that we get 82 reported per month this is only the reported incidence will probably get much more

and then the standard variation 13 which means that more or less we can use this number as an average and so let's try to find how many attacks are currently in progress the concurrent attacks in order to find this I need to take a dwell time the blow time is the time from the first intrusion until victims is actually detecting the attack right and so this explores us that it is four months so I'm taking four months / 82 incident per month there are 328 transparent attacks 328 companies under attack at this moment and that's the bare minimum now we were behind those attacks well security companies identified around 50 adversary groups there are thousands of hackers and there are even

more script kiddies but big huge adversary groups that can create damage that over 2 million records stolen there are somewhere around 50 and you can see some of them here there are anywhere between nation-state cyber crime which is the ones we're going to focus on hacktivist and cyber terrorists so let's look at the cybercrime one again I'm just playing with numbers here and I think that based on the 60 percent of hackers working in cybercrime the 30 is an average number for this high-level sophisticated hackers that can create a damage of 2 million records stolen stand up now again looking at the statistics we have around 35 incidence careers that match the size and so what I can

calculate here is how many successful attacks I have pair adversary that make sense that basically tells me that each one of those 30 adversaries is having at least one successful attack period now it's be taking those records and selling them in the black market in the dark web they will get at least $10 per record so bring an adversary one of those 30 plays off now after each and every of those incidents we have an incident reports an incident reports describes what has been done during the hack try to get what kind of evidence Clues similarly to those forensic evidence from a chronic crime scene and those are called indicators of compromise which the mirror with in the case of compromised

the resin hand great so as you know just this could be IP addresses domain names URL and so on those will indicate a probable intrusion in your network so you want to get that list and we're going to give it to you there are many many many threat intelligence providers out there from open source to commercials to private community all those create what I call a virtual threat library that's kind of answering most of the questions you have regarding threat intelligence and you can go just like in the library to each one of them and try to get the relevant answer to your question now threat intelligence platforms can help you in here since there are so many providers

you probably wants to cross correlate between all of them and look at them under one single pane of glass and so as a threat intelligence platform company we have visibility to all of those indicators and we see that an average indicators we get from each source is anywhere between three hundred and ten thousand indicators per day if you do the calculation how many indicators you get pair ear just the bare minimum I'm taking 300 a day and let's say I'm subscribed to four sources I get almost half a million indicators pair ear now from a security operations perspective this is not practical it's not manageable you cannot push half-a-million indicators to the sensor grid such as the firewalls or ideas and

from security threat intelligence perspective it is not practical as well because there's no way you get enough time to investigate so many indicators so we have to do something and my first advice is using classification so we cannot talk about application without food right so let's see which what we had so this was in st. Maarten I took this photo and if I ask you which restaurant this came out out of is that a French Dutch Caribbean it's really hard right what if I tell you that st. Maarten has half batch and half French side so I'm going to help maybe but still classification could be a very hard task because classification is the process

where you create categories and you decide which categories you need to put each and every item grouping is the next step while we decide which classes you work grouping is taking those items and just assigning them to the right category but classification is very hard and it is hard with retic diligence as well and so the different classification with threat intelligence feeds by the vendors are by eye by type by indicator path right we have IP addresses domain names URL etcetera when I suggest that classification using attributes which are the context behind those indicators can help you reduce the noise as an example we can look at a malware family if we create grouping by malware family

we can create grouping by the kill chain process so identify each indicator which phase within the kitchen they're related to we can identify space where geographical location languages and so on another way to classify is looking at target attacks and looking at adversaries that are after finance or after health care after infrastructure and then identify our industry and look at those that are relevant to ask incidents or events those are indicators that all came from the same event or the same incident so regardless of the type there kind of part of the same kill chain and I want to put them together and see if there's any correlations I can understand what really was the

process of events okay and finally to my first point if you can find out relevant indicators to your organization that would be the best if you can sort your indicators pair the specific devices you using the specific vulnerabilities that relevant to your company that's probably the best that make sense great so the third question research team was conducting a small test around the attributes what we did here is we took four different sources and for each one of the sources we measure the number of attributes that describe the different indicators now it's a little bit hard to see it here but on the left-hand side those are the different sources and you can see some

of them give 12 attributes one of them is giving three on the right hand side we trying to cross correlate we're trying to see what is the overlap between the Bender's and unfortunately there is a very little overlap which means that most of the vendors have vendor specific terminologies and ways to describe the threat intelligence and that's a challenge because it makes it hard for us to normalize the data and so there are some community driven efforts in the market in order to standardize threat intelligence communication open IOC and stick taxi are the some of them and what they're trying to do is they're trying to create terms that will be able to be exchanged between an organization

and other having it vendor specific it's going to be a standard however none of them is really adapted 100% yet and even if it's going to be adapted I suggest that they're still going to be noised needs to be cleaned out on indicators so here's one example of a very noisy indicator so this is what our research group was looking and they found that hash values are relatively much more noisy than registry key for example you can see that there are many more indicators in average now this could be a result of having a lot of command and control information and a lot of malware is coming down and hash has been created and as you know

adversaries figured out a long time ago that the defenders using hash values to detect and so what they do is they keep on creating those mutations right a way to go around it is to create a sectional hashing what is the sectional rashing sectional trashing is when you look into the malware and try to create hashes of the specific functions within it and so if the adversary will change a portion of a malware still there's a chance that this time other hashes will match those i/o sees another way is fuzzy hashing so I like regular hashing which changed totally different the hash value whenever there's a small change fuzzy hashing is showing the similarities between two files so it's two files have

a little different between them the fuzzy hash will have a different little as well another relatively noisy indicators we found is the indicators associated with the command and control so looking at the kill chain steps the attorney sends the library relatively has a manageable number of indicators the command and control is very noisy for many reasons first they are a lot of botnets out there that communicating to different servers but there are also adversary created techniques such as the domain generation algorithm which had the malware trying to communicate with thousands of fake domains and so sandboxes will create thousands of indicators around it however none of those domains could be registered and so my suggestion is try

to subgroup the indicators related to command-and-control see which one are a generation based on the domain generation algorithm and which one are a result of an existing valid register domain by the way they are already machine learning techniques that are trying to figure out whether it's a valid domain by analyzing the name and see if it makes any sense in English if it doesn't make any sense in English it's just gibberish high chance that if the results of domain generating algorithm so we spoke about classification let's speak a little bit about racing but we have to get back to a vacation so before coming to the education we have to pick up the resort and like most of you probably do by the

way if you want to know which resort we went to you can come after around let you know the way we did it is probably like everyone does which is looking at waiting reading reviews trying to score and then decide which one of those we should go with right we can do the same thing with threat intelligence speeds so this is the project done by Alex Pinto he was creating the analytic open source project and he had some interesting tests here he calls the novel peak test and what does this test is trying to see is how many indicators are added with the time and how many indicators are removed within time over different types

of threat intelligence sources so this one is interesting you can see here that there are indicators added within time but there is almost no indicators removed over time right which means that this list is going to continue grow and it's also a challenge because we know indicators have a lifespan like I said earlier sorry keep on changing them and so if we're not going to expire the indicator or remove them from the list we're going to end up with the noise at some point this is another pattern here you can see a spike of removed indicators and a little bit added again some quiet time then another spike quiet time and another spike and that could be

a challenge as well because theoretically threat intelligence stream should be steady should be continuous it could be some new malware that is being released out there that will create a spike but for the long run it couldn't have this pattern this might suggest that the source is collecting all the information and not sending it to you in a timely manner which means that you might get your indicators too late and another example of a pattern where this is an extreme situation where the source is adding and removing the same amount of indicator every day very novel very very up to speed so that's one way to look at the indicators now we can use the pyramid of

pain to understand which indicators we need to expire or retire first so the pyramid of pain created by David Bianco suggests that on the lower part of the pyramid has values IP addresses and domain names are relatively needs need to change by the adversary and therefore they're going to be having a short life cycle on the other hand be truth like the malware's infrastructures and tactics techniques and procedures who are more behavioral indicators those are harder to change by the adversary and therefore those are going to be relevant for longer so that's one way to look at it this is another interesting test which called the overlap test and here what we try to do is we try to compare multiple sources

to the other sources and we compare the source to themself in the middle here and 100% overlap is obvious because when we compare the source to itself is 100% right but look at the indicators coming in from multiple sources compared with other sources you see all the white Shari here represent zero or close to zero overlap between the Bender's which means that the multiple threat intelligence sources they all come to us they are all telling us we have the best threat intelligence that you have or you should have and none of them has an overlap which brings up the question which one should I go with right they're all telling that they're good it says that were scoring gets into play

that's what scoring can help us identify eventually which sources are relevant to us and there's no right and wrong here all those sources might be relevant to specific companies you just want to find those that are relevant to your organization and so what are the scoring key materials well indicator parts we spoke about it there are indicators that have a little bit lower life cycle but also your organization life might be more driven towards host-based indicators rather than network based indicators sources generally speaking or intuitively you might want to score commercial sources higher than open sources because you pay them right and then maybe community-driven that are specific to one sector or even higher than

commercial another score can be pair attributes so like I mentioned earlier all the indicators coming in with attributes and so at some point you will get a feeling on those attributes that's really contributory investigation versus those that are not and you can have an indicator with twenty attributes that's not helping you at all or an indicator with two attributes that are the ones you really need and finally adversary similarly to what I mentioned earlier you want to score based adversaries and pairs target so marvelous is interesting with every one of sort indicators pair modelers but what do we do with that this is an example of the mowers and the number of indicators associated with them and the

first thing I want to note is that the number of indicators associated with the map a malware doesn't say anything about the popularity of this malware doesn't mean anything for example we know lucky was malware number one for ransomware during 2016 you can see here we don't get so many indicators around it but j.strother is just noise the type of dropper and it creates a lot of indicators around it so the way to score is definitely not based on how many indicators you have around the malware but rather what is the distribution mechanism is it a weaponize document that this malware going to target you or maybe the vector is exploited is that malware piece responsible for actually

installing dropping what is the process within this whole malware installation that you're looking for those are the criteria that relevant for you and those are just three additional ideas that I believe we can have a whole new talk for each one of them on what scoring could be based upon so I personally believe that correlation with your internal tools like the theme or the user behavior analytics is a great criteria for high scores why because if you have a specific indicators that found within your theme and the same time there's an external source telling you that this indicator is bad either it's the post positive and then you can take it out of the way or it's something really bad

happening in your network right you definitely want to know about that signal-to-noise with time you get a feeling on which indicators are more noisy like I mentioned earlier earlier using signal to noise will help you reduce the amount of work of your threat intelligence thing and confidence one of the things you have with everything to do with threat intelligence is confident how much you trust this sort how much you trust is indicator being scored by one source or being scored by four sources is the margin of error ten percent or 40 percent you want to go with the one of course with a higher confidence and the lower margin of error so we establish the scoring can

help you now question when I say umbrella what's the first thing that comes into your mind rain anyone else Cisco umbrella drink with umbrella very good now that you referred to the umbrella you're going to see now that the umbrella I think of when I'm on vacation in the Caribbean on a beach so the point is context is very important right and similarly with indicators you want to have context if you have an IP address and the IP address is related to spearfishing it's totally different than an IP address related to a command-and-control right and so context is very important unfortunately we're not always clear in the context there's many reason for that but there are known

unknowns and we should be aware of the known unknowns why we don't know everything about all the incidents well to begin with organisations not sharing all the information they have about their incidence right mainly information that undermined their reputation of vulnerability they will not share everything with you they will share what they feel comfortable with the other things are steps within the kill chain that you know should be there and you're missing the information so an example for spearfishing I know there was an email address I just don't I know that there was a file and I know that there was a subject I just don't know what the email subject was and so I have an idea

what I need to look for since I understand the the process of the attack and statistics tells us there are somewhere around 40 percents unknown techniques used out there whether it's a zero day the tax you don't see out in the world because they are targeting only you or just you technique the attackers are development the same way we developed our new defense mechanisms and so in order in order to answer those sorry you know in order to answer those questions we use enrichment tools and the way I see enrichment tools is like this lifeline in Who Wants to Be a Millionaire television show we have different ways to feel at the gap right you can have the audience question for

the audience which is kind of crowdsourcing so an example of the audience is virustotal anyone familiar with my throat oh yeah so what you do you upload the malware then you ask multiple scanners what they think about it and you get your answer then you can decide what you do with it the friend or the telephone friend I think they call it that's when you have a specific question let's say you have extreme rats and you want you had a question about the delivery mechanism and you know that specific vendor is specialized in extreme Erised you will go to that specific vendor and finally like I showed you earlier you can use statistics also to reduce the noise so

50/50 is good enough hopefully you'll get better another interesting investigation tool is what we call link analysis and this is example from social network analysis from 911 so this graph shows the original suspect of the 9/11 attack and what it illustrates is all the nineteen hijackers are within two steps from the original suspects coming in in 2000 also by looking at the grass and the egocentric part here we identified the recruiter the leader we've recruited all those hijackers so this illustrate how you can use link analysis in investigation identify who are what are the answers in question was what is the relationship between the recruiters and the hijackers and we can drill down and do the same

thing with adversaries in the cybersecurity world we can also create relationship between transactions relationship between objects between servers between IP addresses so this is an example of big panda which is a Chinese Chinese adversary actor and how they relate to this the Kula malware family and you can see here this is coming in from a seagull and the graphic shows that want to I'm good that the deep plan that was connected to a specific IP that I see was a searcher associated to this domain name who try to spoof well point if they put one instead of the El and well point is the further is another name for anthem and then you can go track back to the

droppers and to the original malware and to the stand book it was running under

so the way I see it threat intelligence investigations is very close to any other investigation you have a cyber security area where you have a crime scene you have adversaries who are the suspects right you have the fingerprints who are the indicators you have the body who is the victim and you're trying to solve a mystery and it's very similar to the game glue which I'm not aware of because I'm from Israel but they told me you all will be aware of and and that that is that is the way I look at it it's almost like a detective work that's why the human element is really important here there are all questions that no one by person with intuition

would be able to answer so one more photo from a vacation and this is the nightlife this is my daughter I didn't take many photos at the the night there so sorry about the the quality but I like it because she's much more live by this band playing outside hotel and what I like about it is that it illustrates how a group of people can create an experience that is much higher than each of those people individually I really like this photo and I think it's also relevant to threat intelligence because threat intelligence teams work in collaboration can create much better results than each one of the individual the collaboration is really relevant for threat intelligence investigation

however I do believe machine learning has some type of a role in threat intelligence investigation in the future so I can see how for example machine learning classification algorithms can create a prediction on whether an indicator is a false positive or not another example is decision trees I can see how machine learning can create some kind of a workflow for the aya team that will be more informative than intuitive for example so we're getting to the end of fabrication and like I said machine learning and human elements is something we need to work together in order to get a more efficient threat intelligence but what as we can learn about threat intelligence efficiency how we can

really measure the effectiveness of our work well the way I look at it is looking at the time - detection in time to respond so if there is anything within the threat intelligence activities we do that help reduce the time the detection of time to respond we did our work we contributed to the technology we help to the defenses and eventually I see the role of threat intelligence is supporting the threat defense effectiveness and said the threat defense effectiveness is basically a way for us to look at any new technology getting into the market if you're looking at any technology at some point it is very efficient like firewalls IDS's then the adversary get ahold of this technology

and then they compromise they found all those vulnerabilities and become much less effective and then the vendors are bouncing back and creating some some type of sustaining kind of mode and so our goal of threat intelligence teams is to push this curve up into the right into the red dotted line and make it much more hard for the adversary to attack us make it much more expensive and less profitable for the attackers help share our information with the security operation teams would use the a symmetric between us and the adversary and eventually identify those exploitations both in applications and in credentials so getting back into beginning threat intelligence is all about this common ground it's the overlap between the

organization goals from one side and the threat intelligence landscape for the other so I want to thank my wife for organizing a great vacation when I think my keys were not so bad behavior and those are the numbers that we spoke about during the talk we talked about how data sets will grow up continue to go how we can use data mining for classification and for scoring and how investigation and effectiveness can help us assess our 13 touches thanks and have a great day

what any type of questions in anyone wants a shirt or a giveaway can come here but but meanwhile questions can anyone bring him there yeah yeah so we use threat intelligence feed to verify the link between threat actors so some of the feeds our indicator base but other ones are also related to specific adversaries the other thing we're doing is we cross correlate so if there are two threat actors that for example share the same infrastructure or there is some similarity between the two mal was coming in from them we can identify it within a threat intelligence platform yeah we're looking at specifically on malware's history of using them malware and the location if they are sharing the

same infrastructure yes yes absolutely so the question was about the Carnegie Hall paper which we already have it as well I was actually thinking maybe to put that as well I think the problem the main problem is that there are different ways to look at threat intelligence I don't think any of the vendors is you know giving you bad or trying to give you bad data but I think each one of them is looking at different aspects the fact that there is very little overlap between the attributes doesn't help it and so I believe that unfortunately we do need to take all those traits and that would one threat intelligence platform that can process them and

filter the noise and get the right indicators that's relevant to you internal intelligence is very important one of the things that I kind of mentioned is that a lot of the feeds were using our internal feeds so it feeds from your end point solution speed from your firewall screen from your seem your user behavior analytics close correlated those with the external feeds will give you the best intelligence thank you so much [Applause]