Sharing is Caring: Understanding & measuring TI Sharing Effectiveness

honey so how much time do it tell you I but my name is Doug Spindler I'm one of the organizers how's the conference going so far yeah I take the compliment there's another guy downstairs to take the complaint okay is that fair so I take all the glory any questions about the conference we have two minutes while people are still walking in anything we could do better for next year oh I take that as a compliment Holy Smoke what more space yeah yeah you know that's easy to fix pay more free beer yeah okay rooftop gardens beer garden so I got a question let me let me kind of just throw this out here I teach part time

it's san fran city college San Francisco ccsf ccsf has a campus downtown that's about a 58 minute walk it's in chinatown if your local from the Moscone Center so the question I have is would you like to have it there but the trade-off is we can't have alcohol but the space would be free and we could have more sessions and more room you cannot bring it in because this is our education system and we don't want to pollute students with alcohol no no no no that's a big no no no so what I'm hearing from you is alcohol is more important in space space well you know this is a trade-off that we have to think about so

you know do you have comedy night baby okay right alright so my my response is yes there is thurs i think a hyatt or there's some hotels come on this is downtown San Francisco there lots of bars and I'm sure we could get a sponsor and I believe the restriction is you cannot consume alcohol and premises but that doesn't mean that you can't have consumed alcohol off premises and come on premises in other words we have no drug testing for students yet all right so let me introduce I think we're time right is this close enough okay so Alex Pinto very good so far chief data security scientist of Knittel which is like a needle so I'm assuming are you

trying to find a needle in a haystack funny you should see that it actually was a very clever clone you just did I tell you what I would like to see your presentation please take over this is like everyone let me hello hello how we doing this is good I'll try not to move a lot okay hello okay how about now okay all right let's get let's get that is show on the road so my name is Alex Pinto chief data scientist of needle and like the very well presented I've also been doing a lot of research under mo sac project right and this is sharing is caring right and this is the in a way

the the the ending of a like maybe a two year two years long story of research and presentations that have been doing on the overall quality of threat intelligence indicators and what we can do about it to make it a little less sucky and actually be able to use them for stuff right to some definition of stuff really depends on what you're going to do so I'm going to take you guys through a journey right this is a long presentation I'll do my best not to overshoot my time I think I think now that I know that it's long it's gonna be a little bit better and please enjoy and if it's the first time you are watching

presentation for me I apologize in advance and anybody on the twitters right we have a hashtag it's DDT I and that stands for data-driven threat intelligence it's not anything you will find an urban dictionary or anything like that this is completely safe for work doesn't mean it can't be now right you just guys just have to be creative but probably yes so anyway let's get the show on the road so anyway let's this is this talk is specifically about threat intelligence sharing right and its really what I've been I was going to say wasting but that would be unfair I have been spending my time analyzing and trying to to find interesting metrics and things that would work right when

we're thinking about threat intelligence Jeremy because it's all the rage right it's something that's very very popular right now but first I have to take you to a very teeny tiny journey through what I have talked before about threat intelligence so then you guys can be in the right context right because like most of my presentations this is a work of spite right I make a point right and then somebody you know what you're actually wrong because of this i'm like oh really okay here's a year of my life trying to prove that actually you're the one who's wrong sir I mean like anything on the internet so it's important to take you through the story and it's

important for you to get the context of what I'm trying to explore and why why should we care why this is important for all of us so one important point is that this is a data-driven talk right of course there will be a lot of opinions right but these are opinions that came from data that I've actually gathered from several different threat intelligence providers several different tracks intelligence sharing platforms and things like that right so the idea is here and like most of these things when you actually look at the data you get this your your maybe your understanding your gut feeling is validated right but it's always important for us to try to look at the data and the data

will help us be better the data will help you understand how we can improve in all those different things anyway what's the story here what have we done before so a year or so ago maybe two years now I that time gets really funny around Silicon Valley I was working around trying to find metrics on threat intelligence indicators so everybody has a threat fee you can buy buy you can buy you can get a bunch of threat feeds on the internet how do you find which ones are good which ones are bad which ones are maybe more attuned to whatever you need to defend or maybe oh this one is just replaying that one right so you

actually shouldn't have them both so all sorts of different things like because I mean everybody was fighting for oh I have the more indicators and everyone on it and nobody really cared because people maybe were just making up IP addresses I mean they had like distributed systems of honey dark nets everywhere you know I mean lots of IP addresses so the point is let's let's have a look at it let's let's analyze this right so there were there to open source projects that email sack project put forward one is called combine which needs a lot of love it has been like left on the wayside for a while but I find some I've actually found someone

who's willing to help me save it right hey he's over there he's over here somewhere but anyway it's really a Python collection of Python scripts that will get a open source intelligence I mean threat intelligence feeds that are open in finding list you find in different websites and put them in a format a CSV you can input anywhere I mean send it to creds send it to some other no threat note integration though hopefully soon i have my friend Scott over there in the back is going to develop it for me I'm sure but anyway you get the feeds already you use a TI q test which was the real star of the show right which was doing statistical

analysis and all sorts of different tasks to see Oh does this threat intelligence feed updates itself do people clean it up right all sorts of different questions you think like you assume that if you're buying something right the people would take care of the feeds but you know what sometimes you just just look right so anyway there was a bunch of tests right I'm not going to go through the whole story I just want you guys to know that there was an overlap test and there was a uniqueness test and they're actually my cousins right the idea is how much overlap there are between different threat intelligence feeds so when you see a chart like this

right this actually means like the darker the the square is it actually mean there's there's more overlap between a fee that's on the horizontal the fitteds on the vertical right and it's not symmetric because some feeds are bigger than others so one big feet can contain the whole of the smaller feed right but that obviously doesn't work the other way it's not really a transitive right but at least with something like this you can tell how similar or how different the threat intelligence feeds are and this was one of the funniest things on my research right this is one of the most interesting conclusion is the conclusion that from that research I want to share

with you guys which was you know I was expected to be a lot of overlap and this is specifically about outbound data right when i say outbound data it's like places that are hosting malware or c2 servers or your favorite apt threat t nation-state threat actor infrastructure right this is places you go to and they're bad there's very little overlap there's a lot of and this is the flip side there's a lot of uniqueness so maybe ninety five percent of the data from those providers there was no actual intersection right and I'm talking about IP addresses domain names I'm talking about I didn't even look at hashes but hey if you look into hashes this number

becomes even worse right and I think everybody kind of has this sense as well right and I was actually surprised when I did this because I was hoping that like all the paid feeds they would be pretty much more on the same ball right it would be tracking the same actors they would be doing the same work right I know very talented people working in all those companies but no it's like everybody's searching for the same things and not finding enough right and it sighs to uh to a more an interesting statistic clusion right and this is what this is what we were talking about in the dbi our last year which was the fact there's

so little overlap and it doesn't really matter how many threat feeds you get in and steal the overlap becomes very small it actually means that probably our surface right the number of people that are actually out to get us is so much more larger than we were able to discover right because if you just like you you get a sample of the people who are bad let's say and this is what thread feeds are there are samples there are examples of things that have been vetted and i have found to be bad right and then you you just keep getting more samples and more simple and there's very little overlap it means that your population is probably very very very

large right and the idea is that what was a hand up there ah maybe the end you're going to forget it I mean I really want questions please don't forget it so uh so the point becomes when you are doesn't matter how many feeds you got doesn't matter how many feeds you buy that's never a winning strategy there are diminishing returns you're never going to get all the intelligence you're never gonna know all the badness right you have to learn from it you have to look the get the data analyze the data hunt with the data do all those things with just the data doesn't do you that good right I'm not saying don't buy them I'm saying that

just to invite them all and make sure watch whatever you're buying is good for you right and this was the story about the tests so I ended up with this conclusion right so guys it's it's not about how many feeds you have right it's about what you do with it what's your threat intelligence program like how you make it reasonable right and you go into all those things like oh yeah I don't buy a feed before you know what your process is going to look like just get the free ones for now and make it work right understand how you would use something like that before you actually commit money to it right and I was super

happy because I mean this was a good conclusion right it's kind of stuffy like you go home and you like yeah I I did my job today right this was good research but then the Internet happened and I got a lot of constructive feedback and one of the most interesting facts that I got was that oh you're absolutely right about the overlap about uniqueness but you know what that's what threat intelligence sharing is gonna is going to solve and I'm like really I I can't see that so I decide okay let me have a look right why not if I mean knowledgeable people thought leaders rice are telling me that this is what threat intelligence sure is gonna solve

so let's do it I mean at least I can be a thought follower right I can you know yeah right so anyway and I was like why do people like threat intelligence sharing what's deal lure right and then I figure out actually it comes to a very simple plan of information security platitudes right so point number one right everybody has this argument on threat intelligence that the best threat intelligence is the one that you built you generate yourself right and I call this the artisanal handwoven indicator theory right it's like yeah we you know it no this is really good because this is mine I got the knife and I chipped this for ages this is this is perfect

right and I'm not saying that's wrong I'm just saying there's there's problems when you think it like that right and the second one is that the kind of the herd mentality and herd immunity kind of things like yeah if all everyone from the industry gets together everybody's going to be protected right I'm like it okay we're going to talk about that as well and then something magic happens in the middle right after we take to care of all of this and then we're at we're all secure right so the first thing that came to mind when I was thinking about the hand woven stuff is well I don't know how many of you guys actually tried

to make threat intelligence operational in your organization's it's super hard it's super hard because of the volume of data it's super hard because of the volume of threat intelligence data of the volume of log data you have to compare it to or whatever other magical thing you're doing you're pivoting you're running all T goes you are anyway no this hunt found that people do nowadays the problem is there's a lot of articles in the industry of pushback on threat intelligence because it's just too hard to use right and there was this article 44 45 one where it was you know what it's just only for the one percent if you're like I see community if you're

like that just said I see community doesn't make any sense if you are like a big bank if you have oh I have a couple of dozen people to spare to just two tracks hunting all the time sure this is perfect for you but then if consuming the data is for the one percent what is the percentage of people that actually are going to generate good indicators that other people are going to be able to use right so it becomes the question right it's I mean I'm assuming that whoever generates it has gone through the whole process of actually using them so he knows what at least what's good for him right so it becomes a problem

fundamental problem of who are these people who are sharing and who are these people going to be right so we go to the herd immunity thing which honestly doesn't make any sense because the whole point about the herd immunity thing is that you are maybe you were vaccinated and most of the population is vaccinated let's let's call vaccinated you got the IOC right but I mean it's just because my office my bank's office is like beside the other banks office doesn't mean that guy is protected right because the internet kind of connects us all not through like specific space constraints right so everybody's searching everybody and touching each other germs all the time on the internet so that doesn't

really work like that everybody would have to have all the iocs all the time I'm not saying that's impossible right and that this is something all of some people to be able to speed up the sharing and make sharing more pervasive right and I'm an honest believer that this should be the goal I'm just trying to point out here that this is actually super hard to do and we're not even close of figuring out what those things mean right so it's very easy to go into platitudes and just think that oh yeah munities yeah and verticality you know and and and get lost in the details they like everything in security the devil is in the details a hundred percent and

anyway the rest of this is not important so let's let's let's let's let's handle the yeah I have to think about the time right so let's let's let's talk about the question marks right I think the question marks to the really important part because what are we sharing right and here's the thing if you if you look at the way threat intelligence sharing communities have been put together they I mean they have existed forever as long as I that there has been security there has been sharing communities that has been closed mailing lists there has been forums right and this is something that we inherited I mean everybody so it does the soil threat intelligence what does

threat intelligence even mean blah blah blah you should be an intelligence analyst we inherited this from the intelligence community and the way their trade craft or whatever fancy word you want to use there is actually around you have very close trust groups that people who know each other people who can that for each other right and then oh I'm going to tell you that this bad thing happened right so then you can act on it right but I'm doing this as a trust thing right I know who you are you know who I am right if this gets out everybody is screwed in the same way so I mean there's a lot of things there

right but and this is the culture that permeates this to this day the problem is now we're trying to move to a more something that's not analyst centric to something who is its automation centric right we're trying to push this ioc's as fast as we can and that's the work that sticks is trying to do right and I mean you can see whatever you want from stakes they are trying right they are trying to put together a standard so we we stop sending emails and csv files to each other and that I I I mean I applaud that that is amazing someone has to step up and create it like the first version which was terrible now the second

version is going to come out it's going to be a slightly less terrible and open hopefully we'll get to like okay this is almost fine but we have to start iterating we have to start doing this and there's a lot of and there's a lot of like people oh you shouldn't just be sharing IP addresses sending IP addresses to each other or domains or things like that you should be doing the TTP's you should be doing the higher quality things but then these are the same people who they are they don't have enough analysts to do anything with that right it's it's almost like a job protection thing I'm the only one who can understand what's

on the mailing list and it's it also becomes it also becomes uh-oh let's just not do ioc sharing but then sticks is too complicated I just want to send an IP address so nobody has any clue what they want right and again as a human as a human field of study this is perfectly normal and fine but we just have to be aware where we are trying to put our ships where we're trying to understand where we're trying to move forward as we do this this this all sharing thing and again this is the the trust piece that I was talking about and this is also a cornerstone of the of the of the thing

on one hand we get all these things that was the the Caesar bill that passed everybody should be sharing ioc's with the federal government right everybody should be sharing sharing is amazing but then it's all permeated about the circle of trust mentality right and it's very hard to reconcile those two things right these are two completely different ways to see the world right that somehow have to be reconciled for us to be successful into this right and one important thing that people don't realize about trust is that is the two-way aspect because not only I want to be able to send something to a group and I know that everybody on the group are people I trust so that

they will consume the data properly they will take good care of it they will not I don't know leaked my source whatever you want to call it they will not tell the threat actor that we're on to them but on the other hand that creates liability right because me as company whatever I am Oh company wherever is sharing those IOC's you know what you think they got owned by these guys you know and there's there's there's definitely a legal argument there and one there is a significant barrier as well as far as legal as far as being able to put your name into an IOC and that ties into the whole trust aspect of it but on the other on the flip side of

that if everybody just shared anonymously right if you Neal okay Alex does a kind of a good job when he does the IOC stuff so anything he pulls any time you post something I'll have a look right but then it just starts coming as anonymous I mean who's this guy why am I going to waste my time analyzing what he put their right and and you know and is it even worth it so you're always in the danger of the other guys threat intelligence team be three toddlers on a trench coat you never know right especially if it's anonymous right and there's there is on the trust aspect there is a reputation thing so anyway

enough about the ranting this was this was me this was a literally me like okay this doesn't make any sense but you got to think about this is gonna think about that you'll think about this girl think about that and I'm like okay let's given this let's try to look into the sharing communities and see if there's something we can learn right and I'm really true again this is all constructive right this is like maybe if we understand that we have those limitations like and limitations are the most important thing in any engineering endeavor that you do right you have to understand where the limits are and then you can start to measure against those limits and maybe

you can find this is how far as we can go with this model right and then you can maybe think of a different model but if every if you just like wishy Hopi that things are going to be great that's we're not helping anyone so let's buy so I got a couple of friends to play it was you probably heard of a threat connect and you've probably heard especially here on the on the on the on the Bayer you've probably heard of the work that Facebook is doing on a thread exchange it's super cool it's an EP it's very API ish so if you don't have a lot of developers you might struggle a little

bit to get started with them but they're they're doing good work for sure and some other folks and want to be identified that's perfectly fine we love you right but we love these guys most because you know they they like me because yeah you can talk about us I know you're not going to say anything too bad if they only knew but anyway and special especially facebook i find it funny because everybody was like so outraged when facebook started the the threat exchange thing oh my god i'm gonna send my data to facebook and facebook is the devil guys guys sharing communities our social networks right and maybe this is the thing that we are not making the link in

our heads right the more you understand that the sharing community and the incentives of sharing communities may be a more aligned with your Facebook feed and your twitter feed then you think right there may be some things we can learn from this in order to make the sharing experience across the board better and people get more engaged you want engagement right and I sound like this kinoti marketing person right now and I apologize for that but the secret is people have to use the system the more people use the system the more people will use the system it's as simple as that so let's start with the indicators themselves right so some of these some of these sharing communities

I was actually a part of them so I could look at the iocs some of them were like no you have to be this tall to ride or you have to pay this amount of my name like a not today so they just send me like aggregated in so the ones i could see right I had a look I used those tests that i showed you before so that we can try to understand try to at least get up taste right an idea of the quality of the data so the overlap was amazing actually right and this is comparing against all those open feeds that i was telling you about a couple of private feeds right that unfortunately

you will have to torture me for me to tell you who they are and one of the this was not an invitation by the way and one sharing community of those right and you see that the actual overlap of the data that wasn't the sharing community against the private feeds and against the rest of them was actually pretty low right so if I was a part of the sharing community regardless if I was um if I was having paid feeds or not I was getting value out of it I mean at least I wasn't just getting repeat ioc's or things that Oh everybody knows about this at least people who are monitoring whatever all those open feeds and things

like that right which was pretty much the experiment on the other on the other presentation right and the uniqueness also I mean it's the flip side it stayed pretty high 92% it reduced dramatically from the other one the one was like 98 because I added the private feeds on this the private feeds have an impact but I've never seen it go down 90 so I mean like I said it's not okay i'm going to turn all sharing communities not my problem is solved that's not it but as far as if you're making this portfolio so to speak of threat intelligence feeds and you're trying to figure out how many is enough you know what to me it looks

like if you are in a good sharing community it's pretty much as good as a pig feed right it makes absolutely no difference right the important thing is that the sharing communities they do not want you to use them as a feed they want you to contribute back that's the whole point of the sharing part right so personally I think you have to start somewhere right and it's just by see by seeing the activity right you can let me move away from this by seeing the actor it's very distracting by moving away from the activity from to getting there and being a part of it you might feel motivated and you might feel in able to

share as you see other examples you see other people doing it so based on all of that based on the trust issues I was telling you about and and not my interest issues the community structure issues okay and I try to put together some matrix the matrix I think would be important in addition to the Holte IQ test stuff when you're trying to think about sharing communities right so that's activity diversity feedback and trust right so let's let's see what they look like so the point of the activity metric is is there anyone in there is this on right are people actually sharing right and very soon and this was meant to be super simple it's like okay how many

indicators are shared per day right and pretty soon you start to see the whole trust scenario showing up right because when you look at the smaller group the average numbers of ioc's that are being shared every day is at least Obama well in the in the mean thing it's it's it's more than half it's more than double actually but across every single day as well right I find super interesting the spike on the on the large sharing community there what this probably means is that people were sharing this privately right and then suddenly it became public somebody leaked or it was oh the apt twenty thousand million dropped and navigon oh my god everybody's got a published use it

because you have never have you guys ever realized everybody someone publishes a report everyone else is like yeah we were tracking this guy's too yeah we were so this is pretty much what happened there this is pretty much what happened there so it probably became public right and so there's a huge a huge dump of data so less data more delays more data timely so again it talks back to the whole circle of trust thing I mean we are not worthy at all right there's this dark rooms where these things are going on right and you can see that on the activity right so across the board this is this is only one example right but across the board

you could you could find a relationship as the the small of the group right and you'd see there was a tighter ratio of sharing / / member or things like that right so they don't want to lose control you don't want to lose control who's watching this right it might be sensitive it might especially if you're in the middle of an investigation and I i do i'm not telling people who are in the middle investigation to share their data there's a lot of very important considerations with law enforcement and all those things when you're doing this but even if you're not even it it's just commodity stuff or it's stuff that you see in your network people are

still very they hold their cards very tied to their chest right so let's talk about diversity right and this actually talks back to what I was telling you about around the one percent kind of thing right and diversity was it was very interesting to me because I figured out at least on the sampling of the of the of the communities that I had and some of them they didn't want to share the data but they were like they were okay to like doing an interview and me talking to them so across the board it's like ten percent doesn't matter how many people you have in your community only ten percent will share right and several

reasons maybe they're they're really ahead of the pack right and usually what this boils down to is that there's actually a couple of people on the team who are very vocal they are very like oh yeah no we should be doing this for the betterment of mankind or something like that and I'm trying to be non-ironic here they really want to contribute back they really want to help that right and so they help push the org to be in a position where this kind of sharing works so here and here's a and it's funny because I was and I was talking to people like isn't this too low right shouldn't we have more people when

sharing and I had more than once people tell me you know what I think if it was more people I wouldn't I wouldn't consumed because it's probably going to be crap data it's funny how it sits inside people's heads right that the scar city is related to quality right and again this becomes a very latest kind of exercise only the very few have this capability to share and we can only share of the sprout few right who are doing this thing so i mean the the shirt on the left is the actual numbers like you can see drops like dramatically so i don't know if Michaels on the audience i have not checked if this is a power law

so leave me alone right you you do power law I do this okay and and this is the log thing you see it's it's it's bad right the guy who shares the most really shares the most right so again this is not terrible like if you think back to the whole to the whole social network thing this is distribution of Twitter follower number of accounts by a number of Twitter followers right and there is a correlation how much you tweet to how much followers you have right and for some people it's actually a negative correlation because people see your super dumb and just and follow you but it's usually a positive correlation right so but you see it's the same power

law ish kind of things the same assassination kind of think some people are more naturally nodes there more naturally connectors they're more naturally talkers right and I don't know it gets them off well they like it right they maybe they think they're helping maybe they think that they just want the recognition right so it's normal it's a relatively normal in fact as a human thing so this is where it gets dicey I was like okay everybody's sharing but what is this data right somebody's looking at it right I mean it's the data any good right and here's what the whole thing fell apart it turns out that i'm going to say almost none defensively right now because somebody's going to

raise their hand we actually do that but almost none of the threat intelligence platforms actually provide you a way to give feedback to whomever posting right so if I go in and I have the best of intentions I'm not maliciously trying to mess things up and oh my god this 8888 is super evil I have a lot of machines it's totally it's a DNS X rotation point how many machines are talking with DNS with it right and I posted you know to the to the sharing community nobody says a thing it goes unnoticed I mean and I don't know if it's because people are not consuming it's because i already have that filter so i don't care

right so it becomes very hostile right i mean maybe people are I don't want to bother the guy I don't know what it is but people don't give feedback and this is one of the biggest problems we have it specifically when we talk about the how are you doing now if we're fine I think how we do the the whole bar you must be this tall to share you must be distal to be a part of the sharing community you know generating ioc scan behind but giving feedback is not that hard right I think everyone who has a any sort of monitoring thing which is like more than oh there's just one security guy and he does that for fun

when he doesn't want to go home right you can look at something like that and just give some feedback oh I didn't like this or I think this is wrong or you know what I actually saw this right but it still it's dismal right then again i was looking at so i was i was actually i was actively trying to like what they say i was looking with kind eyes and I was like oh maybe there's an updated timestamp let's assume if there's an updated timestamp it's the same indicator but then the first the first entered and then last updated someone went that I gave feedback so I'm counting that feedback or someone you have you published the IOC and then on

on maybe acts amount of days someone published thats I say my OC again with a different thing I'm gonna count that as feedback right because the the wiring is not there the controls are not there to do the feedback so maybe people are being created still only zero dollar can even read it from here 0 dot 0013 that's like that's awful that had any sort of feedback otherwise do you have just those indicators they're swimming and they're they're like I'm super cool i'm an indicator right i have i'm one of several million we have and nobody knows if they're good or not it and having this feedback process would theoretically again increase the trust on the quality of the data on the

members themselves and this is the this is one of the main things that I think the automated sharing if we can call it like that uh hasn't cracked because this is natural when you would go about the analyst rent centric world because they're talking to each other on email lists so somebody sends oh I've just seen this come by and the other one yeah I was investigating this here's what I've been able to extract right and people just help each other and like oh I saw this other email from the same campaign and then you as again as a trained human analyst right and you know good luck hiring those people you can see the story playing out on the on the

email you can reason through it and you can input it on whatever system you have right but we got it to better than that we gotta do faster than that we can't be like reading emails all the time right so this is a really an open question right I'm not claiming that I have the solution here right I probably would be my yacht right now if I had the solution but this is a this is a question that we must ask ourselves and we must try again driving this thing forward which again ties into the trust metric right which is again what we're talking about let's share for everyone right but what about the people who we trust right so it

turns out that maybe eighty percent is shared privately right if you look at the data there and I was enable of didn't see any of the data but it was like okay tell me how many things are going on on your open community or its TLP white or whatever you want to call it so everybody I just signed up to this to the system right what I can I see without knowing anyone and you look at the amounts that people were either almost direct messaging themselves or they create this teeny tiny private groups where they oh it's just the three of us because we are like buddies and we drink beer all the time and we're going

to share this information here you was massive it was massive but again it was not weird if you think again about the social network construct right and this hum I me probably a lot of you guys use slack if you own a channel they send you some statistics every every week or so right this is how much people have been talking you should stop slacking off for it because that's what the name is for you should actually do or i can stop like work it the owner to stop being on the chat room and you see like seventy-six percent of whatever was said and this is like this is maybe a year old place that i usually spend some time

no no no I actually do serious work at a 72 76 percent was was was the direct messaging or private groups of things like that maybe the environment helps put these people together right so maybe they used it as a tool in order to share amongst themselves I mean they use the machinery but again this is not the pervasive sharing that we think we want to build or that we're claiming that we're building on all those things right so again this is important to understand what the limitations are right it's still humans who are going to make the decision if this is going to be sent out or not the good news at least is that

the data quality was a little bit better on the on the on the private stuff and I mean I mean that very flimsy right I'm actually exchanging hashes for IP addresses I'm not sure how familiar you are with the David Bianco's Pyramid of pain he has a whole talk about a whole stuff he talks about hunting and about the idea of the pain is that how much pain you caused your adversary when you figure out different parts of their infrastructure so hashes are worthless because you just run the the packer again and it's a completely different thing d IP address is a little bit step up right because actually you know he had to get that IP address and a

bulletproof provider he's going to have to call the guy and get another one so it's a little bit but so were exchanging happy ashes for IP addresses which I think it's good witch dies again to the maybe higher quality things are happening the back channels then what we actually see out there right so anyway don't feel bad about the eighty percent there's a lot of instances where you have to do the sharing privately right but we just have to realize that this is what we're up against this is the culture we were up against it's not a culture of sharing with everyone it's not a culture of let's help the community it's a it's a

culture of we are we're a group here we kind of trust each other let's just pass some things onto the table right so if the good data is not getting there how can we reconcile right so anyway final stretch guys here's my take right and the things we should be thinking about if we really want to improve this and really like make me wrong and make that guy on the internet right that threat intelligence sharing it's like it's going to solve all of this right so I think it comes down to two things uh one is the trust right how can you increase trust how can you make trust more and again automatic is a terrible word for

it but how can you make it may be more transitive maybe you know you can make again the actual amount of trust points you have to have against someone you know it's actually a little bit lower right and also the technical barrier the technical barrier is a real thing right so and these are the two main things that are keeping people from sharing right I other don't know how to generate something to share or my legal is gonna kill me if I put it there so when you think about the trust thing right I think there is there are interesting experiments to be done with anonymity and reputation right and again if you if I send if I submit something anonymously

you're never going to trust who's this guy I mean how did why should i get this IP address no it's probably it is he doesn't know what he's doing right but then if whatever community are at there's happy reputation system on right I don't know who this guy is but everyone here has been looking what he has been doing and so far he hasn't really [ __ ] up so maybe you should give him the you know should give me the the extend the benefit of the doubt right maybe I should start ingesting there is data right and so reputation systems with anonymity is it's kind of an understood animal right I think a stack overflow

does an amazing job right I mean the actual the people who are moderators there they usually they are usually good in curating the goods the good stuff in the bad stuff right I think I think Reddit and hacker news on our cesspools of humanity but but but as far as their reputation system they are pretty good on just letting get through whatever twisted things they believe is reality so it's working right this is not about an objective truth if I agree with them or not it's working the model is working they only see what they want to see right so maybe right we can take this idea this is an idea that we can play

with right as we're trying to put together the these anonymous sharing communities so alien volt got the memo right I'm not sure how much how familiar you guys are with a GX it's a system that they release the a few months ago I again time is super weird here this place and actually so I signed up as needle right and wow this could be anyone I mean there was no like Oh confirm your email here so that's really you anyone could have come in and claimed it was need a war it was me or anything like that there's no real tight control right but then as I started posting things that made sense people started to upvote me people started to

follow me people started to subscribe to the things i was posting right and this became a measure of my reputation of this accounts reputation into the system right so everyone else I'm now when someone own boards right and and this is like this is UX right this is like engagement stuff when people on board when they get in they see oh these are the top 5 contributors maybe these are the people you want to follow maybe maybe what they're sharing is good for you maybe what their share is the stuff you're looking for start with these right it can work right and there's a whole flip side of that which is the the curation argument right and this is much

much more common on our environment as well at right now which is the oh I actually have a centralized curation system so i am the owner of a threat intelligence community of some sort and i will do the creation as the owner of that and I mean you got to be able to justify some value add right it's just a [ __ ] CRM on the internet so you might as well do some curation right so there's some interesting experiments on that right I specifically like the way the FSI sec does it right which is they actually have an anonymous tip box anyone can send an email there and say this is happening right and within

reason right I assume that it's not super used and it's kind of limited to whomever as a member they don't know who the member is but they independently validate uh whatever the IOC was good at bad information was good or bad and then they post it to the community like this was an anonymous submission our team at FSI sac did the vetting and this is good info so the info got to everyone who needed right and whomever it happened to that wanted to become anonymous stay that way not even if SI sac knows who they are I mean they can probably figure out if you didn't do the cleansing of the a riot I mean there's all sorts of

details I mean if you want to do this please don't [ __ ] it up but it works there are examples of these things working right but this is very human intensive as well right so fssi it mean you pay good money for FSS act for them to do this for you right so anyway just wrapping up I think there is there's an argument here for the technical barrier right I completely understand that not everyone will be able to produce and share IOC's but feedback is super important and it's easier if you have a shoestring monitoring team you can give feedback and you can even leave a nasty no you wasted 30 minutes of my analyst time

with this crap right you can try to be more constructive of course but especially when you're driven by that anger because you someone actually wasted or at your your analyst time why just not tell the community like yeah maybe this is a false positive so maybe other analysts time isn't isn't wasted right this is not this is not hard I mean maybe the tools are not there yet people haven't actually provided a good you wax for you to do it right a OT acts really between me putting the stock together and today they actually have an interesting wikipedia like thing now which is if you see someone posted something you can actually suggest an ad

it and you can add IOC's you can remove a your sees you can add another comment and it goes to the original poster and the original post reviews it and approves it or not that's a smart way to give feedback right you're continuing the conversation you're being engaged so and finally one thing that it's kind it's not really talked about and I appreciate it super hard to do this right it's still amatory everybody's on the internet sadly right doesn't matter who you are it doesn't matter how good your monitoring team is it doesn't matter the your understanding of threat intelligence or not you are on the internet you have a footprint you have telemetry right and sometimes the

question that people want to answer in threaten tell is as simple as is this targeted only for me or are other people seeing this as well right and that makes a huge difference on how much resources are going to put in right and how much freaked out you're going to be because the Panthers are coming out for you or something so telemetry i believe is really key for us to really unlocking the potential of a threat intelligence right so nido is not a threatening thousands sharing community but these pyramid these are the tenants of the kinds of data that we use in our autonomous hunting platform right and all of them contributes a little bit for us to be able to hey mr.

customer you know all this data you gave me this is probably the story that you're looking for these are probably important things right and this is again this is exactly what the analysts would be doing he would try to figure out understanding if the IOC is good or bad is going to be comparing it to the telemetry seeing if it's it's relevant or not right I really i'm not really sure how this telemetry sharing works everybody's gonna freak out but i have you guys ever heard of the shield from sands this is exactly what these shield is I mean it's just inbound 50 logs I mean you can it's it's it's it's a Norse by any other name right but and I'm not

dissing the data I'm not this in the data I'm not this thing a sans for sure right they do an amazing work but this is still Emmett this is good telemetry people use those lists they understand that ok this is I'm not sure how representative the sample is it's probably a lot of us folks but at least I know what's going on under us more or less at least I can know if I'm the only one being scanned by this guy it's super helpful anyway sharing is hard okay but that doesn't mean you should be discouraged right you should you should care you should you should try to share but the main reason why we're failing my

take is is because we're just trying to lift a human centric a activity and just so let's just build a computer system that will handle that and that's it we're done I built the system and I can just walk away now there's a lot of translation here that we haven't figured out what does it mean to have the guarantees to have the trust to have the quantity that I have in this human centric environments in a way where we can share faster right we need to share faster we need more and more and I'm not going to go into this hole yeah sky is falling bunch of malware but you guys know it's all true right you you're living in the

trenches you know it's all true right and having this data at hand quicker it certainly would make our jobs a teeny tiny easier if we were able to do the day if we know how to use the data right so don't buy feeds just join a trade group I'm kidding but use it as you use it as collateral I mean think about it right this might be the best thing for you usual especially the industry focused ones right they're not about to feed at all they're about they will give you IOC's they will give you indicators to work with but they will teach you they will the community will hopefully support you a little bit right it's all

people who have gone through the same challenges and this has very little to do with the actual ioc data and anyway it's so terrible it's all more data and we still need to work through it so i mean just wrapping up right I'll of data so so if you're a part of a sharing community or oh I we have threatened touches feeds and all those I'm always trying to collaborate with falls and do these analysis with them right and understand and maybe helping them I mean if they if they feed their so they think there's some deficiency other feed help them get it better we can all discuss I mean I love the more data we have

available the better job we're all going to do so on all four we having more data and more quality of data right and that's pretty much it I guess we open for questions I think there was a question in the back I hope the guy hasn't forgotten yay hi

okay that's an excellent question so the question was when I was showing the overlap matrix and I was talking about oh this is the threat intelligence overlap what did I mean what did they consider to be threats right so that specifically were IP addresses and domain name domain names ioc's right so that was the data that I was extracting from their fields and I was comparing against each other so if an IP address only showed once it means there were no overlap if it showed twice there was so the question was if there's any process to agree what in what a threat is yeah

so I invite you to go to a threat intelligence panel and ask them so to you what is threat intelligence right because you're gonna get from a panel for people are going to get five different answers no there's absolutely no standard everybody will call anything whatever the whatever the whatever they want so it's like what is threat intelligence whatever you want to buy mr. customer that's pretty much the answer you get across the board so no there is no commonly defined and again a lot of people okay so let's try to take this a different way a little bit less a skeptical there is a people usually define threat intelligence and tactical operational and strategic all right so

the tactical stuff will usually be so there's actually a friend of mine who breaks down the tactical into technological and she's trying to like do the whole oh this is all the iocs moving back and forth automatically since I don't believe that this is a reality yet for real I like to keep the tactical tight just just a tactical and the tactical is this thing that you will put on your IDs you will put on your you actually have something tactical to do with it the operational is more like oh I know that bandas like snow so I gotta make sure to have a lot of like snow melting equipment in my office because their

techniques involves know and things like that right and so it helps you think on their overall strategies and how they get into a company and the strategical one is oh my god what I'm building right now in my company is like super valuable for country X so they're totally going to try to go for me I'm not exactly sure which one of their actors is going to come from me but someone will so I have to be prepared for that i hope i hope i answered but this is like yeah it's a shitshow i'm sorry i'm not the one who's going to solve that problem but some people try to think it like this right

and then and of course whatever they sell is the best one right so that that I can tell you for sure okay are we done we're done thanks a lot guys