Leveraging Data Analytics to Tackle Everyday Security Tasks

okay I'm gonna go ahead and get started everybody just take this yeah all right my name is Ramiz cave and welcome to dinosaur training 101 getting to know your new pet wait no is this the wrong talk alright just kidding I think I'm funny or I you know as my wife would say funny-looking so let's get started I'm here to talk to you today about eating the elephant leveraging data analytics to tackle everyday security problems and deliver actionable intelligence you know as if our job already wasn't hard enough as it is by day I work as a research analyst for NTT security I primarily work on analyzing packet captures malware covert channels communications the fun stuff

that my wife likes to have me explain when she needs to fall asleep by night you know according to my daughter I'm known as the daddy princess and there may or may not be pictures of me dressed up as a princess online but you know as according to her I will be at forever be Anna and never Elsa but hopefully one day I hope I get to play the role of Elsa and be a true princess alright so we're gonna cover today eating elephant I know that's a long title we're gonna also cover with what is actionable intelligence really it's just a matter of perspective in my opinion we're gonna go over a case study

that I've been working on called the c4 URL list some of you may have seen this and then we're actually gonna go ahead and eat that elephant don't worry I have hot sauce other condiments you know you're free to add as you see fit some people like to go you know oh natural and eat it straight up but hey whatever works for you data enrichment and organization that's something that I feel is very important when you're when you're eating an elephant you know have a plan for how you're gonna eat that meal you know it's like going to one of those big family style restaurants and they've a seven-course meal and you know you kind of have to pace yourself

because if you fill up on the appetizers you have no room for dessert at the end it's like one of those situations twenty one questions the data interview is really when you're getting to know your data it is an interview process you got to learn the right ask the right questions you know really know how to get it get involved with it actionable findings recommendations and we'll have a Q&A following that so what is eating the elephant for me eating the elephant is a systematic process of breaking a large problem into smaller more manageable problems my co-workers have heard me say before you know how do you eat an elephant and I'll say one bite at

a time and I purposely use bite there because I'm a on the packet guy as you'll soon learn a really a packet junkie I had going through a 12-step program to kind of get that fixed it's not working at all but I'm okay with that all right what is actionable intelligence you know threat Intel is a hot topic right now for we could ask everyone in this room what is actionable intelligence and they might have different responses everybody's gonna be right people are going to agree people are gonna disagree it's like okay who's on Team Jacob who's on Team Edward the room will divide you know and then they're like forget Twilight I don't like Twilight I want to

go old school you know and that's fine but for me actionable intelligence are things that I can act upon you know something that we can react to have gray gained a greater insight adapt change of that policy you know these are because this is what this is about because when our boss has come to us they want to know how are we going to save the world in the next five minutes before commercial break you know we all deal with this every day so that's what intelligence it's really relative to the situation you're working on the background story so in 2016 somebody posted a list card that they called the c4 URL list why they called it c4 I'm not sure maybe

it's a cool sounding name it had some significance to them in this list it contained URLs for 24,000 alt devices specifically CCTV cameras that were out there that that he or she had found so for me being a data junkie as well as a package junkie you know because you know I have to have multiple vices I started looking into that and this is a type of scenario that we all might encounter your boss somebody in your organization says hey we just saw this list on Twitter or it came from this feed you know this intelligence feed of IP addresses or URLs we want you to use that to protect our organization and we needed it yesterday and it's like so how

do you go about handling this you know and and you know this this list 24,000 you know it could be triple that you know it could be in the millions and your boss is you know or anyone that doesn't deal with data in that magnitude you know what the problem is why can't we handle this how do you tackle this problem so I have some options for that you know you might be thinking what what am I supposed to do with this you know are you serious was this on purpose and then you find out it was on purpose and it's like wow they really want me to handle this so what do we do from there

well we can eat the we can eat the elephant we can add data enrichment to it it's alright the good okay yeah we can add we add a rich route to it to the URL something we develop a free association of the data you know of the plotlines brainstorming of what we want to look into we tie the data into other datasets to add value in context and let the data talk to you because small steps equal big gains so basically what this is is to the person that put this list out there it's just URLs to them you know to them it's like alright this applies to this IOT camera family whatever information they have they maybe they didn't release

it they don't want to release the context behind it but how do we backfill all that information because our bosses you know whether it be for organization or for our own self-gratification how do we you know how do we add enrichment and context to this and you you honestly you want to let the data talk to you as you start building these datasets let the data talk to you you're gonna see patterns that are going to emerge you know everyone in here will see different patterns based on your own experience you know depending on your background like I was talking with someone earlier there are ransomware specialists so if they they might see an IP address or a

provider that to me is nothing you know it's completely an oculus I don't see a whole lot of coming out of coming out of their way to he or she they're like wait a minute that's part of a command-and-control and I've been tracking them for months so now you can siphon that data off to them and let them work their magic on that all right and so here's the URL list I'm gonna walk around here so here's the URL list so as we're as we're looking at any cats in the audience they're gonna run for the screen as you're looking through here there are different aspects to it that are of interest so you see that

there's counter video JPEG a few other things and it's like well what does that mean does that have any significance what kind of value can i grab from that other than really confusing my firewall or ids system and we're going to go into that where do we go from here you know what enrichment can we add so we're gonna go over some some options here are some possible solutions you could block all of these IP addresses with the associate associated URLs it's doable 24000 it's probably going to have more of a negative impact on your equipment then good and then also really what are you accomplishing by blocking 24,000 IP addresses at that particular time

you know you might need to scale it back so we move on to option two we block all the domains associated with the URLs let's go ahead and do any lookups and see all right well let's block it from here and see what we can catch from there also doable but you're going to have the same or similar impact as you would from option one so option three we start to get creative I'm gonna close my eyes I told the data goes away it's kind of like when I'm trying to get my daughter to eat her vegetables and she closes her eyes and she says no daddy I'm not going to eat the broccoli but

then when she opens them the broccoli is still there same thing here except like just how you guys gave me those funny looks for doing that your boss is gonna look at you and say what are you talking about they just did this on CSI whatever you know the code shows up red so when I open up malware on the screen it turns red and it has a skull and crossbones and so I know it's bad and they just cracked that 1024 bit encryption in 30 seconds before the last commercial break you know why can't you do that and true story I actually had somebody say that to me once you know I found some

encryption some encrypted malware and they were like why can't you break it they're like my brother-in-law's sister's father's best friend's cousins neighbor who works for the NSA says that that's possible and they could be done in like two seconds and they said it's easy he said IT security is easy well if it were easy everybody would be doing it so that really only leaves us with option 4 we eat the elephant and what we do with that is we own the problem own it own it and this is how I like to approach things I own these problems you know a good friend of mine and co-worker gave me a list of 65,000 IP addresses once

and said hey go fine something's wrong with this tell me what's what's in there and I'm like well I've never really worked with a query that big but let's see what will happen because I'm the kind of guy I'll go push the red button I don't know what it does but it's shiny and I'm gonna push it so I loaded it in and I actually came back with some matches about a hundred but there were nine in there that were really interesting because it was tied to a botnet that I was tracking and I had just seen the updates go out this morning and I said well if this is it here are the URLs yours agents that

they're using here are the binaries that are associated and then if they are a part of it they were just updated at 6:00 a.m. so look for 6 a.m. timestamps and lo and behold you know that's what he found and that was following this process was you know stepping through the problem and they're probably people in here they already do this you know they have lots of other methods to tie in and this is not telling you to - hey follow this method tooth and nail like Remy says this is step one this is step two no it's really a guidance a suggestion augmenting into what you already do to make your data that much better so when

your boss comes in and they hand you this list of 10,000 addresses or whatever you've got this you're like okay yeah and then you can't make it look like TV alright so we're going to go what's good about this data because I don't like to say that data is bad you know there there's good bait data and then there's good inch data you know the good is data I might not have a use for right now so it's kind of like you know I don't really need this information but I'm going to hold on to it because you never know a context trouble situation may come up where I need to apply it so store off the data you know have some

way to search it and augment it and I do this all with data sets pandas is probably the standard and the favorite and the pandas people are going to hate me for saying this but I use s frame a lot and that's just coming from the courses through Coursera they use as frames in the in the machine learning courses but the pandas people are also going to learn not like this even more because I will flip-flop between pandas and s frame depending on what I need to do now I'm sure I just created a wormhole somewhere at some other part of the world but that's what works for me so the good parts of this we we see that

we have multiple we have multiple commonalities as I have pointed out with the URIs we have source ports that were in their source port information is good for looking for things on your network because it's not just ingress its egress that we're also looking at IP addresses come on who in here does not want IP addresses I mean IP addresses are cold when people want to when they want to know what they need to block what they need to do what are those IP addresses well we've got a whole bunch in here but what other context and we add to it what other things can we can we extend into that the good issue information these

are CCTV cameras allegedly IP cameras allegedly I don't know I'm taking the person's word for it but we kind of need to trust but verify and then also we have volume there's over 24,000 entries in here so that kind of gives us a good perspective as to what is going on so as we're going through here I'm gonna walk over say I wish I had Vanna White as an assistant right now so this is a mind map for it as I pointed out I'd like to do free association what you guys don't know is my background is in psychology primarily focusing on forensic psychology with a minor in criminal justice so what that means is

when you bang on your computers because your code didn't work because obviously it's not your fault your computer comes and talks to me and then depending on how bad how bad you were to the computer it'll tell me your dirty secrets so I have a thriving practice doing that with with this mind map we start with data enrichment so we've map out the Associated binaries other associated IP addresses industry verticals virustotal because we want to know what viruses are being introduced here geolocation and we can also tie in other sources so the malware archive and IP address archive that I have I can funnel that into their detox which is a primary source of information for me I funnel

that in they really hooked me up with with access for this in a real honey net that's my personal honey net that's out there and then the IP camera list and then here are just some of the relationships that we can look to establish you know and this list isn't complete depending upon how you the individual are going through this this list will change there are things on here I'm sure that some people some people will look to and say wow I can add this column here and we can take that data down a whole nother path and this the list is not meant to be complete it's meant to get you started get that brain that brain just going and

so you can figure out hey how do I want to apply this if you have different analysts like for this if you look at it you can see that a lot of it is focused in on like I like to look at the geopolitical spectrum of how they apply to attacks so if you have someone else that is focusing on ransomware you know or some other attack vector you can you can funnel that to them and say hey what are you seeing from here is any of this data correlate to what you're seeing hey here's a mind map of what we're a kind of brainstorming to get things done and then also another positive is managers

love pretty maps so if you want you can always put something like this in there it does have usefulness it does show you know where your minds are thinking and where you can actually apply other context so we add we start by adding perspective to this we start by adding the industry vertical information now and the geolocation we all know that geolocation is pitiful pivotal to our analysis structure so by adding this in there we know we can identify the top originating country's top originating providers categories for further vertically vertical trends in countries so why is this important you know for targeting and also you're targeting industry if you're a global company and you're seeing that that several of these IP

addresses are mapping to specific verticals at a retail really wait a minute we have a huge retail presence and we happen to also sell these cameras we do any of our customers show up on this list and in what capacity what malware or binary are they associated with also industrial espionage is a real thing these are these cameras are not only just for launching DDoS attacks they also are entry points into networks so that way they can exfiltrate data because I've had those conversations when mail server something's been compromised and they are trying to exfiltrate data and it's like yeah your mail server is it's trying to hack me and they're like it's a mail server it's not it's not hacking

you hit delivers mail it receives mail and so you spend an hour or so trying to convince somebody why they need to take action and that's how I mean just go to show hands how many people have had a conversation like that yeah so obviously it doesn't happen these people they're just supporters and they're putting up their hands No thank you thank you everybody for putting up your hands but this does happen so virustotal in detox virustotal obvious I'm sure everybody here has heard of virustotal adding in collaborated identified malware information because when it when it comes down to what you're delivering your actionable Intel to whomever they want one of the key questions viruses

you know does this map out to anything what can we look for hey why wasn't this vendor detecting it well you can have all that in there and then also your virus detect ability can tell you a lot about what's happening like there is a sample looking at a few days ago that's tied into IOT and it's as of a couple days ago it's only detected by two vendors and they're not in the top when I would consider you know the top heavy hitter vendor list it's not semantics or McAfee or you know Norton any of those it's it's - what I would consider you know the off-brand the less commonly known and that's very telling it's like okay

wait a minute all these other guys say that this is harmless but you have these two let's say it's not and when I'm looking at the structures of the Mauer of the binary itself there's definitely some peculiarities in there and the structure which makes me think that's why it's not being detected so also something to look into that you can tie in my bread-and-butter pcap analysis I am a pcap junkie especially when it comes to protocol anomalies you want to really like the this laser pointer and a cat like throw an anomaly at me and I'm like that cat chasing the laser pointer you know I will zip after it all around the room and I can get tunnel vision and

get focused on it but that that can also be a pitfall protocol anomalies you know I categorize anomalies as things that are out of place but have not yet been confirmed to be good or bad and that what that does is that helps keep me balanced so I know so just because I see a a malformed ICMP packet like the kind you would see in an nmap and for folks that know what I'm talking about looking that pcap that I that I showed you yesterday you'll see what I'm talking about so that ICMP in their accordance with the RFC it's in the original design specifications it does not follow it but just because it doesn't follow it

doesn't mean it's bad that's just something that nmap has done so that those kinds of things when you when you start tracking and identifying those traits those are things to keep into consideration other enrichment opportunities the server headers and the index pages and we'll go into deeper into this later IP cameras like most IOT devices are just web servers that are bundled in a pretty pink box that we paid a lot of money for that's really what it comes down to in its simplest form so web servers are very chatty by default hey I'm running Apache 4.2 or I'm running Apache 1.3 and PHP 4.2 you know for the people that are out there they have the pen testers and

the web coders that just screams hello hack me it was like hey I am open for business come on in and root me I mean that's what that's say these webs these cameras a lot of times they do the same thing there's a lot of poor CGI implementations a BTech just there was a Trend Micro just released a report on AV Tech firmware where they had a wide open CGI vulnerability you know forget going and through busybox we just used CGI and let that do my job for me you can identify configuration URLs and in potentially vendors because that's what it comes down to where what are the vendors that are on our network and this

is how you can apply this context to to what you're working on where are your where are your cameras you know are they vulnerable do you have cameras on your network now most people will say no we don't have cameras on their on their network because they know we don't have a CCTV system set up we don't have IP cameras but I guarantee there's probably at least one user that has set up some kind of rogue camera on their network because they want to watch their goldfish at home while they're away on the weekends that's a that's a that's a security vulnerability that's a way in so now you can start identifying these trends and you can hopefully nip

them before they get get out of control [Music] so data organization how do I organize the data as I said earlier I use pandas and s frames and I also will use database to store a lot of the data ahead of time before I load it into into data frames as of yet the largest data frame that I've worked with was just over 176 million events pointing out to about just under a billion data points and I worked all that with data frames doing that with unless you have a really robust setup it not gonna happen you know at least not in a feasible amount of time data frames help eliminate that and then they also make

the data portable so instead of sending a 17 gig CSV file which that would acquit which that would roughly average out to you can send them a 5 mega byte data frame and say hey here's all the data and then you can segment segment all of the information because this is what I did with each one I'll segment the different the different portions of the analysis into different in two different frames and then I'll build a matrix of all the things that overlapped so in this I had roughly 12,000 IP addresses that I was able to gather information on in the overlaps where I was able to sits high in directly attributable information it came back to about 2000 so that that

definitely on its own help me narrow down the playing field for what I needed to look into and those familiar with intelligence analysis matrix building a matrix is something that happens you know it's one of the analytical techniques that you can do for me I like to split everything apart at first and then combine into a matrix because well I get hit in the head a lot I'm married I'll mess up so this allows me to go back start over and and reimagine and reimagine the data and so keeping everything separated and isolated at first really does help you you know in that context and I've done this with pcaps for me I like to split

pea caps into their different protocols and then any any one offs and then I'll I'll merge them all in all have a common identifier will usually be an IP address so that way I can index everything and merge it it just makes it all easier for me this is a sample of what we were looking at so this is the Jupiter notebook that I was doing all this in and once I'm done with all of this I will make all this available it's all the code all the data you'll be able to play with and analyze on your own so what this is this is one particular company that came up trend wind sa so we

can see when all these are all in particular so this is targeting the arm platform this particular associated binary was hitting just over one hundred one hundred and ninety thousand IP addresses there's the content link from the headers or other things we can trend on hey the server says it's a net wave IP camera there just happened to mean that wave IP cameras out there we see it's from Uruguay at Kaspersky McAfee and Trend Micro have all identified this as Mirai and if you haven't heard if you haven't heard about Mirai I mean that's the definitely the IOT bot that's been wreaking the most havoc out there I suggest reading anything by malware must die on the subject amazing stuff though

the men and women that are there they do some top-notch work and it's it's very well documented and I highly recommend that you read it but that's just a snapshot of some of the information that you can gather from here and keep things moving forward so the data interview Goods as I said earlier this is an interview when you're when you're asking your data questions it's just like when we're being interviewed for jobs when we get cold calls from the recruiters they're like hi I know you're all are security professionals but we have a excellent sysadmin job for you you know and it's 500 miles away for you you want to commute I mean how many of us have

gotten that call it's like did you even read my resume I haven't been a sysadmin for 20 years I mean I thanks I appreciate the call but no thank you so same thing when you're asking your data these questions you ask the right questions you find gold it's just like streamlining your applicants if you know the good questions to ask them from looking at the data their resume you're like wow there is an untapped jewel just sitting right there we need to hire that person because I don't even think they know the full potential of what they can do yet you know but you ask the the wrong questions you know like oh we're looking for somebody that you know knows

you know checkpoint firewalls and they're like well I don't know checkpoint but I know these other firewalls and it's like well but we're specifically looking for a checkpoint we don't have checkpoint but that's what we're looking for you know again those are the types of questions you see and you're really not getting anywhere with that so some of the questions that I usually like to ask my data are what are the identifiers are there any vulnerabilities what vendors are affected and all of these can be answered depending on the context of the data and what you have available so your actionable findings so there are some of the things that I found just from looking at all of the data like what is

all this boil down to okay release you're sitting here telling us you've looked at you've got over 24,000 IP addresses you have you've said you found some binaries you've said you found this and that so what does that mean you know because again we still have to solve this problem because you all are my customers you want to know what are some actionable intelligence that you can take out of this like why are we sitting here well just for some observations coming around here like you know I mapped everything to government retail municipal education Internet service and telecommunications some of these are going to be no-brainers to you you know but municipality I added that

because that's dealing with the particular city because again if you work for a city government you know and are we showing up on there Wow municipality showed up let's see what let's see what's happening there until we show up on that list wait we use that that camera system that that municipality is is working maybe we should have a look that actually happened in before inauguration day supposedly the CCTV cameras around Washington DC were compromised by ransomware you know are you running those cameras you know are you also being effective what are some things that have been happening on your network surrounding that and we also saw 28 different camera versions you know based on that net wave was the most common

coming across 27 countries United States and Vietnam were at the top to the United States no-brainer there we're seeing things across telecommunications and Internet services so that's that's not uncommon but what the differentiator is what I would list as telecommunications those are people that provide Internet service and some other type of service like mobile phone so we have a internet service compatibility and we also have mobile phones so now that leads me to think are these coming from their mobile network you know because we know that those there could be security implications there the top ports eighty six and six sixty sixty thousand and one so when you're looking for these devices on your network you

can start targeting those ports do we have any services listing on here for those services for those for those ports should we have those on here now you can start identifying that and training that out and the binary targeting was across two hundred twenty thousand IP addresses a hundred and eighty countries ten thousand organizations basically they were hitting everything some of them were targeted but immensely it was we're just going to plug in some random IP ranges and we're gonna hope for the best

so what does that mean I like to add data visuals to this so what this cluster is telling us besides making our eyes bleed is that everything is tied into telecommunications because I know you all can read that right yeah so this is tied in to telecommunications internet services at which we saw this we saw that everything was clustered into these two areas which is fine but for me how I like to look at this data I'm not worried about so much about these guys here it's like yeah I know that they're supposed to be there I'm worried about these little guys here those are the people that came to the party and they're just kind of sitting

in the back of the room not talking to anybody like why are they there why are we seeing so many little things from them is there something special about those cameras were they isolated incidents you know if you're if your company deals with retail particularly down here you got retail in the Philippines one of those places appears to be a mall in the Philippines and the mall seems to be full of restaurants so if you're a company like KFC and you want to protect your recipe and you know you've got roaming you're not going to have vulnerable cameras in there that can you know where somebody can jump in and steal that because as we talked

about earlier huh yeah get your no seriously when KFC moves that little piece of paper it's like Fort Knox you know because that is their bread and butter there's so many people that claim that they have the Colonel's original recipe but very few do all right actionable findings so tying into this we can see here all the yellow dots you know these are ports these are all coming from the packet captures these are ports that were grouped together you know hey I'm seeing seven five four seven has its established communications I'm seeing this one twenty-one on ports I'm seeing that like the way this column reads UDP traffic that's uh that's all UDP traffic that I saw in there these

are ports that were established connections ICMP traffic those are the frame numbers of some interesting ICMP packets that I should look at and these aren't ICMP packets associated with with a port not being found router solicitation things like that these are protocol what I would deem is protocol anomalies that we should take a look at which all boils down to this this is what why you're why you're here so we had 51 binaries now you could give those 51 binaries to your team wrote them in your sandbox to kind of develop actionable an actionable plan people do that all the time but with this with the script that I'll give you when I release all the data is I wrote a

wrapper for wire for a t-shirt and I and I just looped through the packets and it came up with this so all so these all four segments here we're dealing with these clusters of six malware hashes so they all dealt with established connections on 2023 666 440 375 47 and 23 23 some of those were used to seeing others is that a backdoor port that we're communicating on the ICMP traffic there since it's not consistent across all of them are those packets what are they therefore it was it something specific to the network that was acquired is it something attributed to the malware but either way it's something I should be looked into because you can

deploy that in your IDs and now you can see where you see you can find out where else this is happening UDP traffic I can tell you from looking at these that is all DNS now you have associated domains you can add to your indicator lists you can start building that report and lastly the the non ipv4 traffic that's those are ICMP e6 I'm sorry ipv6 packets that were coming across also something outside we should take a look at because why are we seeing that you know we shouldn't we shouldn't be seeing that especially coming from here those are the server configurations that we had identified a bulk of them were unidentifiable next was net wave but it

still really doesn't tell me what vendors should I be concerned about you know the previous slide it showed you that they were all identified as mirai or bosch light which you can now tell if we can identify these we know that this traffic without a doubt is tied to these addresses and we can start moving forward and so so so how do we how do we dig deeper into this problem and you figured that out because I gathered all the index pages and going through the metadata and the tags I thought how do we maybe they put something in the title page because vendors like to brag people like to brag maybe it's in there lo and

behold those vendors right there that you see those cameras are all majority Panasonic so those those were the cameras those models were the ones that were that were compromised you know that we're able to point one to one these these model numbers were compromised so if you have these model numbers on your network or if you're selling them you might want to look into that you know wow is this let's see what else is going on here now the other interesting thing and I'm still working on trying to figure out there's a bullet mark camera and I couldn't find too much about the vendor but there was one vendor that showed up where they have a very

high-end infrared camera now if you're using infrared you're protecting something important I mean we're talking like sneakers the movie sneakers you gotta wear the suit walk slowly get your body temp up to 98 degrees in the room so that way you know you can walk across the floor I mean that just from seeing that what are they protecting and then also if you have a camera that's that expensive and it's that old yeah are your secrets still there and lastly one of the things that I was able to figure out because this is the piece de resistance the thing you all wanted to know what were the motives behind this they bad guys why did they

do all this and realistically I figured out that they were all closeted Trekkies and they're trying to take over the world because the data told me so so there you have so now what this is this is where this is one of those examples where data visualizations not really going to help you but it makes it funny but what this is telling me is this is what this with this chart actually depicts are the relationships of all the port configurations the destination port configurations and the United States is at the center of this so all jokes aside you know if I wanted to really look into this I would say okay obviously things are originating from the u.s. in there

targeting these ports now we know a lot of this traffic that is originating from the US isn't actually from the US there's you know it's coming from other from other port other countries but this is what we're seeing and you can see back here we got that one port configuration it's just going in between those two like what's that there so obviously I would redo this and present it in a different way but visualization can help explain your data better especially when you add colors and you keep it simple but when this one showed up because there's actually I probably made about 12 or so different graphs and most of them were just not even worth it

putting that in there but this one I thought this was funny and as I said I like to tell bad jokes and I think I'm funny so the recommendations enrich your data as much as possible graphs and other visuals are your friend they can work with you they can also work against you don't be afraid to be wrong sometimes you know you learn we learn from our mistakes so that's what we want to try to do learn from our mistakes I love what I do and I try not to be wrong as often as possible but there was a situation where I got I was chasing a rabbit down a protocol anomaly hole and I got confused in some

of the data and I made an incorrect prediction but you know normally when you do those things that can be damaging it could depending on a situation it could be career ending or job ending but in this situation it actually helped uncover another flaw as we started to confirm it which we saw that the server that the attacker was coding for they didn't account for something specific in the interaction and so it kind of and it helped us identify wow they're only targeting this version of the software sometimes the cigar is just a cigar what that means sometimes there's no explanation sometimes the data is what it is and then also remember to chase the rabbit

because even the every idea starts from from some crazy notion and that's what we build upon and I want to thank D tux for helping me out with they gave me increased API access so I can gather a lot of the binaries and P caps that you saw and it was a very very helpful and I also want to thank all of you you know most importantly for attending this talk and there's my contact info and thank you thank you for coming [Applause]