Tipping the Scales Back In Our Favor

the b-sides DC 2016 videos are brought to you by clear jobs net and cyber sex calm tools for your next career move and Antietam technologies focusing on advanced cyber detection analysis and mitigation or your medical records going and changing your medical herbs but getting a new credit card getting a new drivers license those things that are a little bit easier and part of the problem is that we rely a lot on passwords so I read a Miss posting for 863 moving past passwords so really getting rid of passwords all together because generally most people use the passwords their dog or something it's pretty easy to guess or even you have a multi-factor authentication there's a

difference between multi-factor authentication and two factor authentication alright because when you're looking at the different factors is something you know something you are and something you have if it's two things you know that it's multi-factor ones to factor something you know and something you have I was doing some research in grad school they proposed some ultra secure system and what was the best way to secure we had the architected so why don't we just used three factor authentication something you know something you haven't something you are you know a password something that's biometric and some sort of token why not an ultra-secure environments and also when we were talking to the horizon folks they were saying that in every

instance in every breach that they had investigated when to factor also an if two-factor authentication had been used properly it would have stopped every breach of day investigate not that that's there's no silver bullet but if you do that it makes it much much harder there were some instances where folks it I can't believe this happened even though I had some type of multi-factor two-factor authentication the things that they had an implemented correctly so you have to implement it correctly otherwise we can catch them some issues a lot of browsers store passwords how many people have experts in their browsers no ends down us or password managers and we saw a couple weeks ago that password managers and can also get

breach so what's what's the best thing to do password it's really really tough I read an article someone saying that they are going to have some crazy randomized password and they're not going to remember on purpose every time they need to login something the use of password reset feature so yeah okay that might work with us kind of annoying and getting texts and kind of doing a lot of things but they have identity access systems and password system so you can try to use a single sign-on but personally it does become pretty challenging so how much how much does breach cuts and this is something that gets talked about a lot and people are going in and doing different types of

value assessments and there are different ways of looking so the pointy mom is saying that it's seven on average seven million dollars per breach about half a million dollars this isn't just an average and 221 dollars per record but then verizon says it you know it's a little bit of a sliding scale you get economies of scale as it reaches bigger than you can do some different things so they have some ideas that are going from you know on the average offer for 100 records about three hundred and fifty seven dollars I'm going all the way down to nobody 100 100 million records and start things to start to get cheap other factors that i would

consider is have you done a tabletop SS do you have an instant response plan view plates how are you going to handle key are some things that kind of a little bit outside of security that can help reduce the cost because you know exactly what to do who called good content that you have instant response on retainer even better that way you won't have to wait you can bring them in and get to the rooftops and all this traces back because you know advanced threats are really really hard to define so this is from the mandiant report on the right side every breach that they investigated valid credentials were used they were we have a little time the bad

guys right there for a really really long time so it's a huge gap in in detection and most of the time sixty-seven percent in this study it's not you saying oh my god we've got to reach some three-letter agencies knocking on the door and saying you're David I've is on some Eastern European serve Russia needs to be anywhere right but external entities are note are notifying you that your data is out there somewhere and this is coming from your cyber criminals are looking to make a buck off of your data nation-states this is something it's a little bit part of trying to embarrass the country or embarrass the organization for political reasons and then insider threats which are just

really really hard to detect so that's what we run this it's kind of a workshop where we traced back through through a kill chain and what's happening is an attacker is a brute forcing a website steals a legitimate PDF he attaches malware to the PDF spear fishes and employee the repeated actually has the notion of obfuscation so we can spawn multiple processes so it's a PDF it's a vulnerable application the PDF is going on Hal EMC which spawns an svchost.exe in seeing an SBC votes thought you guys see in task manager pretty hard to figure out which is the right one unless you really get in there oh but the way that we trace it back is because the

svchost.exe reaches out to known bad command control so we catch it with threat intelligence and if you didn't know any better way to say hey let's just go ahead and wipe that machine but if you're going to get really high fidelity information from of your systems you can get down to root also get the example we use a free tool from Microsoft that's called microsoft system on which is a pretty neat because it's able to map the network communication to a process ID and then it's also able to detect parent child and grandchild processes or who's on in what so we catch the perpetrator on no valid threat intelligence and we trace it back through the syslog data and

do it on this one we trace it back when we say hey we know that svchost.exe on the communication that's going out to see two and then we looked at me see that svchost.exe was created from kalka exe and cobb emc was created from the PDF from the PDF we look across that individual and we are simply also have his email data we're able to see hey this came into an email we trace it back to the mail server and we see that five other people also receive an email with the same attachment with the same information inside but they haven't opened it yet so we can go to the mail server and delete that email before

these other systems get infected so we're this is kind of a power of being able to to get to root cause and then we look for that file we see that it's also on your server and we see that some suspicious information on multiple requests coming into a server and that's how you can trace it back but just the importance of having the right data and being able to get all the way back to the root cause and this is why kind of everybody talks about the kill chain but when you're going through that process you're going through delivery exploitation and installation usually it's coming from some sort of human interaction in this example it's a PDF

it could be a browser exploit because you're running an old version of chrome firefox and explorer so are you name all the exploits of running an older job and then your attacker is going to not just get in there but also establish some tools building some back doors and then start to move laterally within the environment which usually most organizations have a hard exterior shell and soft and warm and gushy on the inside and then you can get hop around from one area to another one gather information collect it and then exfiltrated and there are a lot of different ways that you can exfiltrate data you can and usually they're going to do it through ports that they know

are open so what's open 8443 sometimes ICMP so there are ways that you can pad data that you want to exfiltrate inside of DNS data or you can hide it inside of ICMP data you can encrypt it and get it out through 4 43 so there are a lot of different ways you can get data out and if you have the right data and you're analyzing it then you're able to piece together the entire story going from threat intelligence type of information where you know there are paid there's free and it gets a little bit noisy so you have to figure out how you're going to rank the different threat intelligence feeds and give them

different types of priority so you're not getting inundated with information that's heading your way you know firewall information so what's what's normal transmission for this user has this user ever transmitted this amount of data before has this device ever transmitted a certain amount of data before and can we correlate that with other information that we're seeing if we can get endpoint data great you know what processes are being spun under there any rare or unique processes that we're seeing are those automatically being compared against things like virustotal which we know that is going to be a very very high fidelity does that system have a vulnerability for that exploit on it you know it's kind of

trip bringing all these things two together and then also combining some sort of contextual information because you know all the computer systems they don't know that this one server has PII data and the other server has the pictures for the company picnic on it not too big of a deal of it here huge deal over here and just connecting the dots being able to zoom in pivot through all this information really really quickly so that you can perform an investigation and this isn't hey let me write some search click a button go get coffee and come back it has to come back within say about five seconds is the reasonable amount so that you can

perform a deep deep investigation going from you know known bad c2 malware or kind of what are all the different dots and making sure that that you can actually go through and collect them and that doesn't address the issue with insider threats because now you know we don't have to exploit anything there's no cyber security controls that are in there that we're violating any type of you know we're not using malware we're not using exploits so we have an insider has access to all the data and is getting the data out so when can we actually detect that how can we detect that so really there's a new approach that needs to be used so combining the

people process and technology using behavior based components so what's normal for every single user as humans were you know we're creatures of habit most of the time we're going to do the same thing over and over login connect the VPN go on confluence update something that when does the behavior actually start to change and when it changes we want to be able to call that out and then be able to to look at it and analyze it be able to share and collaborate so right now when you look at if you if you look at kind of the dark web and see what's happening in dark IRC channels they're sharing their information really really well they'll

say you know what this exploits available on this server for this company everybody rushes in there it's like a smash-and-grab but on our side when somebody gets hacked they keep all that information and they don't really want to share exactly what they did what happened so on there and they're able to operationalize their attacks and we're helping them by not sharing the attack patterns we're starting to do some things with you know the different ice axe that are out there but that's just I p addresses going outbound and kind of get into it a little bit but if we can get to the point where we're sharing their TTP's or tactics techniques and processes and export them and be able to

share that really quickly then they'll have to rewrite their malware that'll make it a lot more expensive for them and then the go do something else that's that's easier so sharing collaborating is critical talk we talked about analyzing all the data and then leveraging IOC and threat Intel information so you know that we've highlighted the threats are definitely out there the lack of people is is huge so we need to be able to do more with with less people and it has to be efficient so we can't get inundated with millions and millions of alerts in the the target breach they had 80,000 alerts telling them about that attack so that they weren't able to get to all of

them so they just kind of let them let them fall by the wayside and the old paradigm which worked really well for a long time is let's build new rules let's build different types of signatures and then we'll analyze all the information as it's coming in but we can't keep writing more and more correlation rules it just doesn't scale so every day it's look at see what's going out there create a new correlation rule import it and just kind of going through that process the new paradigm what I'm seeing at least is using data science and machine learning to help operationalize what's normal for every user kind of baseline that entire behavior and then start to call out different types of

anomalies and in being able to do that in near real-time is is the real is the big trick there so you need to have a big data foundation and then you're incorporating security analytics and using machine learning what this lets you do is find unknown threats attack patterns that we haven't even seen yet so if I'm able to say that one user is now doing something that's different that he that he hasn't done before and that user is touching one system and a different system we might be able just from the anomalies just because you're deviating from of the behavior that might be an attack pattern that hasn't been seen before so really this is the

focus around looking for unknown threats because we're not really using signatures anymore if you're if you're using you're taking this approach we're baselining every entity users devices applications what's normal for all these different systems and when is there an anomaly and that's kind of what allow you to find different types of unknown threats and when you're baselining really you want to map the different behaviors to different models and there are two ways to approach this two different types of machine learning processes the first one is supervised machine learning where you're going to grab a particular data set analyze it and then you have to train the model I said that there's always a level an

acceptable level of false positives and false negatives so where is that line okay I don't want to get too many false positives but i also want to miss something so where do you draw that line so you have to test it and then you apply an algorithm and then you have finally a product you verify that this is going to work correctly and then you're able to put it into production and this is critical to be able to detect mobile indicators of compromise so kind of looking at it from just command and control perspective attackers that are smart enough malware that's smart enough is going to be hopping their their command-and-control servers so i'll be going from one server

to another one if you're only looking at known bad c2 that's the you'll kind of create some blind spots if you can look at the actual behavior the moment that an insider starts doing things that they don't normally do or that they become compromised because those two things are relatively similar a compromise user and a user that's trying to do something bad they're behaving in similar ways they're doing things that they haven't seen before so in using behavior is really really critical and then finally without actually going in and having to look at the data and massage the data there's unsupervised machine learning so unsupervised machine learning is going to label and categorize the data

automatically without the need for human feedback because remember that the original idea is that we don't have enough time to keep writing more and more relation rules that doesn't scale there aren't enough talent at security folks to go around so creating some machine learning system that you have to train kind of ties back to that issue so if you look you've using unsupervised machine learning it's going to automatically learn it'll know when the model is stabilized it'll know when the model is ready and then it will start to call out different types of anomalies another know I talked a lot of CEOs and enciso and they always say no know thyself kind of know everything that you

have in your environment but I kind of take that one step further and I say hack thyself so really scan everything with credentials or with agents get that deeper level of understanding of what the security weaknesses are on those systems and really pen test it so pen testing is a term that gets thrown around a lot but pen testing is trying to exploit a vulnerability I've had some folks that say they wanted a pen test and then it turns out they just want a vulnerability scan so kind of making it clear differentiation their fish your employees I used to it was so long time ago I worked at a bank in Southern California and I had one of my employees

who was a member service representative she was probably the least technical technically savvy person that I've ever met and we ran a fishing campaign and we fished her we called her she opened up the command prompt and started running commands but it really is once bitten twice shy because from that point forward anything that popped up on her computer she was calling security and saying hey is this okay so when you fish the employees it's not to trick them you know that's not kind of the mentality it's let's make sure that this is what attackers would do and let's make sure that we'll be able to catch those things then provide training you know kind of

just um some common sense goes a really really long way and we provide USB drops so I get really nervous when people are just passing around us he sticks I've played a little bit around with rubber ducky and I see what it can do you can commit you can take over systems pretty easily by bypassing USB sticks around and probably one of the most famous attacks Stuxnet it was air-gapped Iranian nuclear centrifuge how to get in there USB so you us and use these kind of an impressive technology also it goes back to Pentium to MMX if anybody can remember that so USB has been around for at least 20 years and I know like another technology

it's been around for that long but pass around USB sticks drop them in the hallways put them in the bathroom even though that's kind of gross and see if somebody's going to grab it and put it into their computer and let them know like hey you shouldn't have done that don't be putting us v6 or you know put crazy glue on them or something and then social engineering so drive up to to the shop with AT&T or Verizon shirt and the truck and say hey the you know see so or CIO told me that I need to install this it's really important are they going to let him and give are they going to give

him access to the data room or not so pack myself make sure that when you when you're doing this it has to be strategic and comprehensive so when i was when i was at rapid7 we sold pen testing services and you know how many times people wanted a pen test but really the like yeah but don't exploit anything like well that's not that's not a real pen test in one instance there was an organization who we did a pen test against their their web application and everything looked good three months later that web application gets hacked gets breached and they said well yeah you guys gave us a clean bill of health what's going on here and we go back and

we look and we didn't actually pen test the application that was in production we pen tested the test application weep weep and tested pre-production so waking apprentice pre-production is you can't you see is not that it's not an exact copy I've highly recommend pen testing and production do it do it during down hours maintenance windows if not you give yourself a huge huge blind spot sharing is caring so I mentioned this earlier so what can we do to disrupt the techniques that the bad guys we're going to use against us we need a way to export their techniques and then share that so that somebody can import those those techniques things like ice axe help you know solter edge and other

industries and I kind of see different organizations starting to band together whether its large healthcare providers they have a color and are all talking together and I think that's great or organizations and they go by their football affiliations or sports affiliations the big 12 and all those different organizations they get together and they say we should share our information is starting to get there but there isn't an easy way to do that yet but you know we're starting to get the people are starting to think this way because when we actually get this stuff together and we start doing it correctly it's going to make it so much more difficult will pretty much be able

to exterminate a certain variant of malware because they all kind of follow the same patterns and if we're able to say whenever we see this pattern automatically detect it or automatically stop it even better then they'll have to change and rewrite that entire malware or those those exploits this is a something that we talk a lot about at splunk is listen listen to your data have a realistic vision of where you are talk to see so a couple weeks ago and he said you know we're pretty good we feel it we feel like we're pretty good okay cool do you have centralized logging yeah the logs are on the servers like no like centralized logging that means that all of your logs

your server logs should be going somewhere for analysis and for correlation instead of wrapping system by system like it that that just doesn't make any sense and if the bigger you are that should really be something to focus on so having unrealistic visions on yeah we're doing okay when you're not doing the the basics is really really tough and have a path to have a vision have a strategy have a roadmap for the organization for the security organization we talked a lot at splunk that all data is security relevant the moment that you say oh well that data is not that important i'm not going to take a look at that data that's when the

attackers will figure out a way to leverage that data that you're not listening to and use that against you and you if you don't have the data you can't perform an investigation so the locked talk to some law enforcement agencies and whenever they go into a place that's been breached the very first thing is no do you have splunk if they have splunk they hand them these searches okay great I run these searches to see if you have these patterns so the bits and pieces of exporting certain a certain things certain searches is starting to get there and this is taken from the sands threat hunting summit on you know everybody's talking about throw hunting right now but it's all you can't

threat hunt everything so where do you start your threat hunting kind of on the left side is ad hoc searching over this computer is really slow right now let's see what's going on there let's comb through the the logs in the data or you've centralized it and you're starting to do some statistical analysis maybe you find a mean you find two standard deviations out where are the outliers you can use visualization technique so put it on some fancy graph or chart or 3d whatever you can get your hands on and see you know when there are there any spikes aggregation is just correlation so having multiple data sources and you know when something with fire and

something on my firewall and a server log of a certain type and I have vulnerability on that system that has an exploit out there for it boom you know those things all line up then this is something that's really really important and then kind of the the most mature phase is grabbing all that data modeling it looking for anomalies using machine learning and data science so kind of where where are different folks on the on the path so very first part is search and investigate from there they'll move on to proactive monitoring and alerting then you have security situational awareness and then finally real-time risk inside where you're able to almost predict the different things that are

going to happen because you've not just gotten the different iocs from different areas but you've built in your own IOC's also and you're starting to start to create your own threat intelligence and that's when you're really becoming more and more proactive so couple takeaways for wrap up here number one there they're coming for us you know the data is there the data is worth money they're in the business they are definitely trying to to get rich off of this machine learning is something that's that's critical when you're able to go beyond correlation and use machine learning as a place to start your threat hunting you can really go much deeper hack thyself make sure that your pen

test and what you're doing around trying to take advantage of security weaknesses and also talking to the different people that are in the organization make sure that they're a strategic incomprehensive and and share share in use so cyber security is not a competitive advantage if you're kind of hiding all the secrets you don't have to say exactly what you did but just different those different techniques the more that we can share that's where it's going to make it much more expensive and much more difficult for four attackers and be realistic of you know where you are and how you could possibly improve this is just a standard splunk side but these are the different services and products

that we offer and if you'd like to talk to me after just i'll be up here thank you