← All talks

I Thought I Saw a HAXOR

BSides Amsterdam · 201718:5450 viewsPublished 2017-09Watch on YouTube ↗
Speakers
Tags
About this talk
Threat hunting is a proactive, iterative process for detecting advanced threats that evade traditional signature-based detection. Fischer explores the foundations of threat hunting, the practical challenges of analyzing massive endpoint and network datasets, and the tools and mindset required to think creatively like an adversary. He emphasizes the importance of understanding your environment, establishing baselines, and developing the perceptual discipline to spot anomalies others miss.
Show original YouTube description
Threat Hunting refers to proactively and iteratively searching through networks or datasets to detect and respond to advanced threats that evade traditional rule- or signature-based security solutions. But what does that really mean? And what real impact does it have on the security team? Threat hunting looks at a mountain of security data already being produced daily by the traditional monitoring solutions such as netflow data, firewall events and logs. Now include end point data and the events to review explode exponentially. The claim, from various vendors, is that the additional data provides greater visibility but for whom. Traditional incident detection doesn't necessarily take into consideration the endpoint events. Building a threat hunting activity scoped to start with end point data can significantly change the game. This talk is a journey of my experience diving into threat hunting and will cover the principals of threat hunting as a foundation, examine the challenges of working with large datasets that can be generated by end point data and analyse some of the tools claiming to ease this burden. With over 25+ years experience, I have a unique view on security in the enterprise with experience in multi domains from risk management, secure development to incident response and forensics. In my career, I've held varying roles from incident responder to security architect for fortune 500 company as well as industry vendors and consulting organizations. Currently I play a lead role in advising customers while investigating malicious activity and analyzing threats for Digital Guardian. I am also a strong advocate of knowledge sharing and mentoring through an active participant in the infosec community not only as a member but also as director of Security BSides London and ISSA UK chapter board member. (@fvt)
Show transcript [en]

so can everybody read this who knows leet-speak so in simpler terms it's that so I thought I saw a hacksaw okay so this is this talk is about fret hunting and I'm going to go through some of my experiences building a fret hunting type program and also talking to companies that are trying to do for it hunting and the realitor a some of the realities and some of the misconceptions because you'll see you see a lot of people like companies talking about fret hunting and that they're doing for hunting for you and things like that so this is me I'm gonna skip over that if you want to know me just come up during the breaks so what's for it hunting in

practice for hunting in practice is actually an analyst versus an adversary we were talking about actually going into depth about be looking at your your adversary and trying to become your adversaries of thinking like your adversary it's all about being creative and adaptive you know we talk about an instant response you do a lot of procedural based and process based investigations this is a lot more about being creative a lot more about thinking outside of the box and looking for things that you've never seen before one of the things that you need to understand is that it's really about understanding your environment defining what you're looking for and then really applying a baseline of normal to your

environment so to address threat hunting you need to understand your environment you need to understand what's going on in your network you need to understand what's going on in your on your endpoints so you need to have a kind of the bait of normalization of events that you think are supposed to be in your environment versus those aren't supposed to be and it's also about finding really what's evading your automatic detections so you need to it's it's really being proactive right a lot of automated detection stuff that you have already in place in this seems of rules and things like that is it's a reactive type technology right you you're looking at alerting based on something that you

know or something that somebody's told you that's that's but here you're actually looking for things that are outside of that known box you're looking for things that are attacking your infrastructure so you need to understand your your applications your systems analyze the gap between what you think is secure and what might not be secure so you got to think in that mentality of what can be used in my environment to attack me the teams need to get knowledge of their infrastructure they need to understand what's happened what what systems are deployed out there what where the businesses are moving in terms of applications and and what they're doing with those applications you want to identify common business workflows as

well a lot of attacks happen because there's a floor in some of the what some of the business workflows right so you your users are getting attacked because they're doing things that you haven't thought about or you haven't seen outside the box a real key thing to this is contextual awareness right you need to understand the context of what's going on in your what's what the attacker is doing I need to understand the context of what brought this brought what brought an attack on and one of the key benefits is if you're doing it right you're actually documenting what you're doing you're documenting your environment so you're bringing this operational knowledge to the to the

interest to the to the company and organizational knowledge and that's that sounds really good with management right because they're sitting here going well you're putting into place this program but what's benefits does it really bring us well you're actually starting to document your infrastructures you actually you need to do that because you need to understand how the attacks are gonna go through your environment so you know there's a lot of things about perfet hunting the reality is is that you think you're gonna feel like predator but the reality is is it's a person sitting behind a screen looking at a crapload of logs okay so that and I'm talking lots and lots of logs so at the end of the day

you're like that you're not really like a predator so you need some tools to actually get this done most we'll focus on Network aspects of fret hunting today a lot of organizations is it well we've got all these logs coming in from our network devices from our servers but you need to go further than that right so you need to look at the firewalls you need to look at the ids/ips you need to look at your network devices but you also need to start bringing in your endpoint solutions and that's where you're going to generate a crapload of logs right you you feed those into thing if you have to feed those in centrally so what you can use

them there's a lot of good open-source tools too and built-in stuff so you've got bro IDs and passive passive DNS type of queries that you can gather pal shall win win RM but really useful as well because those not only a good for you as the admin or as an IT security person to actually detect but as we saw this morning you can actually use PowerShell to do some really nasty things in your in the in the target environment war is 99% information so the more you gather the better off you'll be the problem is the more you gather the harder it is to manage and harder it is to analyze that's where sims come in you

will not be able to do this without a sim or my favorite right now is ELQ so because ELQ you can actually put in it's a lot cheaper than building a Splunk infrastructure although I've heard some companies tell me that it isn't but so I don't know you know it's like you choose you choose your engine of choice then you start feeding everything into it so typically what we see firewall logs so what you're looking for in firewall logs is unusual IP addresses conscious countries businesses proxy logs proxy logs are really interesting so are you seeing you know the right traffic in your proxy do you see bytes in bi equal to bytes out that's a really interesting one to look

for because biting bytes out means there's something going on if you need if you don't understand why bytes in bytes out is interesting comes to me afterwards and I'll explain it dynamic DNS unique user strings windows logon attempts user added user added additions to privileged groups antivirus logs are useful process Maps or running systems privilege escalation escalations and some of your endpoint detection solutions and that's where I really got into it because you can generate a crapload of data of endpoint solutions right carbon black for example the CBR stuff on a 20,000 Park installation can generate act up to 16 gigabytes of logs per day right so you're gonna be managing a lots of logs

this morning of mitre was referenced right they might attack there's the tack chain here's the problem I have with that is that in fact attackers don't actually follow any attack chain right our colleague this morning mentioned said that you know it doesn't really come in this order but the fact of the matter is think of it's these these attack chains and definitions are really great for reporting and giving management an idea of what happened in your environment but use them for detection is a whole load of crap because you cannot determine which what was the order by just saying oh it was an exploit then it was an infiltration then it was a persistence technique they don't do it that way they

do it the way that they feel like doing it and that's where we come back to being that creative out-of-the-box thinking right you need to think that things aren't happening in a set order in your environment so there's a lot of tempting technologies out there to consider when you're doing threat hunting the idea behind this is that you could want to reduce the load of work that you're gonna have to do or the load of the amount of information you have to go through because you're going to be going through a lot of information so you're going to start applying to solutions where you can actually look at things like reducing the noise so matching against IRCs and TTP's matching

against feeds so if you can match those you've already know that something's going on and you can pass it you could pass it on to the next phase which would be actually incident handling another really good thing is user analytics we're starting to see some really good stuff around user analytics and machine learning the idea behind machine learning is that if you apply it right you can start to weed out certain activities and you can start to highlight certain activities are suspicious that you could then go dig that's the idea but there's nothing better than a pair of eyes in front of a screen the human brain works in a much different way a computer and if you get the right

people to do threat hunting the creative types the guys who can think outside I'll keep think saying think outside of the box because you're not thinking and it's in that process driven manner if you can find those people then staring in a screen is going to be a lot more effective than machine learning so let's just do a test for those who know this please don't spoil it for the others please stare at the screen for like few seconds whoops sorry okay so there is a so Amy Herman she's like she's an MD and she does these classes on and she teaches them to to forensics people like you know like the police type forensic

something like that it's called the art of perception and she has a book on it in it she describes how we don't look at things properly because we've got a prebuilt disposition to look of things in a certain way so we need to retrain ourselves to look at things in a different manner to actually build to actually go back and assess what we're looking at to then describe it and then go back and look at it again see if we've described it properly and then keep going for that rotation until you've optimized the way of looking at things and I think this applies really well to threat hunting because we tend to look for what we know about right and

the whole idea behind threat hunting is what you want to look for something that you don't know right that you've never seen before so you need to teach yourself to weed out that predisposition of actually looking at things in a specific way and you know Henry David Thoreau said we only find the world we look for so I feel if you think oh I need to find that apt group from China that's attacking my company because they want all my they want my intellectual property that's all you're gonna be looking for unfortunately in the back of your head that's what you're going to be looking for so you're going to use some of those that might attack definitions and look

for that specific pattern but patterns evolve things evolve the attackers evolve they start to use different types of solutions to attack you and if you're looking for a specific attack pattern you're just doing insulin response right you're just doing traditional way so this response where you've detected something and you want to go back and look at it so does anybody know what this is those who've never seen this image before I heard it who said a cow it's a cow and that's what I mean by you need to I don't want to get V ology but this is the I that's the idea behind we need to retrain the way we look at things right because a lot of people see

things differently but once you've actually once you've actually looked at it once come back somebody's highlighted a piece of it for you you start to build a better image if I take away the line now you still see the cow right and that's what the art of perception kind of teaches you it's a really good book I recommend reading it because she teaches you she goes through steps on why you're not looking at things properly so where what time I'm going good so what I do is I do a lot of endpoint work right so digital Guardian we have we have an endpoint solution things like that but it generates a shitload of data as well

but I've always been into the endpoint protection because from my perspective you know perimeter died a long time ago and the fact of the matter is the attacker is actually looking for the endpoint right he wants to get to the endpoint if he can hit the endpoint he has access to environment he has access to the server's you can potentially have access to anything he wants so what you're looking for the import is the process of running do they normally run in your environment which ones don't actually belong so if you have standard images in your environment you can actually start to build maps of processes which should be running on an image and those that shouldn't right so

you can actually start to weed out the ones that are running and then actually focus on the ones they should be running which ones are running in privilege escalation or execution typically should only find three or four accounts running on a machine if it's not a shared machine right so you should only find the system accounts plus whatever the user counts name is if there's a secondary user account name or an admin account name you'd have to start to question why is that running at the same time network activity network activity coming out of an endpoint is actually really interesting because it can be quite different from the network activity that you're seeing in your firewalls because

well for one thing if it's going if you've got somebody's put in at or proxy or anything like that you're gonna see the Tor traffic you're not gonna see the applications initiating the connection through that tor tunnel but if you're monitoring the endpoint you can actually see those connections so you can actually see the original destination IP address what the users got trying to go to or that the process is trying to go to and you know a lot of organizations oh sorry wanna cry tourists that's not true you know when I did this presentation a few months ago was like a lot of organizations practice pretty far with local firewalls on their machines then

one a cry happens obviously they don't but anyway if you do have a policy where you're running local firewalls is it on has it been turned off you know look for the command that turns it off it's one of the first techniques that attacker uses to bypass any kind of security more system information so stuck Nets I use stuck to make it as an example because stuck legs actually change some of the kernel drivers right or install additional kernel drivers so are you able to actually detect that with your endpoint solutions do you know if it's except that's actually happening persistence a lot of people when they look for persistence how many of you focus on persistence on the Oinky be

honest okay good because persistence there's a lot of technique seduce persistence right so this you could do things like links inside inside Explorer you can have Service host let's get that scheduled tasks and you can do services as well know what your services are running sorry I lost track of what I was thinking on my endpoint my endpoint I said it earlier the endpoint is the target of the attackers right so then looking at the end point they want to hit that end point they want to hit the admin end point because if they hit the admin in point they might get the passwords well they might have privilege escalation capabilities so the negative effect of that is that you're going to

collect a crapload of data when I say it again it's a crapload of data so what do you do of all this data well you know you kind of have to put it somewhere so you put it in a similar or net or elk or somewhere so that you can analyze it what you can also do is if you have the right type of endpoint agent you can tell the endpoint agent to start to weed out information that you're not interested in so like if you if you create that baseline of your environment you create the baseline of your systems you can start to whitelist we'll call it white listing for the lack of a better term but you can whitelist

certain events and tell the agent not to send them back up or not to log them so you're creating a partial view yes but you're focusing your view on on the information but it's important that you actually need to have available so what services are available so some of this list is old because they did this about a couple years ago but each different agent has different capabilities you know you've got bit 9 usually there's a hash file properties registry at carbon black so those two actually work together now Mangin has certain things count attack has certain things there's a whole bunch of others underneath there but for the sake of time I've just put

if you don't want to buy one of those big systems try sis one there's a lot of good work going on around system on that's been spearheaded by Swift on security and there's even some really cool dashboards to analyze as witticism on data and the whole thing is this one is essentially one of these agents right I mean it's just collecting the information locally on your on your system that that's what those people who have Windows environments for Mac OS environments you can just just start pushing Mac Mac OS logs if you start because it's a UNIX system as long as you're tracking the right level of its this mahn you can send up six months

here to your L stack some really good stuff there on sis Mon so what you need essentially once you've gathering all that data what you need to do is start looking at it so you want to start looking at things that are weird so like so this one is hard to read but this one is downloading of an executable from a URL and saving it into temp you've got certain IP addresses going there's this is caught off to a server so the funny thing is is like this is where it gets interesting because it's actually a call to a free dot F our public server right so what users website so users website was being used

to actually distribute piece of malware the problem is because it's free dot F are most of you most of your proxies will or most you as you know proxy level security will just ignore it because it's a real valid site and free dot F are you know you probably have white listed because they come up as a valid provider so this is the kind of stuff gets missed a lot of times on the network side and that's where as a fret hunter you find these types of activity you want to go back and analyze it versus what you