CG - Excuse Me, Your Sword Is In My Eye: Responding to Red Teams and Intrusions in 2019 and Beyond -

so you guys are here to see excuse me your sword is in my eye responding to red teams and IRL threats in 2019 and beyond so you're here with myself Jeremy Elaine I've been doing the security thing for a little while get to work for a pretty cool company doing it so I wrote a ton of great content for you guys so we're gonna move through at a really quick pace so first I want to talk a little bit about hacking history what's changed what hasn't talk a little bit about how modern tech companies have shifted in recent decades spent some time talked about red team's IRL threats Incident Response we're gonna wrap up talking about how it defenders can

prepare so does anybody recognize this book feel free pass us around flip through it good grab it flip through that's a volume one copy of hacking exposed it was published in 1999 there is no digital version of that book I used to see it all the time walking around bookstores do you guys remember bookstores as published in 1999 and it really captures the hacking zeitgeist of the era and right on the cover is a quote from a lf1 amongst other things he was known for smashing the stack for fun and profit basically the first step by step guide to exploiting stack based buffer overflows so this was published in 1996 in frack and I know 1996 sounds

like eons ago it was actually really contemporary to this book and around that time early 90s or sorry late 90s early 2000s there were not a ton of resources out there to learn hacking and security online it mostly looked like this like shady Easons super shady forums like really sketchy looking places there weren't you know threat reports to read or like bug bounty write-ups to read it was basically this and these were sort of the elite like underground hacking groups at that time late 90s early 2000s I feel like really represented the end of an era but at that time inside books like hacking exposed this was the typical view of a corporate network you got a firewall a perimeter a bastion

host the internet and you'd always see these phases of hacking listed out recon scanning gaining access and depending on who published it there would be some steps added or removed maybe vulnerability vacation or Prevost but you typically see phases like footprinting enumeration password cracking buffer overflows clearing logs and stuff like that so 20 years later we're literally still using the same language to describe hacking and intrusions so again some of the steps are added some of the steps to remove but you can see that they all sort of follow this similar formula the similar model so fast word up to 2011 and Lockheed Martin introduced the cyber kill chain and you can see it's really

not all that different from the classic phases of hacking paradigm which has really continued to shape security so sort of back in the day these were a lot of the heavy focus areas and I'm kind of here to say that a lot of modern adversaries and attackers don't need to follow many or most of these steps so let's pivot for a second shout this out if you think you know it blank is the enemy of security there is only one wrong answer the wrong answer is users anyone have any guesses the answer the answer I'm looking for is complexity complexity is the enemy of security unless your company is really really tiny it's probably really complex like

wildly complex hundreds of users thousands of off attempts thousands of servers millions or even billions of logs so when Dan gear Mikko and Bruce Schneier are all saying the same thing it's time we listen so if you're to make a map for a modern tech company it would probably look something more like this maybe something like this it definitely does not look like this so what do tech companies look like on the inside well they're filled with sass they are filled with sass for just about everything you can possibly think of so the whole idea of a perimeter you know that's been gone for more than a decade but it is completely gone at this point

so in modern tech companies Windows is really on the downtrend you know there's always Karen in accounting that needs her spreadsheet macros but we know companies like Google and CloudFlare give Chromebooks to their employees some devs prefer Linux laptops and it's really not uncommon to see large fleets of Mac's in tech companies so this is the typical view of a university basically Macs everywhere but it also reflects the broader trend of Silicon Valley so being able to detect and respond to threats on Macs is increasingly becoming mandatory so modern companies may be built on hundreds or even thousands of micro services so if they're not built with security in mind they can be just as vulnerable as standard servers but I

do find that they tend to reduce the overall attack surface while still letting developers rapidly deploy code and services so modern companies everyone's a remote worker at least sometimes so it's really normal to have your critical apps and critical services available over the VPN for those users that in turn means you are opening up your network to all sorts of users home machines they're infected androids and you're getting logins from unknown places unknown devices all the time so modern companies probably have some form of sam'l or SSO off that this could be something like a Centrify octa auth0 something like that and if you have sam'l you probably have some form of 2fa implemented as well so this is sort of

how companies tech companies in particular have really shifted in the last 20 years let's take a look and see how red teams have been taking advantage of these changes so I call these the not so new faces of hacking the first thing in attacker might do is fish our users credentials fish their second factor and attempt to add their own to fit two-factor device they'll then gain some form of persistence that's not going to get wiped out with a password reset and they abuse the trust and the access they have to gain more credentials and expand access now in the case of a red team they're gonna keep doing this more creds more access until they raise the noise

level until they're detected so let's zoom in on the first step phishing anything that can potentially send a notification to a user is potentially a phishing vector so there's a lot more out there other than just spoofed mail things like public calendars public slack boards or Trello boards public code repos basically anything that lets you add mention a user Lucroy you know me you know me thank you sir and your badge actually will get you into [Laughter] I hope you have some carrots thank you Cheers so anything like public calendars anything that lets you act mention a user and send them a hyperlink is potentially a valid phishing vector so these services tend to let you set your

own display names so when you set your display name you can use it to fit your social engineering pre checks maybe that's expressing urgency or using an emotional appeal but the really dangerous part of this technique is that it bypasses tons of email gateways security tools so it's super dangerous so we know - if a fishing has been happening in the wild for quite some time I encourage everyone to check out this report from Amnesty International it was co-authored by neck's who is a really awesome Italian hacker first of several that I'm gonna shout out in this section here so we know there's tools out there for pen testers as well so mud leashes reverse proxy that claims

Universal to FA bypass it can easily intercept OTP tokens intends to steal users and credentials there's evil in Gen X - it's a real heavy hitter in this space it's another reverse proxy but the focus here is on man in the middling web sessions it has these customizable fish 'lets to help attackers steal creds there's cred sniper from Black Hills InfoSec shoutout to you stay ready and John strand the whole Black Hills crew but you can see it could create some incredibly authentic looking fishes and you can see from this diagram there's a lot more going on than just grabbing browser form data these mere transparent proxies are sending real data to and from the sam'l endpoints so this tool

Maryna was introduced at hack in the Box Amsterdam this summer and it was developed by two really awesome Italian hackers the first guy's name is Opie he's the lead author of better cap which is the man-in-the-middle framework to be using right now the other co-author is a guy named antis natcher he's the co-author of the web application hacker's handbook and he's also a core developer of beef for ten years by a show of hands raise your hand if you've ever popped a shell with beef anybody I love that tool so when these two guys got together they made a brutally effective phishing tool I encourage everyone whether you're attacking or defending networks with 2fa check out

this talk it is super super scary so they released a companion app with marina called necro browser and it sort of handles a lot of things that you'd expect a headless browser like selenium to do the focus here is on automated post exploitation of hijacked account things like automatically backdooring accounts with SSH keys automated password resets so I cannot talk about 2fa without mentioning this blog post from trail of bits I think it is an absolute must read there's so many standards changing in the 2fa space we have Fido u2f web off and there's a lot of subtleties in between them so give this a read for sure I probably don't need to tell this

crowd this but if you're doing 2fa over SMS you should stop doing that as soon as possible I think a lot of us saw the story this summer with this guy lost over a hundred thousand dollars from his coin base account due to a symporter attack so fortnight did a really awesome campaign this year where they gave users a free emote for turning on to FA and I think that's brilliant if you're a defender and you can incentivize to FA adoption you should consider it so another common tactic of red teams is hacking without exploits this is something HD in Val Smith talked about more than a decade ago look I love root shells but ultimately hacking is a

means to an end it's not about those sweet shells it's about the data behind it and you know vulnerabilities and exploits they come and they go but attackers that target the people and the processes are not only more dangerous they're harder to detect so that same year Johnny long gave a talk no tech hacking has a book by the same name where his focus is really about how you can gain access without using malicious code so he's the author of the Google hacking database which takes advantage of these very same principles I had a co-worker Alex give a talk this year at purple con and at crikey con is all about stealing chrome cookies without a

password so the old thinking on this topic was first you get a low-level shell then you need to either fish the users password or use some form of like a local privilege escalation exploit to gain root and then you can get access to the cookies but he realized hey there's probably a way to get this without using exploits check out his talk to see that really awesome technique my main point being if exploits are your hammer everything looks like a nail so something else modern adversaries are doing is hacking without implants by implants I mean things like meterpreter cobalt strike core impact in the case of this screenshot sub 7 so how how do you hack

without implants you use what's already on the operating system you live off the land hence the name wall Ben's living off the land binaries and scripts now this concept was really popularized at Derby con 3 by Mac graver and Chris Campbell the idea here is you use valid built-in operating system tools to do your dirty work rather than dropping malicious code on the system there's no reason to drop meterpreter on disk and have it upload and download when bits admin can do that for you there's even lobe ins out there for Linux I didn't know until I saw this that them can be turned into a reversed shell so even Microsoft security intelligence has adopted the living off

the land nomenclature so there's a lot of reasons to use loll bins mostly because it makes detection a lot harder so talk about hacking hacking without exploits hacking without implants but I want to talk about hacking without compromising in points at all is that even possible so if you're an attacker imagine your target is running a reasonably secure operating system something like cubes OS where every application is individually sandboxed or more likely they're running something like a Chromebook which if you have exploits for a Chromebook you can easily parlay those into cash so in this scenario a user's running a reasonably secure operating system and they're on a network using SSO well remember how we

told users not to reuse passwords and then we implemented Enterprise SSO we made an ultra-powerful credential with access to everything so essentially we've created a skeleton key for the user so that once they've been fished attackers have massive amounts of access and once an attacker has to FA hijack the first place they're probably headed is to the users inbox and once they're there it's all about grabbing more credentials they are going after plane cracks plaintext credentials everywhere they can in the mail messages and ConnectedDrive accounts in documents and again if they have access to the inbox that they can control users password resets so when you have two FA hijacked on a network with SSO the VPN is your c2 you don't

need to have a beacon on a server that's just going to get detected so topic really important to both attackers and defenders is persistence but how do you gain persistence when you haven't owned a computer or a server well all of these apps have varying levels of access to the data in my Google account now Google is really clear about warning users before you give these apps access many of them have almost full access to the data in your account and you can create ooofff clients service keys there's a number of ways to gain persistence probably a lot of people remember this uh this incident from a couple years ago where a Google app named itself Google

Docs and it requested access to manage email and this was spam to tens of thousands of users now Google has gotten a lot better about locking these services down there's still quite a number of ways to gain persistence on a cloud account that's going to survive a password reset so even if you're not using gee suite if you're using something else there's other similar options and other services maybe it's an API key or an integration or a connected app but once an attacker has access to a user's inbox they have tons of ways of keeping that access they can enable mail forwarding malicious mail filters can be deadly they can catch those password resets they can also catch alerts and

warnings from IT and security so when you combine all of these techniques together you make detecting compromises extremely difficult for incident responders so at this point I absolutely have to clarify something I think exploits and implants are awesome attackers use them because they work infrastructure hacking isn't going away but if you're a defender you need to be able to detect these other types of threats that aren't using exploits and that aren't using implants so let's shift for a second and talk about IRL threats by IRL threats I mean threats that pose a significant risk to your company and take serious effort to contain and remediate I'm not talking about someone that pops an XSS and your company blog or does a

sub-domain takeover so unlike red team's IRL threats don't care what fiscal year it is they don't care who's on vacation they show up totally unannounced although sometimes technically they do announce themselves they're probably highly skilled and have access to some O'Day there's not much you can do about zero day you can try to sandbox your apps you can focus on detecting post exploitation but if you do get hit with zero day try to make the best of it if they consider that these attackers might be financially motivated think about the data that you're protecting it might be worth a hundred thousand two hundred thousand three hundred thousand dollars or more likewise IRL threats are often well

funded they can afford code signing certificates they can afford botnet access they can afford VPS and if they're using implants they're probably using advanced implants okay well what do I mean by advanced I mean staged packed corrupted novel form of situ novel form of persistence yes I know some apts are still out there using modified versions of China chopper and modified versions of poison ivy shoutout to AP AP t10 that's not really what I'm talking about IRL threats are highly organized they're not putzing around on your network and numerating SNMP and trying to crack your Etsy shadow password they go in they get the data they're after and they quietly leave they're by far the most difficult

threats to detect and therefore they're the most important threats to detect doing so requires detection in depth so this is sort of the vibe and the feel when you know that you have a confirmed incident on your network everything is burning everything hurts but as an incident responder you have to be ready to switch roles as an incident responder you got to be ready to ride into battle and I feel like doing IR is a bit like being a firefighter not that we're doing anything heroic or dangerous we're not but firefighters focus a lot of time on prevention in education but they all know that eventually sooner or later that alarm bell is going to ring and

when it does you must be ready if you're an analyst you might be seeing dozens or maybe hundreds of lurtz of a day you have to take every alert seriously pulling the smallest thread can lead to something much bigger even if you don't do Incident Response 40-hour a week as your full-time job when it becomes your responsibility it becomes absolutely essential that you take it extremely seriously you have to hold yourself to the highest standards of Investigation all the training you've done all the millions of dollars of security tooling all of it comes down to this responding to a real incident and look I think I'm good at catching hackers if you think you're good at

catching hackers an incident is the time to prove it it's a lot more than a cat and mouse game but I think it's okay for defenders to have a little fun hunting hackers we know they're having fun as well it's okay for us to have fun too and it's really important whoever is leading the incident whoever is drafting the communications whoever's making the high-level decisions it's important that they focus on doing that and delegating work ideally whoever is leading the incident is not doing technical work so let's check again I want to spend a minute to talk about truth with an uppercase T so this is a really weird image if you look you can see that

there's a ship on the water but it almost looks like there's another ship upside down floating on top of it this is what's called a fata morgana it's a type of superior Mirage and it has been confusing sailors for centuries the exact same thing can happen when you're looking at heaps of log data so when an incident cracks off and you're looking through all these mountains of logs you need to be really open-minded about what it is that you're even looking for try to drop your assumptions try to drop your biases the nature of an attack is going to be really really really complex your logs are really simple they're just black and white try

not to lose sight of the fact that your logs only tell you part of the story they tell you the part that you have data for and you know when an incident cracks off you know it's everyone has their own pet theories like I think it's this I think it's that but you need to impress embrace skepticism and change the more you investigate your theories are going to evolve and I know when an incident starts off like I'm like full of adrenaline and it's really exhilarating and exciting but you have to pace yourself because it's likely about to turn into a marathon so logs are the lifeblood of both analysts and incident responders it's crucial

important you have the logs you need when things go bad and there are countless ways logging pipelines can break down try to ensure you're getting all of your logs all the time because you are desperately going to need them one day and yeah the more log sources you have the more in-depth those detection can be remember all the saps that we just talked about earlier you need logs for every single one of these things plus every access to customer data to confidential data every single OS command executed in your company needs to be logged this is something os query can help with every DNS request every email link click and the more log retention you have the better if you

have a month of logs that's good if you have three months of logs that's great if you have six months of logs that's even better the more log retention you have the more of a baseline you can build so let's talk about standardization for a second this is what will really bring your detection in your response to the next level there is a really good reason why militaries push conformity and push discipline it keeps people in sync and it helps minimize errors what am I talking about standardizing just about every part of the incident process the acronyms you're using you want everyone on your team obviously to be using the same timezone same terminologies even down to the

detections in the alerts that you're writing so I had a couple co-workers give a talk at Circle City Con this year it was all about standardized alerting pipelines and really it's just about how we use tools like confluence JIRA get and Splunk to standardize the detections written by disparate analysts so that way we can all read and understand each other's detections and we're all speaking a common language check that talk out so when you're documenting it's really important to document how you found something as well as what you found and always always always include context in your investigation notes you don't know what assumptions you have or what assumptions the next reader is going to have and I think it's really

important to create a timeline when you're investigating an incident not only is it going to give you a holistic view of the incident but you know executives are gonna ask for it and I think it's really important to maintain a list of unanswered questions when you have a group of people working on an incident different people are going to know different things so I cannot talk about standardization without talking about the mitre attack framework this has been one of the best resources in recent year in recent years to standardize attack of terminology now it's more of a knowledge base than it is a descriptive set of phases or steps but I find it incredibly helpful so let's

talk briefly about containment if you look through hacking exposed in old books from the 2000s they describe containment as the second step of the IR process or the third step of the IR process I find that it's completely continuous you may be doing containment from the beginning to the very end of your incident and some credentials are easier to reset than others you know resetting a user's Active Directory password should be pretty simple but imagine if you have to reset you know a service API token that's customer-facing and in production you know how quickly can you get that credential reset if you need to a lot of these things require working with other teams and again

there's a lot more beyond password resets if Ana tilled adversary compromises a user's account they have a huge number of ways to persist they can download to FA backup codes they can change password reset questions they can modify source code so what can you actually do to prepare for this stuff you should consider implementing bug bounty programs a lot of people find value out of plugging programs a lot of people find they have a really high return on investment Atlassian one the bugcrowd program at the year this year in total we have given out more than four hundred thousand dollars to attack the two hackers and the security researchers and we have gotten tons of

bugs remediated so zero trust it's not the easiest thing to implement on a big network it can be pretty complex but the idea here is that it's an IT security model that requires strict identity verification for every user and every device and the emphasis here is on least privileged restricting users to only the data that they need you should be writing detections lots of detections not just detection for low-level hacking tools but you should be writing detections for anomalous behavior and detection specifically customized to your organization and if you're ever having to do Incident Response you should definitely be threat hunting you need to be threatening for the compromises on your network that you're just not yet aware of so

talking about hacking talking about Incident Response it's really easy to get lost in the technical bits but doing security is about working with humans it's about working with other teams so this is something Werner Herzog's says to all of his film students read read read read read read read this applies equally to analyst as well read threat reports read bug bounty write-ups read the New York Review of Books it doesn't matter read as much as possible it will only help you as an analyst and look you need to understand how your company makes money and how it works if you want to effectively protect it I think it's important that you interview users that are affected by an incident so if a

users fished one there is no victim-blaming you treat your co-workers with compassion and respect but most importantly you listen to them listen to what they say listen to their version of events and then attempt to validate that with log data you have to be comfortable working with teams outside of security I'm sorry of a legal privacy compliance all of these different teams are going to help you when an incident does arise you should consider qtr a quantitative risk analysis can help you get an understanding of what your most significant risks are and also what your accepted level of risk is and there's a lot of established frameworks out there that can help with this and so important

as a defender you never ever waste an incident and by that I mean you get the most out of it maybe it means asking for more budget maybe it means asking for more headcount maybe it just means asking to get those old backlog tickets worked on but as a defender you make every incident and you you make the full advantage take full advantage of it as possible so a few takeaways I want you guys to remember from this talk if you're a defender you should feel comfortable thinking outside the phases of hacking paradigm exploits and implants are awesome but attackers don't need them when they can abuse trust and least privilege defenders must be able

to detect threats that do not use exploits or implants you need to have a reliable logging pipeline and a focus on standardization is what's going to bring your detection and your response to the next level in conclusion it's been 20 years since hacking expose has been published guys protect your networks like it's 2019 not like it's 1999 that is my time if we have time for questions let's take them [Music]

or you know lunch I have a mic if anybody has any questions

hey great talk man yeah yeah so I don't focus on security I'm more of a software engineer of course I should be but what's the recommendation that you would give to software engineers you know security was first thing I would say is yeah read read the bug bounty write-ups like whatever your software stack is that you're developing for there's surely bug bounty write ups for people that have exploited that software look at the techniques people are doing if you have any type of filtering there's probably ways to bypass those filters and there's other filters you can make on top of that just read read the bug bounty write-ups read the threat reports read on read read read yes thanks any other

questions all right thank you so much