BG - Watching the Watchers: Exploiting Vulnerable Monitoring Solutions

good morning welcome back to b-sides this is breaking ground uh before we kick off i have a few quick announcements uh first of all i'd like to thank our sponsors especially lastpass palo alto networks and our gold sponsors amazon intel blue cat it's their support that makes this possible uh because these talks are being live streamed the camera's right there please turn off your cell phones because we are in person please keep your masks on so with that please welcome matt and rock hey everyone uh good morning uh if you saw the keynote this morning uh hopefully what you have over the next hour uh probably not gonna go that long are complementary to a lot of the things that mr hoff had so in my experience every great talk starts off when you insult a large portion of the audience if not all of the audience and uh let's do that so uh there are some of you in the crowd or those watching online uh that either do not have a threat model that you're defending against in your you know home environment your work environment and there are some of you that absolutely have an antiquated threat model and what do i mean by that that you're still looking at the insider threats the nation-state actors the criminal actors uh that wish to deploy crypto miners in your network uh yes those are all viable threats that you need to be concerned about but there's a more nefarious one that we believe you should pay more attention to [Music] and this is a threat that has access to all of your emails it has access to all of your workstations your servers your routers your switches your netflow all of your bring your own devices and more importantly this is one that you probably paid several million dollars to put in the network welcoming it into your environment so what am i talking about i am talking about the security appliances and solutions that are there that you rely on for your security in the event that there is a zero day in the event that there is a compromise of one of these trusted solutions these have perfect access in your environment and over the next several slides we're going to talk about ways that you can even with a zero day you can't stop that zero day but what can you do to mitigate the impact to in your organization when you have an appliance with damn near perfect access to your end price so uh with that uh we're gonna go through everyone's favorite talk compliance uh everyone loves that uh what are some implications of having these work from home uh scenarios uh we'll go through a very common uh exploit vector into your environment we'll have a guy checking for things just looking for experiment yep and uh we'll also talk about uh you know the newfangled uh aiml that people are moving towards in enterprise environments [Music] so on to compliance so these are uh you know if we're lucky once a year people within their environments will take a look at them because it's audit time these are extremely prescriptive they're going to tell you what to do and where to have it but it's going to leave you up to what's in leave up to you is to figure out how to do it securely how do you do it in a manner that is actually not inducing more risk in your environment and another thing that you have to worry about is you know what are you actually attempting to secure with these compliance standards and like i said earlier it's really the old threat model the insider threats maybe some of the compliance standards actually don't actually consider that a lot of it is you know security at the perimeter uh but now there's things like the cloud and it's the compliance standards don't really incorporate that as well and i have not seen a compliance standard to date that actually accounts for when a mandated security solution is the point of vulnerability what do you do at that point the good news is that we can actually rationalize this threat and we can identify that yes we must do these things to stay compliant based off federal state local standards however we can understand you know the implications within our environment and what is not covered and so you can take these you know very abstract very broad uh requirements and build something that's more tailored a complementary checklist that allows you to go in and actually define the individual things you need to be looking for based off of your architecture based off the security applications based off the software that you're running in your environment and then based off that hopefully what you've done is create something that's usable something that you look at more than once a year who knows maybe monthly right and it allows you to uh you know stay ahead of emerging threats because it's internally developed and it's useful so uh some things have happened in the world since 2020. i have seen some compliance standards that have not been updated since 2014 but we are still required to adhere to those 2014 standards uh despite you know things being different at work and so with that matt's gonna talk a little bit about some of those changes [Music] all right good morning everybody so rock presented to you some some of the old modeling and what we're doing as organized as professionals security professionals but how does that feed into our organizational mission right we can isolate ourselves and say my job is to just look at these dashboards look at the systems that tell me day-to-day what is actually happening like with my alerts and with that structure but going forward the challenge as we go through these this presentation is to marry that to your organization's mission to then inform them of new models to follow new threats to advise them on and a way to structure it out so that it's not a burden to you and to go back to the very i gotta go back and rebuild this whole 2014 model not necessarily no right it's laborious to do those things but if we understand the context of the mission that our organization is doing in our mission we can then provide more agile responsive measures to then manage these threats and model them right so there are we're going to give two examples and for one is the work from home model right so we've over the last years with covid had integrated this work from home systems into our our networks now when we did that we took a lot of managed and assumed risk for the end user we're like okay we're going to purchase this resource that we're not going to own but we're going to rely on it to provide them access into our networks right that structure and that management are we are we 100 aware of what the implications of that are right as security practitioners and for me as i was going through this the answer was no right people started managing things differently the net flow was was odd people were touching different things that we weren't aware of because of the structure that we had put in place so our understanding had changed we went back and tried to look at our baseline and well we had to do one but it wasn't as robust no it meant the prescriptive measures of my yearly requirements to then answer those questions be like okay this is what it does this is generally what people are doing here's what's happening well there are massive deviations when we started integrating this work from home model right we had i.t and we had hr accessing the networks externally to internal so now we have to make comparative analysis of understanding what the implications of that are right and start building a model for threats relative to that so then security changed our our context changed and that's a difficult problem to overcome because reframing and re-un and understanding holistically what is now critical versus what was critical is different right it was easy on-prem to say hr you're just going to be doing these things with these things i t you'll do this with that i can manage that i can monitor that but it changed well now we have to support hr accessing multiple different things different people with different software with different devices tying into this vpn concentrator than accessing i i can't necessarily manage those things right now because i didn't take that into account as i was building this work from home structure so as i try to understand what threat modeling is and understanding the data making sure it's one common information model making sure i can understand it and then advise people on it and then also in part is security practices on top of that was very difficult then the impact was massive right it was very hard to then understand okay if they do gain access to whatever our vpn concentrator is when are they going to let us know and who does it impact and how big is that impact going to be to what they touch on my infrastructure right that's a massive lift to say your organization might might have 100 people in hr right and they're they're touching several different things i didn't expect them to do but have i taken that into account to understand the data so what can we do how can we respond at the end of that long road and how can we understand it am i going to say destroy it and rebuild it now that you have the understanding no because it's not feasible it's too much to say hey as security practitioners we now understand hindsight being 2020 we need to tear it down and build it back up well i don't know if you're gonna keep anybody who's gonna be like i just worked on this thing for two years it was fine two years ago why am i going to rebuild it well let's just apply some principles that we can then manage and monitor and integrate to our day-to-day to have a better holistic understanding and one of those principles is just enough administration in powershell right as a security practitioner and having people external come into my environment i want to understand and define what they're functionally capable of doing so i can monitor it and if something is outside of those bounds i can then respond to it in a more agile manner now with just enough administration we can define groups that over time have become bloated right but we can give them the responsibilities of using specific modules or specific commandments to do whatever they need to do right and if they come to us and say that thing i was doing last week isn't working what did you do what did you change well maybe i didn't have an understanding and inventory of what it is you were trying to accomplish so now they're in a roundabout way advising me as to how their interactions on that network are going to impact me right not just as the the manager of it personnel but as an advisor to those that purchase products right integrate products integrate solutions right if we're just building on building on building with with a product and we're taking the human thought and the decision making away from from your your junior analysts or your junior i.t specialists well you're not staying ahead of the curve realistically you're just maintaining the status quo that prescribed measures that are necessary because that's what keeps your job going but it's not progressive it's not keeping you ahead of the adversary in this type of work from home environment right signing scripts is a simple solution as well you can manage and track them right you now understand as let's say an adversary gets into your network and wants to lock down the entire enterprise by pushing out a script to then you know encrypt everything for ransom well if you're signing your scripts and you've denied the access of unsigned scripts you've now imposed the cost of time on an adversary right and now we're building a slog a quagmire into our networks but it's it's a designed one we understand it because we're designing it we're imparting these principles right we need to define who our trusted hosts are when we built the work from home model initially there was a lot of what we got to let them do what they got to do we don't understand how they're do like hr is telling us logically how they're doing it but how are they doing it how is it doing it right well okay they need to touch a lot of stuff so let's just give them access to touch a lot of stuff but when it comes back to that baseline deviation and what they're actually doing have we gone back and refined it because if our adversaries are there they are they're absolutely doing that they're getting ahead of you as individuals right you as an organization use a team because you're taking prescribed measures but not defined measures right and then isolating remote systems understanding why steve has to come in via one jump box and that he told his whole team that he has to do that same thing when we have 17 different right now isolation and understanding to then make more documented refined and defined decisions we can then impart all four of these onto our network and respond more effectively advise more effectively and build out models that aren't burdensome right and then we also don't have to tell our leaders hey this thing doesn't work because when we built it we missed too much so that we can then enable and empower our teams to then be like well this is a new way of doing it we can impart this on there this is a new function or feature that we need and those advisements make what we're trying to do more effective and we don't always have to start over yeah so again you know back to the whole principle of this talk this is not a you know if you do this you'll be successful list this is a subset of what you could be doing but again it's about checks and balances if you are implementing these controls in your environment what it's going to do is provide some mitigation against a zero day against your vpn concentrator but now what happens when one of these have been compromised it's always iterative right so if you are allowing only signed scripts what happens when the adversary can now sign scripts and deploy throughout your network do you understand the baseline the additions the unauthorized editions the deviations from all of these activities someone just added a new host a new trusted host overnight who did it why is there changed management in the environment that's robust enough to account for these anomalies so [Music] so then we're going to go to a more defined model right so when it came to exchange server abuse with proxy log on there's plenty of documentation of when it the the the the after effects of that event right we identified that our exchange servers are internet facing we know that showdown exists we know that there's an api that's going to be scanning for something whether it's an icon with an md5 hash that's tied to a version of an of a vulnerable system or server right they're then going to take that and they'll use cve trends on twitter and be like okay does this apply can i use this this resource is there a poc that exists on github that i can then leverage and use on this on this environment right then they're gonna aim for privileged access and remote code execution right but their ultimate end state isn't just that privileged access it's they're leveraging the peers and the trust relationships that we all have to find because of how the enterprise has to work right we have to support these relationships to then manage monitor keep keep up to date understand the health and welfare of these systems but they're also interconnected throughout my enterprise my admins are going there because there's no fault tolerance we have we have a load balance system with eight exchange servers a primary and an alternate and they're going to these boxes right how are they doing it why are they doing it this way are they using rdp right are they using secure rdp because they're trying to do a two-hop jump without with which is difficult with standard rdp or are they doing it in some other way right now our goal as defenders is not to disrupt those essential services but it is to inform and advise to then protect them and like i said provide that quagmire for an adversary to then impose the cost the the the cost of time on them so we can get ahead of it we can understand it so when things start happening that are outside those bounds those left and rights that we have to find we can then answer them so i looked at a few things for this right and i know i just mentioned like they're using rdp well they need to stop like it's controversial but stop doing it right we can start using openssh with windows for windows right and we can tunnel through the network and use it for both domain joined and non-domain joined right so we updated our modeling we said okay if we're going to be doing these things and we're going to try to stay ahead of the curve how can we do that if we're doing these prescribed measures that were there three years ago our adversary if he's if he or she is there they've been there for a while right they know they're aware so we change we modify and we adapt right now change frequently and i know what you're thinking i know what nist says about steve changing his password is not great practice but we're not telling steve to change his password we're looking at it from a organizational and a team perspective right so regularly updating local administrator and global service account passwords cool and then rolling regularly roll kerberos every 45 days and like these are simple we know that the golden and silver tickets are going to exist we have integrated systems that are legacy that aren't necessarily very uh the question hasn't really been answered right for those systems because we haven't integrated them into our new features but we don't have the team that understands the legacy systems and their integration right so we need to understand that you know just like certain viruses and diseases they're going to come back right so for not paying attention to them in part making making simple solutions a part of our day-to-day in our process then the the argument can be made that it's going to be leveraged we need to go back it's just an example of going back and understanding what was relevant what are we seeing and how can we defend against it when it becomes relevant again right it's iterative and we should be able to understand and document those things and build them up over time now the big picture seam data integration we're ingesting data into our seam solution all the time all the time but what i what i haven't seen in in my job uh was we weren't integrating the it team solutions or the the local defender solutions into these data sets we were saying oh they're doing whatever they're doing we're doing script block logging but are we looking at what it is are they signed and if not why aren't they how do we define those things and integrate them into the seams so that we can then do checks and balances on ourselves so that those that do have access to that splunk enterprise solution when they get access to that admin do they not have access to splunk as well right how can we understand that how can we audit that we need to provide those solutions into what we do or i would say we are the problem we're advising but we're not checking ourselves towards that advisement that we're making right we're like oh hr is doing something weird but maybe sometimes we're doing something weird too and we need to be aware of that now as i said the leveraging peers for trust relationships so hear me out right bloodhound is a fantastic tool at the end of all these things you understand what the adversary is trying to do you know even going back to the principles that work from home you defined it all you have admins running everywhere you have users accessing things you have sessions existing here and there you have all this stuff happening but how do you provide somebody that's so what right i can tell you all day there's microsoft has a bunch of scripts that'll remotely query my active directory pull down these relationships and map them out for me tha