Hunting for Credential Dumping Attacks in Modern Windows Environments - Andrew Case

Okay. I'll send it to you. Cool, thanks. From different angles. Let me see. I'll pick the good ones and send it to you. Oh, perfect. Thanks so much. No problem. We have plenty of time. It's not like this morning. It was crazy. Would you mind here for a selfie? Sure. Thank you. Thanks so much. No problem. Great talk. You okay using our dongle? Yes, sir.

I'm Angus Chen. Nice to meet you. Nice to meet you. Andrew Case. Andrew Case. I'm thinking of credential dumping in modern Windows environment. The one before, you know, RPC Windows. So you're pulling your time. I don't know if you were here. I'm here a little, like, closer to lunch. Because it was, it was chaos. Right now, whoa, you're like. I like it. Yeah. The closing, this is great. You're copying things, right? Yeah - I see, that makes sense. I work for my very own company. So a minute before four o'clock, I'm just going to give people's attention so they will start to have a headset. I think Gary has-- - I kind of figured out,

I think people hear the talk, hear the, that makes sense. - His name is William.

Hello everyone. I hope you all had a wonderful break. Here's your opportunity to put the headsets on so you can get the best out of everything here. If you can hear me could you raise your hands? Let me know. Great. Our next speaker is Andrew Case. He is from Velocity and he is the director of RNA at Velocity and the clients include NGA. He's also a co-author of a wonderful book about memory. memory forensics. Thank you. So his topic is about hunting for credential dumping attacks in modern Windows environment. Let's welcome Andrew Case. - Okay, thank you all for coming to my presentation. Today I'm going to be talking about one of the latest research projects that our team worked on.

So as was mentioned in my introduction, we deal, we work in a lot of environments where the organizations themselves or individuals at the organization are heavily targeted and so we always have to keep up with the latest threats. We have one advantage of the groups that target those environments are using the the latest techniques that are out there. So as a team, what we will do is gather all of those tools, whether they're open source, whether we've collected them during our incident response engagements, and we will do these research deep dives where we want to make sure that we understand fully, you know, as much as we can with reversing and everything, how these techniques

work, and then we want to make sure that we can detect them. We chose credential dumping attacks for one of our latest efforts because once this occurs in an environment, things get very bad very quickly. So if you're not familiar with this, there'll be a lot of examples through the slides, but the idea is attackers are going to get access to a system. They're going to be able to extract credentials from it. If it's one end user, sometimes maybe the attackers can't go very far from there, maybe lateral movement can't be done, but that's not realistic in most enterprise environments. Even if you compromise one machine, you can often get more privileged access to other

machines, and once you start talking about things like the domain controller, you have access to the entire network. So over and over again, what we see is once credential dumping attacks are effective, once credentials are taken, the entire network is compromised at that point. Incident response becomes very difficult, even if your team is all senior level experts who have done IR forever, you have to realize that the attackers have access to every account in that network. So you have to, you're not focusing on one account that was targeted. You're potentially looking at every account in that domain. And if you don't have adequate logging, if you're not familiar with these investigations, it's very hard. And

even if you don't work in this space yourself, if you just follow the public news, things that end up on Twitter, LinkedIn, even mainstream media, companies end up in a lot of cases rebuilding their network from scratch. Once every account is stolen, once every resource that you have is accessible, it's not always the case that you can just fully kick the attacker out, so you're starting from scratch at that point. The goals for our research, we always have two main goals. We want to be able to make detection techniques that are effective and scalable. Effective means if you point it at a thousand machines, ten thousand machines, a hundred thousand machines in an environment, you have little to no false positives. It doesn't do us any good if it

works on our test VM, but then you put in production and you get so many false positives that you can't get anything useful from it. And then we also wanted scalable. So again, especially when you have these credential dumping attacks, you're never dealing with one machine. In many cases, you're dealing with the entire environment and some of the clients that we have that's hundreds of thousands of systems potentially in play. The places that we look for artifacts, ways that we're going to be able to detect these attacks and then actually dig into them, there's two places we like. The first one is RAM. If you're familiar with volatility, we've spent many, many years keeping that

tool up to date, making sure we could do memory forensics to recover that data. Memory is where malware is going to operate. And then if we want to be able to look at historical records and information that the operating system's keeping, we look at the event logs as well. So when we do these deep dives on a particular topic, we're looking at memory and we're looking at the event logs. If you're not familiar with memory forensics or event logs, there's one quick slide we'll go over for each of them. With memory forensics, we see where if you look at modern payloads, so especially payloads that you'll see in environments where you have skilled attackers, attackers who don't want to get caught, everything they do is going to be in

memory. So those payloads are going to operate in memory, anything that's sent across the network is going to be encrypted, and they're not going to put anything on the file system. So we do get questions sometimes of what's the point of these memory-only payloads if I compromise, say, a domain controller with a memory-only payload or a server, if that machine reboots, then the malware is not there. And that would be correct. But the whole point of this talk is the attackers have all the credentials or most of their credentials in your environment. So what we see in our investigations is if a critical server reboots, the attackers are going to notice that and then they

just compromise it again with those credentials they have. So there's still no motivation or reason for them to put anything on the file system. So we see where if you're doing disk forensics, there's nothing on the file system to find. And if you're doing live forensics, so you're running one of those tools that will inspect process memory or look for like YARA signatures in a process, those are gonna be hidden from as well. The same APIs that those tools need to operate are being manipulated and tampered with by the malware. And the whole reason the malware does that is so you don't find them. So you really have to look in volatile memory if you

want to do this. I have a screenshot here at the bottom with some things highlighted in red. There's dozens or hundreds of these reports at this point. I just like this one because Microsoft summarizes it well in two sentences. So this was them looking at, I believe this was a like hacking for hire group out of Australia that Microsoft just burned to the ground and the company went out of business. And they talked about how the payloads from this group all reside exclusively in memory. And then those payloads that are in memory allow for things like key logging screenshots, recording the microphone and so on. So everything you could want as an attacker, you don't

have to put anything on disk to do that. So again, this is why we like memory forensics. As far as the event logs, the first great thing about them are the timestamps. So if you haven't seen event logs before, always keep that in mind. I'm not going to mention the timestamps over and over again, but every time I mention something's in the event logs, just know that that is timestamped down to the second that it happened. So if you want to know when did something happen on a machine, when did the malware appear, when were credentials accessed, you have the exact second that it happened if you follow these techniques. The other nice thing is

in many organizations, the event logs are centralized. In most cases, in many cases, that is just the critical servers, domain controllers, email servers, and so on. Other ones that we see, that will be for critical users as well. So your domain admins, some of the people who are very public in these organizations that might get targeted directly on their own. So within this one place, you can take these event log sources that I'm going to be talking about and immediately see if a credential dumping attack happened in your environment. Our plan for this research, we wanted it to work on modern systems. We wanted to know what was available on modern systems. So we built out a large number of systems, Windows 10 and Windows 11. And then we

used them ranging from their default state, which is none of the protections are turned on, all the way up until every knob, the most security you can get on Windows 10 and 11 as far as preventing credential dumping attacks. So we had all these different systems built out, some VMs, some physical machines. We then executed every credential dumping technique that we knew about, again, through open source tools or private toolkits. One thing to realize is some people will say, oh, well, that's like Mimi Katz doing it. That's not what it really looks like in the real world. But the thing is, there's only so many ways to accomplish a task on Windows. So whether that

code is injected by an open source tool or a private tool, the end result is the same. If you can detect it, you can detect it. So we executed all of those techniques individually. So one technique per set of data. And then as I mentioned before, the forensic set of data that we captured was volatile memory and then the event logs of interest. In these research deep dives that we do, once we have those data sets, we're doing two things. We're looking at timelining. So as I mentioned, the event logs all have timestamps, and then there is a large number of artifacts in memory that have timestamps as well. So when you combine the event

logs and the memory timestamps, you see essentially everything that happened on that machine down to the second in order. So we go through that very, very slowly, every entry that might be relevant. And then we also do a deep dive on the memory artifacts. So if code is injected, if there's payloads in memory, we're going to extract those out of memory, throw them in IDA, figure out what happened. Obviously the open source code ones we can just read it, the private toolkits you have to reverse them. But the idea is we want to know what exactly is happening in memory and what is the effect of that. And then the end result are all these

artifacts I'm about to talk about. When we look at gathering credentials, there's two ways to do that, or two ways that we break them down at least. There is the direct method, this is what we're gonna talk about here, and then the offline method. So with the direct method, it means you're going to go after those credentials directly. You're going to pick on the lsas.exe process. That is where all authentication activity flows through the machine. If this is a domain controller, then it's all authentication across the network. If it's your own machine, if you're running Windows right now, when you log in, when you use other forms of authentication, at some point, LSAS is going

to be involved in that decision. Depending on the security level that you have on the machine and the version of Windows, LSAS may also have credentials. In some cases, they're stored directly in a data structure that you can read out. You can get hashes or you can get the clear text credentials. And then in other situations, those credentials are just floating in memory, right? So LSAS takes your password, does something with it. It doesn't necessarily store it in a list that Mimikas can go walk the list and pull it out. But with memory forensics technique, you can go find it on the heap somewhere else in LSAS memory and then you have it available. This

is a situation where the event logs are not usually super helpful unless you have like Sysmon. Sysmon can help with some of this, but most environments are not running Sysmon. So if you have just the default Windows event logs, that will not help too much in this category. We'll see it will help a lot with the offline ones. So if we want to find these direct attacks against credentials, we need memory forensics. And as you'll see on the next two slides, we're going to be looking for things like injected code regions, injected threads, handle manipulation, and so on.

This is looking at volatility versus Mimikatz. When you run the commands that are going to list out the clear text credentials off the machine or some of the tokens and so on that Mimikatz can recover. What Mimikatz is going to do is inject into lsas.exe. you can see that volatility is telling us about two regions it's injected. This first one has a bunch of strings that if you're familiar with credential access, so it's accessing the SAM APIs, that's where credential data is going to be. and so on, and then down here is legitimate code. So both of these are related to Mimikatz. Again, we're not targeting, looking for strings of Mimikatz in this case. We're

looking for generic attributes inside of lsass.exe that shouldn't be there, and when we find it, we report those memory regions that are suspicious. So whether it's Mimikatz, your own tools, some private tool out in the wild, any of those that implement this technique, we're going to find them generically. This is versus Metasploit, so it has its own separate ways to access credentials. What it likes to do, or one of the things it likes to do, is inject threads into that remote process. So the idea of a thread is that is a unit of execution inside of a process. And every thread, if you're doing normal programming, should point to a DLL. So when a thread's running, it should be running code inside of a DLL or the

main executable. If you're not executing code inside of an executable, you're just out somewhere randomly in memory, that is something we want to flag on. So you can see here, this is volatility saying we have a thread, the thread's inside of lsass, and then this address where it's executing doesn't belong to a file. So this is just shellcode injection, and we are catching that directly. If you're familiar with memory forensics, and you see things like the page execute read write that we're keying in on here. There is malware like Cobalt Strike does it and some open source ones that purposely don't make this attribute or don't generate this attribute so we can't find it that

way. But if you want your code to execute it has to be in a thread so we can find it a backup way. So as I mentioned, those direct ways are very easily detectable with memory forensics. In fact, they just look like injected code in general. They're not even specific to credential dumping attacks. How you key in on that is if those plugins and volatility start telling you there is code injection in LSAS, that's when you have to worry, right? Because the only reason to put code in LSAS is to get access to credentials. So basically all we had to do for the direct techniques of the tools we evaluated was just make sure there

wasn't a bug in volatility and it found all of them right away. Now the more interesting part, or maybe the newer part, if you're only familiar with generic memory forensics, is all of these offline techniques we're about to talk about are very credential access specific. So if you haven't studied these in detail, and maybe you knew what was on the last couple slides, I think I'm 100% certain you'll find some new stuff in here, because as I'll point out, there's a lot of stuff that has no references on Google that we uncovered. So with this, we have offline. What we want to do is get their credentials offline to then dump them out. So instead

of injecting code into LSAS and then we find the credentials as they're cached or we try to scrape them out of memory, what we're going to do is get a memory dump of the lsas.exe process. And this happens a lot as well. Attackers will get the memory dump of LSAS, you can then compress it, send it out to your C2 server somewhere, and then the attackers have all the time they want to decrypt those credentials, try to brute force them, whatever it might be, whatever technique they're gonna take, and then after that they can come back in and reuse their credentials. So rather than being allowed to EDRs or memory forensics with injected code, the

end goal here is to just get that process dump out. So to do this, Microsoft is nice enough to provide quite a few APIs where you can dump the memory of an arbitrary process, a process that's not your own. And so malware is going to abuse this. And as we'll see, this is going to leave quite a bit in memory. And we also get to use the event logs now. The first thing that we see abused quite a bit and it's a pretty interesting API is create process with logon w user. That long string is an API that programs can call. And what we're seeing here is this is the documentation from a tool called mouse logon. So this is an open source tool that you can go

back and test if you want in your own environment. So the idea is, it says instead of opening a handle to LSAS directly, so the EDR won't see that your malware is opening a handle, you can abuse the sec logon service, which is this API is going to drive that service, and then it will create the handle for you. So it's an indirect way to get a handle made to LSAS. The end result is, as you'll see, is that handle will let us access the memory of LSAS, but we don't have to make that handle directly from our process. So as I said, this is an open source tool, Malsec Logon. One of the options

is this SecLogon handle leak. So this is the source code from that, and there's a few things I want to point out. The first one is, here is the malware using the API. And then when you use this API with the net credentials only, you're telling Windows that basically you're just using credentials that you made up. They don't exist locally, but supposedly they exist on some other machine. But we're only going to use them on this machine. So you can see in the parameters, the malware is just saying, "Hey, the username is 'malsec_logon_user' and it comes from the Windows domain of 'malsec_logon_domain' and the password is 'malsec_logon_password'. Obviously none of these exist in the SAM

hive or on the domain controller. So this is how that process is going to spawn. That'll be the user account. And then the other interesting thing, there's a comment in the code that essentially says the handles that come back, so the handles that get leaked through this API, the author of the tool couldn't figure out how to close them. So we're gonna have these open handles as we'll see on the next slide. So when malware uses this technique and we go to look at a memory sample, we get what are some very strange results. We're running the PS tree plugin in volatility, so this is going to give us a process tree. If you've used like process hacker before, other tools, you can do the same thing.

The notion is these indentations are based on the parent-child relationship. So this is saying that LSAS was running and then it spawned off a process called mouseklogon.ex. There's not enough room to store the name, so mouseklogon.ex. In normal operations, LSAS should not be spawning processes, so that's automatically going to be suspicious. You can look in your environment for children processes of LSAS and now you find malware. And there's an interesting thing here in volatility. So if you look in the red here, what these are supposed to tell us are the number of threads and how many handles, basically if the handle table is valid for the process. And it's saying for mousec_logon there's no threads

and these dashes mean that the handle table was invalid so we couldn't parse it. So this mousec_logon process is not actually running. But the reason it's still in the process list is Windows can't close this process until all of its handles are closed. And if we run the handles plugin, which tells us all the handles for our process, we pick 872, which is LSAS for our PID. We're looking at process handles. LSAS has a handle back to this malicious process. And when you look at the access here, this is full access. So malware is going to request full access when it uses that API so that it can access memory and create a memory dump.

So it makes this incredible artifact. So even if, say on your domain controller, the credential dumping attack happened a month ago, two months ago, you haven't rebooted it since then, you're going to have in your memory sample the exact process that caused the credential dump. The other thing is on Windows, processes have a create time. So you see down to the exact second when credential dumping happened on that machine. So it gives us this great artifact that we can key in on in memory. The other thing we noticed, and this is something that did not exist on Google. We asked some people we know in the industry. No one noticed this before. If you remember, the name of that technique was the SecLogon handle leak. So what we're

looking at here is we're looking at the service host process. So a service service host that exe can run many, many services in the same process. In this case, we're looking for the service host running the sec logon process. And if you notice the crypt, there's two things here. Crypt based dot DLL is loaded into that process. And then if you look at the timestamp of when the DLL was loaded, so remember I said there's timestamped artifacts in memory that don't exist anywhere else. If you look at when the process started, so the main AXE, all the other DLLs leading up to this, they started on what was October 26th last year. And then a

day later, we see where critbase.dll loaded. And we saw this when we were doing timeline analysis. So it was, if you see here, When our attack happened, we actually ran a few attacks. So the idea is we only saw CryptBase.dll loading into SecLogon when we were doing these credential dumping attacks. And we saw them at the same second that the credentials were accessed. So we're like, okay, that's interesting. Maybe there's a side effect here of that API being used to force CryptBase to be loaded. So we have access to what's hundreds and hundreds of memory samples with our internal set as well as that customer customer environments. We queried for service host processes running the SEC log on service that had this deal loaded. The only ones across what

was well over 1000 memory samples were ones where credential dumping attacks happened. So, anytime you see cryptbase.dll in the sec.logon service, you know that credential dumping attacks happened, and again, you know the exact second that they happened. There's another thing, and this has been documented in a lot of places, but it's something you'll want to check in your own environment. If you have logon records enabled, unfortunately they're not enabled by default, but the vast majority of enterprise environments enable this. you have what are called logon type 9s. So if you look here, there's logon type and then 9. This means that you're using those net only credentials, or what's called new credentials. And if you remember, the malware basically just made up a new username. So

if you use that API or you use /net only on the command line, you're going to generate these logon type 9s because you're using new credentials. And then if you notice here, these were the parameters to that function call, the mouse sec logon user and the domain. So again, if you query across your environment for logon type 9, in 99.9% of cases, that is a reference to a credential dumping attack. Again, this was documented in a lot of places already, but it has to be there if you're going to talk about these attacks. And the downside is it's not enabled by default, which is painful if you're doing somewhere, you get thrown in a random

environment to do IR, you don't control all the settings before you get there. So what we wanted to do was find an event log source that was enabled by default and that we could use to find these attacks. So again, this popped up in the timeline directly. We didn't have to go look far. And we found this source, the CryptoDP API, and then this particular event ID. And what we noticed is when you use that API, it generates one of these records and it's going to have the username and the SID. If you are familiar with Windows SIDs, they are supposed to be a unique identifier per user. But what we're seeing here is that

the administrator user, the -500, has a different username. You can see the -500 SID and the username is "mousec_logon". So the only way this kind of conflict would happen in the real world or in a non-malicious way is if someone changes the name of the user account, but you don't exactly expect like administrator to change, and that's gonna be the user where that's gonna be targeted by this. So you can query across your environment. If you wanna do it fully, you'll need some scripting. You can do this in most, commercial tools that you have or if you have open source stuff. So what you want to do is basically query, key in on the SID,

and then you want to find where there's different user accounts for the same SID. And again, in our test, again with a few exceptions where the name of the user account was changed, this always pointed us to those credential dumping attacks, and the great part of this is this is enabled by default. So all of that was ways we could detect abuse of that very popular API. The second technique that we saw where a lot of tools were using is what's called process forking or cloning where forking is obviously more of a Unix term but the idea is you're taking the currently running process and then you're going to fork it and what that means

is you're going to create a new process that essentially, I know there's some caveats there, but essentially you've made a new process that has the same exact state as the previous process. So the same memory, the same DLL, same executable, same handles, et cetera. Now, what the malware does is they want to abuse this feature to hopefully bypass some EDRs. Because if the EDR is monitoring the real LSAS and then you clone a secondary one, hopefully when you pull the memory of the secondary one, the EDR doesn't know to look for that. So that's exactly what this technique does. There is the NT create process EX API. You can tell Windows that you want to

clone the process. And from there, you can target that secondary LSAS. From a detection standpoint, this is again awesome, because that API is super weird, right? So it's going to make a child of lsass called lsass, which doesn't make any sense. You're not going to have the handles closed, because that process was opening a handle back to the original lsass. And then what's also super weird is you can look at those, your process creation event logs, and you're going to see where the parent process name was lsass and the new process name was lsass. So lsass is spawning and lsass makes zero sense. Again, if you just have a generic detection for lsass spawning processes, you'll see the Malsec version, you'll see this version as well, but it's

definitely not something that would happen on a normal Windows system.

There's also a technique called silent process exit. This is something built into Windows and you wonder why does this exist? What useful purpose did Microsoft expect when they made this feature? So this abuses the where fault executable that gets abused in several different contexts by malware. The top part here is the documentation from the tool. So this is from NanoDump, and it says you can force the word fault process to create a full memory dump of LSAS. To do this, when that API is called, it's going to create registry keys in the background. In most cases, the malware is just going to delete those registry keys, so that's not what you want to focus on.

And you can see here we're running NanoDump. This is one of those credential dumping tools. We're telling it we want to use the silent process exit technique, and then we tell it the directory we want to write to. So from a detection standpoint, in memory, this is kind of rough. We're not really going to detect this in memory. It's Windows doing all the hard work to get access to LSAS memory and then write it out to disk. But luckily, one nice thing Microsoft did is there is an entire event log source called Silent Process Exit. You can look for entries from that event log provider. And then the names are not documented and they're kind

of useless, but you can figure out what's going on here. Param1 is the victim. So our victim that's being dumped is lsass.exe. And then param3 is the name of the tool that forced it. Now the interesting thing here is the name of the API and if you read all the documentation about it, it's supposed to be silent process exit. Like you're forcing that process to exit after you make a memory dump of it. Obviously if you force LSAS to exit, then the machine's gonna crash. So there is a variant or a way to use the API where you still get the memory dump out but the process will run afterwards like nothing happened. So the

best thing you can hope to key in on here is this event log source. It is there by default. If you query out these records and you see where something is dumping LSAS, it means there was a credential dumping attack in your environment. So with the two techniques we discussed so far, what we were hoping for are cache credentials, right? We can inject code into LSAS, go scrape the credentials out, or we could abuse one of those APIs to get a memory dump of LSAS written to disk, and then we can go scrape credentials out. What Microsoft has done in the last couple years which we're going to go through all the details in a

few slides Microsoft has made a real effort to either not store credentials in else asset all or if it's going to store credentials it's going to store it in the protected else that will talk about so what we see a lot in. modern attacks and especially against Windows 10 and 11 is where the attackers want to get access to LSAS so that they can then get credentials as they later come in. Because if there isn't cache credentials, say I get access to your domain controller today and maybe there's no cache credentials. Well, if I could keep my malware active on that domain controller inside of LSAS, I can just get all of the credentials going

forward and it really doesn't slow me down that much. So the first way we see this happen are what's called security support providers. They are normally just abbreviated as SSPs, so that's what I'm going to call them. And the idea is this is a Microsoft designed way, so there's an API for it, where you can load arbitrary DLLs into LSAS. Again, you wonder like why does Microsoft do this? Their reason for it is for password auditing, so your random security tool can enforce a password policy on the machines. Really the only legitimate use of this, which it's arguably kind of cool, but Microsoft probably could have come up with a better way for it, is you can have custom authentication schemes. So there's providers where you can have public

key authentication for SSH to like a Windows server, but the only way to do that is that DLL has to live in LSAS, which is like a huge security risk. And then obviously malware is going to abuse it every time they can. Because Microsoft is going to be nice enough to load that DLL into LSAS and then once you're in LSAS memory you can do whatever you want. So for getting a SSP loaded, there is an add security package API. This does leave keys within the registry. So the few versions of malware that actually do this, they have to write a DLL to disk, then get it loaded. They're then most likely going to delete

their DLL from disk as well as delete those registry keys. The bad thing about this though is there's a separate API on Windows called enumerate security packages where that's going to look in memory. So even if you clean up your registry keys, clean up the file from disk, Microsoft or Windows will still be able to report your DLL. That led to the memory only versions. This is in a large number of open source and commercial tools now, but Mimikatz was the first one to publicly do it, so they got to pick the name. The name for the technique is MemSSP, which you might gather is a memory only way to basically be a security provider. The generic idea, there's a lot of technical details that aren't super important. The

generic idea though is there's a function inside of LSAS called SP accept credentials where your clear text credentials are going to be sent to. So if you can hook that function, then you can see all of the clear text credentials, you can write them out to disk, you can send them across the network, you can do whatever you want to report them. The idea is you have access to those clear text passwords. This is easily detectable though by API hooking. With volatility you can run the API hooks plugin. If you only care about credential dumping attacks you can tell it just to look at lsas.exe. But the only way to pull this off is through

API hooking and memory forensics knows how to do that. So this is a Extremely effective technique. If you have this happen on your domain controller and you don't find it for three months, it means the attackers have had access to the clear text passwords for everyone who logged in for the last three months. You're probably in one of those positions where you need to rebuild your whole network. So this is easily detectable from a memory forensics perspective, but it is extremely effective also from an attacker perspective. I did want to point out the on-disk artifacts for this though. Like I said, you do have those registry keys, but the first thing malware is going to do, if it even has a persistent version, is it's going to

go clean up its registry keys. So I wouldn't waste a whole lot of time looking for those. There's also an event log source for this. It's not enabled by default, and I queried across customer environments and our internal tool sets. No one has this enabled. I don't even know if people know about it. I only actually found it while we were doing this research project, so it's not well known. But if you enable these, which you might want to consider, there are four different types of these security packages. I didn't go through each one of them because the end result of all of them is you get a DLL inside of LSAS. But they're separate

IDs for each type and then if you have this enabled, Windows is going to create that event log. Now remember, this is only if you call the nice API that Microsoft can track. If you do the memssp version, that's not going in the event log so you have to do memory forensics at that point.

So what we covered so far were the ways to get credential access, whether, like I said, you can inject code into LSAS, you can get an LSAS memory dump written, or you can just start getting credentials as they traverse the machine. So what I what we're going to talk about now are all the efforts that Microsoft made to stop this from happening. Some of the ways the tools had to adapt and things you really need to think about environments you protect or if you're on the red team and they don't have these security protections, then you can take whatever credentials you want. So there's some really good reads here at the end on my references

slide. I have the links for these, but the idea is Microsoft invested what was easily millions of dollars to try to make it where you can't just go on a domain controller, run Mimikatz and have every clear text password. So they put protections in for known attacks, for bypasses and so on. One of the things they did was disable WDigest by default. If you've used Mimikatz or other tools and they actually have options of what technique you're going to use, WDigest is the provider that will authenticate credentials by default on Windows. One of the downsides to it is it stores the clear text credentials in memory. So if you run the W digest command from

Mimikatz, it's just gonna go inject code into Mimikatz, find the list of those credentials, and then give you all of the clear text passwords. So this was such a horrible security idea that it was disabled starting with Windows 8.1, Windows 10, 11 all have it the same. But what we see over and over and over again is the first thing that malware does, especially ransomware and malware with similar goals, is they just go re-enable WDigest. Remember Microsoft has to always keep things backwards compatible. They can't just shut things off and then hope nothing breaks. So the way that WDigest is disabled is through a registry key. So this is an example from another Microsoft report, but like I said, we see it all the time. If you Google

about it, it's talked about a lot by different companies. And the first thing these malware samples do is re-enable WDigest. And then after that point, once people start logging in, that list will populate in memory. This is how the malware does it. You can see we have the security providers. Remember that's the name of the DLLs. WDigest is one of those legitimately written security providers. And then use logon credential is the key that says whether the credentials are cached or not. So this is something in your environment, hopefully you're on like Windows 10 by now, if you still have Windows 7, Windows XP, ignore the next couple sentences. But if you're on Windows 10, Windows

11, or the server equivalents, if you have the ability to query the registry across your environment with PowerShell or anything else, or at least do it on your domain controller, query out this key, query out this registry key, query out this data key, and if it's enabled, that means the malware did it. It means malware came behind you at some point and re-enabled this. Registry keys have last right time, so you can use that to try to figure out when the malware was there. But this is something, like I said, Windows 10, 11, all the OSs going forward, it's going to be disabled by default, but it's definitely gonna come back alive if you have

malware interested in your credentials. That way is loud, the one we just talked about. You have to operate in the registry and a simple PowerShell script will find it. So there's ways, memory only ways, to enable this as well. So you can have WDigest disabled if you look in the registry, but what that does is populate two variables inside of the lsas.exe memory. So what A few open source researchers have done, and we've seen this used in real attacks as well, is they don't want to be found by a simple registry query. So they're going to inject code into LSAS, find these two variables, the isCredGuard enabled and useLogonCredentials. It's going to flip those so that first off credential guard is disabled and then use logon credentials

is set back to be active. And then once you do that, again, it's highlighted here, you don't have to change the registry key. The code inside of LSAS that's running in memory is going to think that you flipped the registry key even though you didn't. And then all of those credentials are going to come back to life and will be available to attackers. So as I mentioned, you want to definitely make sure that W digest is not enabled in your environment. But if you read 99% of the guides out there on how to do that, they're only going to mention that registry key. Nothing's going to mention the in memory checks. So you need to be able to do memory forensics as well to make sure that

those flags were not set in memory. Otherwise your credentials will still be there and you will be none the wiser.

I mentioned Credential Guard quickly before, so now we're actually going to define it and talk about it. The idea with Credential Guard is, again, Microsoft didn't want that situation where you have the ability to run with elevated privileges on Windows, you run Mimikatz or a similar tool, all their credentials just come out of LSAS memory and you walk away with them. So what they did is they came up with this scheme called Virtualization-Based Security, normally abbreviated as VBS. And the idea is, if someone in the audience right now, if you have a laptop that runs Windows, you're running Windows 10 or 11 on a laptop from say the last two, three years, what you're actually

seeing, so what you log into, where you run your browser, all your applications, that environment is actually a virtual machine guest of a very thin hypervisor. And then, on side of your hypervisor, or your guest inside that hypervisor, are several other guests. One of them is the LSAS isolated process or container, and that's where credentials actually go. So the passwords that are put in, any tickets that you might have for your domain, those go into a separate virtual machine. You have no access to those. So even if you get a full memory dump, not just LSAS memory, even if you get a full kernel memory dump like you would do memory forensics against, those credentials

are not going to be there. But the thing is, this sounds really nice, right? So the credentials are somewhere else, the attackers can't run Mimikatz or similar tools and get the data out. But if you think a little further, there's a lot of limitations to this, right? That data doesn't magically end up in the isolated container. If you look at how it works, there were some great Black Hat talks and similar conferences on this. Those credentials still have to go through the vulnerable LSAS in the first place. Sure, the end result is they end up in that isolated one, but if your malware is running inside of LSAS, the vulnerable one, you can get all

their credentials the same. Attacks like memssp that we just talked about, you're going to be able to get their credentials before they hit the isolated container, and then you don't care where they are after the fact because you already have them. So Microsoft then invested a lot of time and effort in stopping this. They wanted to say, okay, we still have this vulnerable LSAS. Well, we don't want that either. We don't want you tampering. We obviously don't want you in our protected guest. And we definitely don't want you tampering with the vulnerable LSAS either because that makes having the protected guest kind of worthless. So then what Microsoft did is they came up with this

notion of protected processes. And the idea is with a protected process, you cannot inject code into it. So all of those direct techniques that we talked about don't work. And you cannot open full access handles. So the APIs and different techniques that we talked about in the offline section, those don't work as well. Microsoft went way out of their way to kill every technique possible. But the problem is this only stops processes from doing that. So if you can load code into the kernel, which is where root kits live, and if you're familiar with bring your own vulnerable driver, the very complicated thing to say at a talk, the idea is there's dozens, hundreds of

vulnerable kernel drivers that let you load your own root kit even if you have driver signing enforcement on. So it's not that much of a boundary because you can load your kernel driver, you can still read lsass.exe memory, you can write to lsass.exe memory from the kernel. It's an extra step, but it's really not doing a whole lot to stop a determined attacker. The other thing you could do, and this is what Mimikatz did originally, is when you have a protected process, that's literally just a bit in memory related to that process that says, hey, lsass is protected. So you have the options I said, you can read/write LSAS memory from the kernel or you

can just go flip that bit in kernel memory, now LSAS isn't protected and then all the techniques that we talked about before worked. So once you can load a rootkit, if you don't have the HVCI on, which I will define on the next slide, if you don't have HVCI, all the other protections really aren't doing a whole lot for you because again, you can You can either get access to lsass.exe directly or with the help of a rootkit you can get access anyway. So the main takeaway you should have from this talk besides the detection techniques that you can use is if you're in an environment where you really want to stop credential dumping attacks, you really want to make it where you don't have to be so concerned

about attackers walking out with credentials, you need hypervisor protected code integrity, which again is a mouthful. You're going to have better luck searching HVCI for it. this takes things to a whole different level. So it not only runs, not only puts credentials in a separate VM, all the code that's doing protection of the operating system runs in a third VM that you don't have any access to, and it specifically protects LSAS. So first off, if you try to load a kernel driver and then flip that protected bit, the security code's gonna detect that and blue screen the machine, which obviously is not very helpful to you. So you can't just take LSAS that's protected and

make it unprotected. The other thing it does is, I'm not going to make this too technical, but the other thing that it does is that extra VM is monitoring the memory of LSAS. So even if you load a rootkit into kernel memory and you try to write into LSAS's memory from that rootkit, the VM is going to detect that and again crash the machine. And that's the level that Microsoft has to go so you can't make LSAS unprotected and you can't inject code into it. Now the thing they can't stop which would just break be way too bad way too difficult from a compatibility perspective is they can't stop rootkits from reading lsass memory so

if you have something where there are cache credentials or you're basically racing against logins you're finding maybe like the page where clear text credentials go you can still read those from a rootkit you have to win race conditions at that point it's not very fun and it's a far far cry from what we had before where we had direct access to all credentials. So again, if you want to really have protections from this, plus a bunch of other ones, you need HVCI on. It does have some compatibility issues. It is fairly new and so on. You would need to test it a lot, but it makes it where you really are protected. And like I

said, without that, there's five, 10 different techniques that we talked about that you can use where after that you would have access. So in conclusion, hopefully you learned something. I learned a lot doing this research, and like I said, several of the things I discussed have no references on Google, which makes me think they're pretty unique, pretty new things that were found for detection of credential dumping attacks. If you can query the event log IDs and sources that I talked about across your environment, why not do that on Monday? And if you find access to credential dumping attacks, then you're going to have a busy week. Hopefully you don't find any hits. As I mentioned

before, you should be doing memory analysis at least on your critical servers. We have many clients who are doing memory forensics frequently, like very, very frequently on the domain controller and other systems. If you're not doing memory forensics, There's a lot of malware techniques and a lot of malware that could be in your environment that you're literally never going to catch. So you want to be able to do memory forensics at a minimum on your critical servers. That will find the other set of techniques that we talked about. And if you're more on the engineering side, the configuration side, your goal should be to get where HVCI is at least enabled on your critical servers.

Otherwise, you are fighting a losing battle. So I managed to time that perfectly. It's now 4:45 at 45 minutes. If you have any questions, email me. I'm on some Slack servers as well. I'm also on Twitter. And the last thing I'll say before questions is all those numbers that I had in the slides, I have my references slide. Don't try to take a picture and type this. Just email me and I will gladly send you a PDF of the slides when I get back home tomorrow. So thank you all for attending.

I do have a few minutes for questions, but I also know they're going to transform, or I was told they're going to transform the area for the closing. So I will be at, I don't know where it is, so I'll be listening during the closing. I will be at the, whatever the social event is after as well. Hi. Yeah, so it's still not perfect. I just saw it as a plan. If you have the agency, I guess we're, yeah. I saw how equivalence, like you have to have one.