Using Volatility 3 to Detect Sophisticated Malware

the speaker. >> All right, good afternoon. Um, hopefully you guys all had lunch and if you haven't had lunch, there's lunch upstairs. Um, so this next talk is about volatility, memory forensics, and Andrew Casease is going to give us a little bit of insight about what's going on, what's new in volatility. Uh, take it away, Andrew. >> Thank you. >> [applause] >> So, is the mic okay? Okay, cool. I don't hear any echo, which is maybe a good thing, but that's usually how I know. Uh, so for those who haven't met before, my name is Andrew Casease. I am the director of research at Vexity, where we get to monitor and do instant response and lots of analysis in some of the most

targeted networks, at least in the west. So we get to see the latest malware, the latest toolkits that your really sophisticated actors are using, get to see a lot of ransomware from real criminal group groups, right? Not like small groups. So the best stuff, whether it's target attacks, ransomware, so on, we get to see that type of malware. And then on the other side, I'm also one of the core developers of volatility, been involved with volatility since about 2010, 2011. And you're going to see all of that combined today. You're going to see the malware that's out there that's considered sophisticated that bypasses your EDRs that makes it where you have no telemetry about these very

sophisticated attacks against your networks. And then you're going to use the open source volatility as I'm going to show you in a bunch of slides to automatically detect such malware. Uh what's the magic to make the slide move or is it not in the right mode? Oh, I don't know if you hit record or was I supposed to hit record? Yeah, I'll hit it. Okay, hold on. Okay, so it's good to come in person because I technically wasn't recording and I'm not repeating all that. So, moving on to the the second slide. Um, there we go. All right, now the buttons work. So, what is memory forensics? Why have we spent well over a decade now with volatility? Why

do we spend so much time doing the research and tracking down this malware? And with memory forensics, the idea is we are looking at the full state of the system. So you can imagine if a machine is infected and you turn it off and you get the hard drive, all you have is the files that were on that hard drive. And as you're about to see over the next 40 or so minutes, the malware that you care about and the payloads that you care about don't write a single bite to disk. So if you're not doing memory forensics, all that malware is gone. So with memory forensics, we walk up to a machine. Let's say this laptop has 16 gigs of

RAM. I go to capture memory from it. I walk away with a roughly 16 gigabyte file and that has all the state from that system. What was the operating system processing? What were all the applications doing? What was the malware doing very very specifically? And so then we're going to analyze those memory samples to figure out the exact parameters of an infection. Now the other thing that's really important, I put very few things in red in my slides. I like my old school black and white, but there's one thing in red here. And if it's in red, you need to pay attention. The idea is when I'm talking about memory only malware which is going

to be a big theme for the rest of the talk. If you hear myself or anyone else from vexity or volatility talking about memory only malware this is malware that does not persist at all. So if you walk up to that machine and you turn it off or you reboot the machine that malware is gone forever and the proper properly written malware that we see in the wild doesn't write a single bite to disk. So when I say memory only it truly lives only in memory. And you might wonder what is the point of memory only malware? Well, the idea is actually there's been two talks today that I watched that covered it in great detail.

In modern enterprises, you have Windows, you have your domain controllers. If you have stolen credentials, there's no reason to leave persistent malware around because if the machine reboots, you can just use those stolen credentials to get back onto the system. So when I say memory only malware, I truly mean only in memory versus fileless malware, which is largely a marketing term that doesn't make a lot of sense. Fileless malware does make files. It just doesn't make aexe or a DLL your traditional executable. It makes a shortcut file or it writes itself into the registry. But with fileless malware, you can reboot the machine, turn the machine off, and the malware is still there. You can take

your disc drive and do forensics on it. So again with memory only malware that is not the case. There's also two reports I like to call out if you've never seen memory forensics before and you want to read some incredible reports. Neither of these are vexity reports. So they're not me just advertising work I did or something like that. My all-time favorite report at least uh in this category is from Microsoft. It's the URL is kind of small at the bottom. If you email me I can send it to you or if you search Microsoft and Cororlump which we'll get to in a second you'll find the report. So I said this is uh my top one or two

favorite malware threat reports ever. So what happened back in 2022 uh this company out of Austria where Microsoft calls them cororlump as well as the malware that was used corlump. And the idea is this company out of Austria was a mercenary hacker for hire group. So you could hire them to hack basically whoever you wanted. They operated for a long time but then eventually or apparently hit the wrong customer because they made Microsoft so mad that in this report Microsoft completely put the company out of business. They detailed every payload they used, every piece of malware, every attack technique and the company went out of business overnight. So it's incredible technical detail. But the reason I call it out

here is I have it boxed in red, but this is a quote from the report and they said the core lump group and their malware, all of the main payloads, so the payloads you care about reside exclusively in memory. So all these victims that this hacker for higher group was targeting, nothing was left on disk. You had to do memory forensics to find it. And then if you look at the second red box where what happens, what do these payloads do once they're in memory? It's all the stuff you don't want to happen if you are targeted by a government or some intelligence agency or some criminal group. This includes things like recording your microphone,

recording your web camera, key logging everything that you type in, exfiltrating files, running command, getting screenshots of the desktop and so on. Your whole life is stalked at that point. And this is also something we deal with a good bit at lexity. We work with uh very targeted individuals and it's not just your office, right? if you unless you live alone, if your laptop gets hacked and you're working from home, now they're recording the audio of basically anyone in earshot of your laptop. So, it's not just you. So, these are things like the exact opposite of privacy, right? Uh so, Microsoft burned that company to the ground in an incredible report. More recently, just from May this year, Google put out a

report. So this is Google's uh cloud security team, former Google Tag, putting out a report where they called it innovative tactics from AP41. And what they're calling innovative as of 3 months ago was the fact that all of the payloads from this nation state back group were memory only. And one of the main things they did was process hollowing. If you've never heard of process hollowing, don't worry because there's about 10 slides on it coming up. But it shows that even as of 3 months ago or so, one of the best groups out there, the tag group from Google is calling it highly innovative and all the rest because it's memory only payloads. And then I assume we're all familiar

with CISA to some extent. What they do, they have their emergency directives. They just put one out for the F5 devices in the last week or two. This is where CISA tells all the state agencies and national agencies under their purview how to respond to an incident. And ever since Solar Winds, uh, the part that is in red and the sentences around it are copy and paste into every report and they change the product name. And the very first thing they tell agencies is to in very formal word words get a memory capture and do memory forensics. And then if you look at the IoC's that are in a lot of these reports. So CISA

is saying hey every state and federal agency go check if you're infected in many cases all of the IOC's are memory only. So if you don't have the capability to do memory forensics you can't even comply with the directive because those IOC's those indicators of compromise are only in memory anyway. So obviously we want to be able to deep dive this. Now with volatility just to clear the air a little bit get everyone on the same page. Volatility 2, the original kind of main public version of the framework was released at Black Hat 2007. That was a long time ago. So when a framework is designed in 2007 and what memory forensics looked like then versus

now, uh, that was not something we could keep up to date. So we started from scratch. Volatility 3 is now released. It has fully replaced Volatility 2. All of the screenshots and discussion we're going to talk about today are from Volatility 3. And if you're not too familiar with it, it is uh the most widely used open- source framework for memory forensics. You can go grab it on GitHub and do everything you're going to see uh in the slides today. So we have the announcement blog post here and then it just so if you have the notes you want to tell your team or something, uh Volatility 2 is deprecated. So even though we deprecated in May, a couple

months ago, people still email us. People actually email us and say, "Hey, your GitHub repo for Volatility 2 is read only and I can't create an issue for it." Which is literally like I locked, you know, you lock the door in front of someone and then they go to open it and they're like, "Hey, I can't get into your house." So that is the whole reason it is uh deprecated, it's never coming back, switch to volatility 3 and then hopefully this talk will show you why you should do that. Now the research motivation for this talk, it's not just flipping through a bunch of basic volatility commands. There was a big effort um for this. At Vexity, we

have our research team where our one of our main goals is to keep up with all the threats, all the malware, all the everything across Windows, Linux, and Mac. So, our team will do these huge multi-week or multimonth deep dives where we get every piece of malware out there. We have all the reports that are public that we have access to that aren't public. We have all our internal reports. And the goal is to make it where we can detect everything that's in the wild. When we looked at volatility 2, so the old version of the framework, it plugins were a bit dated, right? We were spending a bunch of time getting volatility 3 working and up to date. And

so there were techniques that volatility 2 could not detect that we wanted to detect in volatility 3. And then the other thing to realize is a lot of what volatility 2 was doing to detect malware is now in EDRs. Volatility has been open source a long time. There's been a lot of talks. So EDR started doing the same thing. So malware wanted to stop doing that so it wouldn't get detected. So again, the goal here was to make sure we were up to date with every code injecting malware that was out there, all the payloads that were out there and make sure we can detect it. If you've never had to chase down injected code,

uh the idea here is instead of starting our own process, which is super noisy, right? If we drop an executable into some user's temp folder and then we run that executable, that's going to generate a bunch of logs, a bunch of alerts, and we're probably going to get caught really quickly. Does that make sense? So, we don't want to have to put random new executables on the machine versus in this case, uh, I'll give you a little background before I give everyone the trick question. The idea here is we're looking at TCP view. This is basically a guey version of netstat. So, it's from CIS internals. It shows you all the processes that have connections

open and listening sockets and all of that. And I can tell you with a 100% certainty that is a there is a very nasty piece of malware injected into one of these processes and doing network activity. Now, it's a trick question, so don't hurt your brain too much, but can you tell me which process is hosting the malware here? Yes, >> Elsas. >> No, it is not LSAs. I'll give you a hint. It's one of these service hosts, SVC hosts at the bottom. >> What? It's a trick question. >> So, there's no way to figure it out from this slide. The reason I set up the trick question is this is why code injection is so popular. With code

injection, you're hiding in a longunning process and you're just injecting a little bit of extra functionality into it. And at that point it becomes basically impossible uh without memory forensics just looking at normal system activity normal telemetry things like that to figure out what the malicious process is because you already for example the service host you already have 50 hundred of them running on your system they're already making dozens of network connections. If malware makes one more extra connection are you really going to detect that? Probably not right and so this is why it's so popular. So we want to be able to detect this. We want to be able to find code that's been

maliciously introduced into a victim process to then run long term. So there's two things we have to understand for this uh what's on this slide and what's on the next one. And then we can start detecting malware. Now like I said, I don't use colors a lot in slides. In this case, there's two blue entries. So these are things you're going to have to memorize to understand the rest of the talk. What we're looking at here are memory regions inside of a process of one of those service hosts. You can see the PID in the first column is 732. So the idea here is going back to our scenario, we have a 16 gig capture of

RAM from our laptop with 16 gigs of memory. Inside of that 16 gig capture are individual regions for every process that's running. So we kind of saw that here we have an LSAS, we have multiple service hosts, we have services.exe. they all have their own regions inside of that 16 gigabyte capture that are separated because obviously you don't want one process reading another process's memory or writing to it and vice versa. So all these processes are separated and isolated inside of that 16 gig capture. So from a memory forensics perspective, what we need to be able to do is figure out where inside of that 16 gigs of memory are all the little chunks that make up a individual process. And

what you're seeing is a plug-in called VAD info. That is the name of the data structure Windows uses to isolate those. And so for each region, we're going to get some useful metadata. It's going to tell us the starting and ending address. And then very important in blue, it's going to tell us the protection. So what protections and what access can be used on that memory. And then the last column here is whether it maps a file or not. So like I said for this slide is cut or be overwhelming. Most processes would have like a hundred of these entries but it's the little sections inside of memory that that process has to itself. Now very importantly and this is the key

thing none of the rest of the talk is going to make sense is when we have code that is legitimately uh created by the operating system or memory regions to host code that is legitimately loaded those memory regions that's hosting legitimate code. So a regular DLL on disk or rather your browser that you double click Chrome and now it's running. The memory regions hosting legitimate code have two attributes and they're in blue. The first one is the protection needs to be page execute write copy. If it's anything but page execute write copy that is not the legitimate operating system do it and it's injected code. The second attribute we need is this file column. If we are hosting code we should

also be mapping a file. And maybe that makes sense, right? Because if you have code to run, if you have an application that's running, it should probably be on disk. If it's not on disk, then kind of by definition, it's memory only because it's not being written back to the file system. Does that make sense? So, you're going to hear me say execute, write, copy a lot, and the fact that we need a file. Did you have a question? >> No, no, no. It's it's looking for code. So was just to point out the fact you can see the other DLS the the third from last and second from last are also execute right copy and also mapping DLS

that tells us that is legitimate but what we're going to be able to do is use this metadata to figure out when we find code in memory does it belong to one of the ranges that has the right characteristics or the and then it's the two that we need a map file and execute right copy the other really important thing and you'll just have to trust me on this because we only have 40 minutes is this data cannot be manipulated by malware. Um, we could talk about it at great length after if you want. Uh, but just trust me that if you try to even if like a kernel rootkit has access to all the memory, if it tries to manipulate

this in a way that would bypass the pl the detections we're going to write, the machine would blue screen. Many people have tried talks on it. There's like failed projects on GitHub. Just trust me, you can't do that. Now, the inverse of that is figuring out where code is in memory from the process view. So the data structures, so the information that we use to generate this slide is in the same exact memory as the process itself, which you could probably guess means the malware can manipulate it. Now, this is all the tricks malware is going to do to hide from your EDR and your AV and your debugger and CIS internals and all the

rest. But all that really does is make a bunch of like uh red blinking flags to us from memory forensics that something's wrong. This is why malware persists for years in environments and like I said, bypasses EDRs and all of that, but we're going to detect it right away with volatility.

Now, to understand the really cool stuff, the process hollowing and the rest we're going to talk about at the end, we need to spend about four slides on the traditional code injection techniques, the ones volatility 2 could detect, or the new stuff's not going to make a lot of sense. Now, the first one on the left, remote DL injection. This is not memory only. This is the only one we're talk about that isn't memory only, but it's going to set us up for the other ones. In this case, we are legitimately loading a DLL from disk, whether it's into our own process or whether we're using a technique to force a victim to do it. So, that DL has to

exist on disk. So, that already makes it kind of an old school technique. And then we're calling the regular APIs to load a DL, which means it's going to show up here. The name of this plugin is DLL list. It's going to get the DLS uh that were loaded through this legitimate means. It also means to detect DLS that are injected in this way. We don't need a special plugin because you just saw on the last slide. We can do it already. So the first variation of getting around this to moving where we're memory only. There's nothing on disk to go look for is injecting shell code. This still gets used a lot. If you saw David's talk

earlier and I forgot the speaker name, but one other one right before him, they spent a lot of time talking about shell code today. The idea with shell code is you're injecting code, so just raw instructions directly into memory. They're executable and then you run them. But this is super weird from a memory perspective because it's like you just have a blob of code hanging out and it's not backed by a file on disk and the permissions are going to look really weird. So even going back to volatility 2, we could detect this. This is the maline plugin. Uh it looks for memory regions that are hosting legitimate shell code or not legitimate shell code,

real shell code, something that looks like actual shell code uh that is not backed by a file and it spits it out right away. This is also something that any decent like commercial level edr that you might have access to uh will do as well. So the second form and this is still used quite a lot. If you've used metas-ploit or some of the private frameworks, you've uh used these payloads whether you used it or not is reflective DLL injection. So this is the best of both worlds. The idea is writing a full-blown payload in shell code is miserable even if you're like the best hacker in the world. That is just a not a fun way to spend your life. So you

want to be able to write a full-blown DL. You want to sit there in Visual Studio, write the most sophisticated malware you want to write, but you don't want it to exist on disk. You want it to be like shell code, but you want an actual full-blown DL that you can program. So, this is where reflective DL injection comes in. This is where you normally have a loader. So, the quick version, the quick rundown on loaders is you have a victim process. So, let's say we'll pick on service host again. You have your victim service host process that's already running. Your loader gets injected into service host and all that loader is going to do is download DLS

across the internet and then load them memory only. Those downloads of DLS across the internet, the DLS will be encrypted. They will only be decrypted inside the victim process and then they will be reflectively loaded which means as was on the last slide they're going to be loaded without them touching disk. So now you have full-blown DLS in memory, never existed on disk, never going to be written to disk. And even if you have like full pcap, your pcap just has the encrypted data. Volatility picks up on this as well through the malfine plugin. So this is the plugin you run looking for shell code as well as reflectively injected DLS. In this case, Malfine saying like, hey, I found this

PE file floating in memory. It's page execute rewrite for the protections instead of execute write copy. it's not mapping a file. Everything's wrong. There shouldn't just be a DLL floating in memory uh in a memory space that's executable. Emoteet has been mentioned at least twice today, just in the talks that I've watched. So, it's a very annoying piece of malware that is pretty brutal. You don't want this inside your environment. It usually runs through run DL32. In this case, you can see the command line is uh rough, right? Like it's a rot 13 encoded path. So if you had process telemetry on and you had something doing analysis on the paths maybe you could pick up on this but also if you're doing

memory forensics it sticks out like a th a sore thumb. Emoteet injects code into literally every process it has access to. So you will get like if you run maline on a clean machine you get no output. If you run malfine on a memory sample from an emote infected machine, you get like 200 line 200 sets of output with zero false positives because it's anywhere it can inject code to live, it's going to do that. And you can see that here we're finding it one of its this slide was obviously cut, right? One of its reflectively injected DLS and then one of its like 100 plus shell code blocks. I just snipped out the ones that

look the best and actually fit on the slide. So at this point we can detect what I would call the traditional techniques. You still see these all the time. Uh like I said, emote was talked about by multiple people today. But what we really want to focus on is the sophisticated stuff. With this background that we now have, we can start to understand it. So as I mentioned, Google as of three months ago called process hollowing super sophisticated, but as you'll learn in a couple slides, it's not exactly new. It's just something that EDRs are at a major disadvantage when they try to detect. So, and this is a bunch of uh notes that I'm definitely not going to

read. If you want the slides uh after, just email me. I can send it. But basically, everything is on that's on this slide I'm going to talk to in this nice visual. So, the idea with process hollowing, this is process hollowing like V1. The very first form of it seen in the wild. The idea is the reason we're call it hollowing is we're going to take a legitimate process from a legitimate executable and then we're going to hollow it out so that the operating system still thinks that legitimate executable is running when in fact it is our full-blown malicious payload. So what we're seeing here on the left is the normal Elsas.exe. So LSAs is

obviously not malware. It's a major part of the operating system. And what we're going to do is abuse a feature of Windows called uh what we call it like creating processes in suspended mode or a suspended process. The idea here is when a process is running and it's going to create a new process, it can create it in what's known as suspended mode. So what Windows is going to do at that point is what we're seeing on the left. It's going to make a new process. In this case, it'll be process ID 920. It's then going to start making those memory ranges that we saw before with the permissions and the addresses. And it's going to make a memory range for

Elsas itself because we're starting an Elsas process. It's then going to make a memory range for the PB that is a bunch of like metadata and bookkeeping. And then it's going to map in the two critical DLS, NDLL and colonel 32. Processes can't do a whole lot if those aren't available. So at that point because the malware that's creating this LSAs told the operating system to make the process in the suspended state, Windows is going to stop at that point and then give control back to the malware. Does that make sense? So the malware is created at this point kind of like a halfbaked process. It has memory ranges for the executable and your two

critical DLS and that's about it. It doesn't have all the rest of the DLS, doesn't have all this other data that it needs. So think of it as like a halfbaked half-created process. So now that the malware has control again and the fact that it's starting this new LSAs, it can manipulate it. So in this early form, what it's going to do is find the memory range that's hosting the legitimate LSAs, unmap it out of memory, and then override it with the malicious executable. Once that's done, it's going to resume the process and then that process is going to load like normal and the malware can do whatever it wants inside the process. Does that make

sense? Okay. Now, the key here is you might think, oh, well, if I run task manager or CIS internals or look at my EDR telemetry, u I would figure this out the but unfortunately you'd be wrong. If you in your telemetry, your favorite EDR, all it's going to say is that Elsas started at a certain time. If you run task manager, TCP view, OSQuery, whatever it is, anything reliant on the live system APIs, all they're going to see is that the Windows signed executable at that path called LSAS is running in a process with PID 920. And you'll have no idea that in fact some of the most sophisticated malware out there is actually running on the machine. And

as you could probably guess, the only way to figure that out is memory forensics. Does that make sense? code is abnormal. >> Um, like Linux doesn't have an equivalent. It's it's used for some like very Windows specific features, but it gets abused by malware like 20 different ways. So, >> it's not necessarily a signal or something. >> No, you no. And uh, talk to me after if you want because I did a whole Devcon talk almost just on suspended like finding suspended stuff. Um, but when you resume it, all the artifacts that would tell you that process started suspended go away because they get like unset. It's basically like a boolean like a one bit right to say suspended or

not sort of it I wrote like a 30-page paper about it if you want all the details but uh so anyway so with hollowing so this is a a memory sample you'll see which famous malware this is in a few slides this is also from 2011 if that gives you a hint um where it did process hollowing by creating two Lsasses so the the workflow that we saw on the last slide it does that twice now we have two hollowed out LSAs process processes and you should only have one. So on a normal Windows system, you should just have one running. So we want to figure out what is the fake one. So we get the process IDs of all the

Elsasses that are running. We send those through VAD info and we filtered it to just where that main process executable should be. If you look at the legitimate one, it's page execute right copy and the path is LSAs versus if you look at the two hollowedout ones, basically they had this workflow done against them. The memory region that should be hosting the executable is page execute rewrite. So that's already incorrect and they don't even map a file and it doesn't make a whole lot of sense to have a process running that isn't backed by a file like you need an .exe there, right? So this is obviously an issue and in this case we're manually picking up on it. If you

had a thousand memory samples that you wanted to sweep through, let me take questions at the end just to like speed up a little bit. um if you wanted to do this across like a thousand memory samples, you don't want to manually have to do that for every process. Right? So in Volatility 3, uh we started off from what was a great piece of work from a researcher. Uh with Volatility, we run a plug-in contest every year where you send us cool plugins and research and if you get like top three, top four, we give you like a bunch of money and stickers and t-shirts and all that. So back in 2016, we had an incredible

submission of a plugin called Hollowfind. If this presentation is the first time you've heard of process hollowing, go read his blog post. It's still incredible. It's like an hourong read. He's literally singlestepping in a debugger of all of the process hollowing techniques that were out there at the time. And then he figured out how to detect them all in his plugin called Hollowfind, which is what he sent to the contest. So for volatility 3 what we did is we started with that plugin for volatility 2 ported it to volatility 3 and then we updated it for all the process hollowing techniques in the last almost decade. So what you saw over the last couple slides that traditional like

v1 of process hollowing that was stuckset. So that's how it hid on machines it infected with our hollow processes plugin. It really freaks out right because it's saying hey there's this memory region where the the.exe exe file should be it is page execute read write supposed to be execute write copy and then it whines about the fact that it doesn't even map a file so you just have that automation across it stuckset's pretty old so looking at something newer this is a fairly recent cobalt strike it does basically the same exact thing that stuckset did over a decade ago so it's uh hollowing out the process setting an execute readr region uh the file isn't even name isn't even

mapped correctly and we pick up on it right away uh before Someone asked me who might know this stuff a little deeper. Are we just flagging execute rewrite? Obviously the answer is no. We're looking for anything that isn't uh execute right copy. So just to preemptively uh get rid of that question. Here's a slide of uh I changed the names of these processes because some of them were real malware. But you can see in this case it's all different process hollowing malware variants and there still don't have a file associated or even if they do have a file associated the regions page read write instead of execute right copy which tells us right away where the malware

is. Now another cool trick malware does is they change what's called the image base address. This is what tells the operating system where the main executable should be. So instead of what you saw a couple slides ago where you directly overwrite the memory range where like Elsas was, a lot of malware will just make a totally separate memory region and then write its executable there. But for that to work, you have to update the image base address to say, "Hey, here's where my new executable is." This is the easiest detection in all of Volatility. It's literally two lines of Python code because in the kernel there's a backup copy of that data that uh regular malware can't

manipulate and it's literally an if statement to say for this process does the section base address equal the image base address? If they don't, it's a process hollowing with 100% certainty. And that's what this is kind of showing you how we can compare those. This is malware. Um, I didn't have any malware that I could like make a slide for publicly. So, I found a GitHub project that did the same thing. It's linked there at the bottom. But basically, the plugin saying, "Hey, the section base address has value A. The image base address has value B. They don't match. That would never happen unless you have process hollowing." And this detects a lot of stuff that is out there in the

wild. So in summary so far, uh if your code injection technique or your process hollowing technique makes it where the memory regions aren't execute right copy or they don't map a file, then you're out of luck, right? Like we're going to detect you with somewhere between one to five plugins uh automatically. So you might try to think of some bypasses and there's two things that would come to mind. One is well what if you could map your malicious hollowed region as execute write copy and assign a file to it? that would kind of suck for volatility, right? Everything would fall apart. And then the other thing is the P hitters. So we'll talk about execute

right copy first. Um talking about suspended threads before. Normally Windows makes life very difficult as a memory forensics person where you want a bulletproof detection. Uh page execute right copy for whatever reason is like the one place they really help us out because as malware uh you can allocate memory regions on your own and you can set the protection. So, whatever that initial protection is going to be, but for whatever reason, it's highlighted here in red. Microsoft does not let you set page execute right copy. You can set literally any other protection status besides the one that we actually care about. And so, this really saves us because malware can't just bypass us that way. Now, if you really know

Windows internals, you might wonder, what if you created sections? Sections are another weird Windows quirk where you can set up a memory region in your process. So in your malware process, you can put all your malicious data, your malicious executable, and then you tell Windows like, "Hey, I want you to actually take this malicious region that's in my process and just go copy it into my victim process for me." And Windows is happy to do that if you have the right permissions on the machine. So that is create section map section. The great thing or the one downside here I guess is you can make these sections that are execute right copy. Windows doesn't block it, but you can't

associate a file with it. So then it looks extra weird like your execute write copy but you're not mapping a file. All the plugins in volatility will still point you right to that process hollowing uh technique. The other thing I'll point out is down here at the bottom. One of the most useful uh GitHub repos if you really want to learn this stuff in the weeds. It's called Offensive Panda. They have probably and then under the process injection techniques repo. They have working fully functional not backward source code for at least I don't know 30 40 techniques. So a lot of the samples here in the slides if we couldn't uh for certain reasons like use a volatility output

from our real cases we just went and compiled their projects and did it. So if you want to test your EDRs if you want to test your own like Windows internals knowledge malware knowledge that repo is incredible and this create section stuff I'm talking about there's a whole project uh just for that on there. So the overp headers this is where stuff gets fun. So you saw before how Malind was able to tell us like hey there is this memory region there that's hosting a PE file. What some of the stuff that the plugins rely on for detections also requires the fact that we can figure out that a memory range is hosting a PE

which is normally that it starts with capital MZ. So this is a problem. So then what happens is uh malware will overwrite those headers once the executable is loaded. The header doesn't matter anymore. And so though, so then whether you're volatility or you're your favorite EDR or your AV engine, if you want to go look through memory for PE files and the header that tells you it's a PE file gets overwritten, well now you don't know that a PE file is there. Does that make sense? So uh we're going to use a backup method, which are threads, to figure out where code is executing anyway. But before we do that, I want to point you out to this other awesome blog

post that I read a couple months ago from Forinet. Again, not vexity, but uh you can see the blog post title here is a deep dive into a dumped malware without a P header. So, these people took it to the extreme. This is another like 30 minutes read unless you're like a super expert uh where they get the malware out of memory, it wipes this P header, they reconstruct everything to the point where they can actually run that malware in their sandbox. So, another really cool blog post. Um and it's tackling exactly what we're trying to fix here. So in this case, we can't just figure out kind of look at the VADS, the memory regions, and say, "Hey,

you're hosting an executable." So instead, what we're going to do is figure out where code is actually executing in the process. This is the threads plugin in Volatility 3. It's way better than the one in Volatility 2. So for any address associated with that thread where code is executing, we'll give you the address and then we'll give you the path on disk if there is one uh of where that is actually executing. So this sets us up to be able to do a bunch of comparisons. So the idea is we're going to do a cross compare. We're going to get all those addresses where code is executing inside of every process and then programmatically we're going to

compare that against what you saw with the VADS. We're going to map it back to its memory region and for every thread that's executing we're going to say for the memory region that it's in. You could probably guess is it execute right copy and is it back by a file. If it's not one of those then we're going to uh print it out and tell you where to go find the malware. So this is the suspicious threads plugin. There was no equivalent of this in volatility 2. It's only in volatility 3. And it does exactly what I just said. So you can see for uh these were definitely renamed too uh because they came from real samples.

Um so this one it wasn't the browser. I don't know what it was. It was called a Chrome. So it's saying chrome.exe. Uh it has a memory region that's hosting a thread. So one of these addresses here related to where code is executing. It's pointing back to a memory region that is page no access. That means you shouldn't even be able to read from that region or the process should crash. So obviously there's manipulation there. So it's page no access. Um the one the next one's page execute write. The next one's page readrite. Then it does the second set of checks to see if they're even backed by a file. So it's saying in south.exe,

hey, there's a thread running. So there's code in a memory region not backed by a file. So it's going to complain about it. To your question about suspended threads, if you find those interesting, uh, one of my favorite black hat talks I watched in the last few years was called dirty vanity. Uh, basically they were able to bypass EDRs by abusing suspended threads. So the first thing I did after reading that paper, which it was like a 20 30 page paper. It was really nice. I wrote a plugin that found suspended threads in memory. It immediately picked up on the dirty vanity project, which was cool. And then I ran it across our huge data set at Flexity, like 500 plus

systems of of mal of memory samples, some infected, some clean, some whatever. Uh, and the only there was no false positives. The only threads that this pointed out were some process hollowing techniques and this dirty vanity technique. So the suspended threads plugin didn't exist in Volatility 2 either. It's only Volatility 3. And then to wrap up, cuz I I only have a few minutes left. Uh, but this stuff is cool, so I wanted to hopefully fit it in. Uh, it's called process ghosting and transaction tampering. This is super in the weeds. Like, you cannot be more in the weeds when it comes to Windows with this. But the reason malware uses it is, it's actually pretty funny. Uh, one of the

things they'll do is start a process for an executable. So you have your malicious executable on disk, but the malware marks it as deleted and then it starts a process from it, which doesn't make a whole lot of sense to have a process running from an application that was supposedly deleted, but that's the state they can force that executable in to the operating system standpoint. So then your EDR sees like, hey, you're starting a process from this path on disk. it goes to read from that path and Windows is like hey actually you can't read that file because it's deleted even though it's very obviously running in a process doing whatever it wants very maliciously. So a super interesting

abuse of the APIs and then another one transaction tampering is where you basically get the file system operations out of order. So the executable that gets mapped into memory to run as a process its contents are completely different from what an EDR sees when it reads it from disk. And so you have this malware on disk that looks like normal uh if you try to use the live system to read it, but in memory it's very malicious. So we have a process ghost in plugin that detects all these variants. In this case, you can see the delete pending bit is set on it. So that's like where the operating system won't let EDR scan it because supposedly this thing

that's called kelp.exe, but you can see has a obviously a very faked like hollowedout path. Uh so we pick up on that right away. This is pure process ghosting where again you tell the operating system like hey delete this file but while you're deleting it run a process from it. And so your edr will be completely blind to it but we pick up on it right away because uh it obviously doesn't make sense to have a process running from a supposedly deleted file. And then my favorite so this will always be the last slide before questions and the conclusion is people still thankfully more rarely will not do memory forensics. they'll see like an EDR alert and they'll turn the machine

off. This slide pretty much describes why you shouldn't do that. So this is Cobalt Strike. One of the things it does is it abuses those APIs we were just talking about to mark its executable as delete on close which what that means is when that process goes away, when that process terminates, the operating system is going to delete that executable from disk. So if your idea is I'm going to walk up to a machine and shut it down or I'm going to walk up to a machine and say hey this is a malicious process. I'm going to terminate it with like task manager or CIS internals. Well the only thing you've done is told Windows to

delete that executable from disk and now you don't have a whole lot to investigate unless you immediately try to do like file carving. Does that make sense? Versus in memory forensics. Look how easy this is. You run process ghosting plugin. the only line of output you get is the process that's been ghosted, which in this case is cobalt strike. You can then dump that out of memory, do your reversing, do all your other memory forensics, and you know, right where the malware is. Versus with uh more traditional approaches, all you did was get the executable deleted from disk. So, in summary, use Volatility 3. Everything uh that I just showed you was on GitHub and you can use right away. If

you find the super gory stuff interesting, uh vexity.com/resources. We've done like a bunch of Defcon black hat talks. They make you write like 15, 20, 30 page white papers about whatever it is. So, u multi-hour long talks with lots more Windows and malware details. And with that, I'll take questions if I have time. Feel free to add me on LinkedIn, email me, anything like that. Thank you for coming. [applause]

[applause] >> Any questions? volatility um so I'm not sure as you mentioned using analysis all that you can share some you can't obviously involve justility and what would you yeah what do you think the future of that is for volatility EDS a lot of what EDRs can't do with volatility when I small socks that can't really have enough time to pull an investigation package you know go dig into the volatility open it uplift >> uh there there's crazy levels of automation so I'll say on the side we have crazy levels of automation that uh that we use and some of our customers use but we've also heard like we have a lot of friends in industry who have

written And you know it requires like an engineering team people can code whatever but especially volley 3 uh by default has the different output renderers so like JSON or CSV or JSONL and um what's it called like parakeet or parakeet I'm going to get burned for forgetting the name is that output format that like a lot of uh detection engineering platforms ingest. So people have like written support for that and [clears throat] the way volatility 3 is architected you just write that function once and it's probably like 10 lines of code because you're just taking like a list and saying hey here's I want to format it or whatever. Um so a lot of those teams have done that because you

know if you're using like a good like enterprise acquisition solution right or product or whatever you can bring back like hundreds thousands of samples and then they're just they have pipelines that just auto process them and then kick it into whatever other automation. And think about like detection engineering if you can write rules in some generic system like you're getting telemetry from your EDR the event logs whatever for different events like process creation DL loading all that like volatility becomes another source a much richer source right across however many memory samples you can process so um even socks and stuff and like MSPs that we work with I mean they're processing like terabytes of memory

samples uh automated right so so yeah but it's all driven by this right like the same det detections. So, so yeah. Any other questions? If not, I'll be around the rest of the talks and at the after event tonight. So, thank you again. [applause]