Surfacing a Hydra: Unveiling a Multi-Headed Chinese State-Sponsored Campaign Against a Foreign Government

Mark Parsons: Hello. Mic, check 123, hey guys, morning. Don't call me sir. So have you guys ever wanted to track an apt? Have you ever wanted to see if maybe that one abt was three welcomed my talk serving a hydra, unveiling a multi let's see a multi headed Chinese state sponsored campaign against a foreign government. My talk is going to teach you, if you don't know a little bit about threat hunting, a little bit about threat intelligence, but through a story based mechanism, specifically, when we identified three Chinese nation state actors, moving through a singular environment, kind of deciding to work together. My name is Mark Parsons. I'm a senior threat Hunter at Sophos on their MDR apt team. This

conference is the first conference I ever attended two years ago. I've only been in the field for about five years. This past six months, I've presented at six conferences, pivicon, Black Hat, sector, Microsoft blue hat this week. So this is my last one. I can't wait to drink a beer with all of you tonight. And if you ever have questions about threat hunting, threat intelligence, anything, this is how you get in touch with me. I would love to talk about it. I have signal and all that stuff as well. So we're going to talk about this campaign in two different stages, while we have stack numbers, Sophos, threat activity, cluster numbers, those are of giant mouthful. I'm going to refer to them as clusters

Alpha, Bravo and Charlie, to be a little bit more conversational about everything. Again, as I was stating that we believe them to be Chinese nation state actors. We believe in to have some level of coordination. I'm planning to prove that to you right now when we walk through the get to the campaign and different aspects on top of that, we initially identified one cluster targeting a Southeast Asian government, and we then tracked them in the second stage of the campaign around that region, targeting multiple actors, as we'll discuss again. So we're going to break down stage one of the campaign, how we identified them. What do they do? How do we form the actual cornerstones of our threat clusters? After that,

we're going to describe the second stage of the campaign. We're going to go into some Situ gap analysis, which is, how do apt actors use both their custom tooling and open source tooling when put under pressure. And finally, oh, and then I make it. I made a tool to do it. So I'm going to give that away. Show you how it works again, show you a different way of hunting. And then finally, if there's time for questions, should you want to ask them here or later? Let's go. So again, as I said, Southeast Asian government, let's see, it was initially a campaign that was targeting this single government agency, while we believe that the while we started tracking them in March

of 2023 we have evidence identifying that their attack started all the way back in 2022 right in advance of when the current president, Bongbong actually was right before he was elected. So there has been a large history of geopolitical tension inside of this region, specifically with China and the multiple governments that are there. This is one of the most the strait that is there. I don't think I'd ever use a green laser pointer as a lot of economic power. There's a lot of trade that goes through the region. This nation, specifically China, has a long and storied history of being a cultural powerhouse. They have a narrative where they believe that they have a mandate, that they will take over and assume

the next position of power within from the US now. So I really want you to be afraid of what this is. These actors are persistent, and they are they have a bold mandate and will continue to escalate their capabilities. So a little few words about this client, specifically in the Manage detection and response team. We onboard clients, and we watch their entire environment, we respond to all the threats that occur within that environment, and we but we act on behalf of our clients. So what that means is that our contract can take a few different forms. We can just be given the right to respond immediately. We can be given the right to say, Okay, I'm going to

call you up if you if you pick up the phone, great. Let's talk about it, and I'll figure it out. If I can't get you on the phone, I'm not going to do anything, because I need authorization. And finally, when the state of this client is that you're just going to notify them if something bad happens, because a lot of people are like, OK, you found this actor. Isn't it your job to kick them out? Yeah, it is. Well, the client didn't respond, or they did respond, but their actions were not necessarily as committed as they should have been to evicting the actors from the network. On top of that, they had what's called a mixed estate. So our sensors were

deployed, and this is a Sophos EDR telemetry sensor only to the servers within the environment, so not endpoint. So it was really frustrating, because we'd swat at them, and they'd go hide in the corners, wait a few days or 24 to 48 hours, and they'd come right back. And so they had enclaves in the environment that allowed them to persist. So again, when you're doing IR, should you be doing? That field. This is exactly why you need to be so complete with your eviction. The reason why I'm talking about this is it left us between a rock and a hard place, and so us as actors and responders, as responders, not actors. Excuse me, what are we supposed to do if we're seeing

all this activity within the environment and we can't get them out. Well, we're going to monitor them, because they're these. The Chinese specifically are known to reuse their tooling, reuse their tactics, techniques, techniques and procedures. So if we can sit there and I can wake up every single day, which I did for a year and a half, and log in and say, Oh, hey, what are you doing over there? I'm going to go look across the rest of the clients and decide and discover what else they're doing. So that's why, for a year and a half we tracked them very specifically. It was awesome. So how do we find operation crimson Palace is what we call this through threat

hunting. And so I'm going to, again, want to teach you a little bit threat hunting is functions under the assumption that we have had that a breach has occurred and that threat actors have evaded our sensors in some manner. Right now, there's lots of different ways that you can do threat hunting. I'm not going to tell you to go grab an IP address off a threat list and then just lower look for a hash and say, Is this in my logs? Retroactive hunting is a thing. It can be effective. But again, we want to be forward thinking and try to protect our customers through actions and techniques that have yet to be identified or understood. So what we're doing here is using

PowerShell TCP client, a PowerShell class for command and control infrastructure. Specifically, what's going on is that that we are seeing a VM net executable right top, left there, and if you look at the context in the bottom, oh, it's covered up. I'm sorry. So apologies. So it's in a it's residing inside of a Microsoft directory, inside of program data. There's a VMware executable inside of a Microsoft directory in program data, using a PowerShell TCP client class to then communicate to a dot info domain. That sounds pretty weird, because the other aspect of threat hunting is using the context of what we apply that thesis to, right? So sorry if I didn't say that in threat hunting, in one approach, is a

thesis driven approach, or a behavior based hunt, where we take a fingerprint what we believe malicious activity to look like, we pattern it through our logs and then identify, did it occur here? Yes or no, and then we use the context surrounding that event to say, is this malicious or not, or is even anomalous. Let's start there. VMware executable, Microsoft directory, PowerShell, TCP client to a dot info domain. Pretty anomalous. Second thing we found we traced out this process tree, is this Sophos UD executable. I work at Sophos. The reason why it's red is because it was unsigned. I can tell you for a fact that we don't have a Sophos UD executable, and we absolutely don't use unsigned binaries.

This is a masquerade. So the threat actor was purporting themselves to be from us because they knew we were watching them, and they were trying to go under the radar. So this is a common technique from threat actors, where they'll rename their binary to be something benign. Ok. Now this last item, SSL, WND, 64 really made us jump in a great, great way. It maintained a direct hash match with phantom net, a piece of nation state malware that had been identified attacking other countries within the South China Sea a lot of contacts. Seems pretty bad, right? Yeah, it was. So we went into their environment and started racking and stacking you're taking all sorts of

behavior patterns and saying, What's rare, what's anomalous, what doesn't happen. We found 13 different malware families across a quarter of their server infrastructure in the first seven days. If this is overwhelming to you, it's overwhelming to us as well. There's a lot going on here, and we called it a critical hike situation, and I was the primary investigator for all this too. So I'm like running, running around, ranting. And by primary my the operations team was involved in prior events, but again, the client was not necessarily responding effectively. And so I took it upon myself to start correlating and collating all of the data. And so when we saw all of this occurring as one, as an offset,

we decided to dig in. Now, how do we go? So I was sitting there, right? And let me just go back one there's a lot going on. And something that wasn't adding up is that when you look at threat actor patterns, they typically form a roughly linear pattern, right? And so what we saw was threat actors deploying command and control infrastructure to different directories they were using this the command and control tooling was deployed through different accounts, but it's also doing the same thing. So there'd be like, two different implants on one box. And so why would you do that as a threat actor? Your tooling is important to you. You don't just throw it all out

there, right? Again, we had unknown malware families. Why are you putting multiple implants that look like they're doing command and control on the same box. And so I'm sitting there in my team. I'm like, guys, hold on, this is really weird. That's me. I do have a giant whiteboard in my office. I am pretty short. I don't smoke, but I have been known to draw a lot of pretty wild conclusions from data. So I'm going to my team. I'm like. Guys, I think there's more threat more than one threat actor here, and they're like, Okay, sure, prove it. So I part of my mustache, I dove into the logs, and we begin to form our threat actor clusters, okay? And

now the way we do this is we weren't gonna we're going to grab every single piece of telemetry that we can. So I had authentication, material tracking, where a threat actor authenticated from into what account they used. I had process data so I could generate my process trees. I had file write activity so I could see if a piece of malware wrote another piece of malware to disk, right? I had network telemetry so I can look at the command and control infrastructure. I got everything I could, and I started to form what's called attack flows. These form the cornerstones of our clusters. And what an attack flow does is the pattern of activity that a threat actor

takes throughout an environment. And the way that I'm going to break down and share this to you is we, when we did all this triage by recreating months and months of session data, right? It sucked, but it was really interesting and fascinating, so I couldn't put it down. So I'm going to share the data and information, and how I form these clusters is by looking at the attack vector or the machine that they attacked from the user account that they used to authenticate to their target machine, the target machine the malware they deployed when they were there. And the slides are online. I presented this at Black Hat, if you're like, this is cool, or I'll give them to

you later too. So yeah, just saying. Finally, after the malware was there, what did it communicate back to in the form of the domain or the form of the domain or the IP infrastructure? So I'm going to walk you through a few of the different attack plays. Let it happen. This isn't the important part. What I want you to take away from this is that when we're doing threat clustering, we have a lot of data points that we can correlate together with strong reasonings and overlap. And that's how we track threat actors across multiple clients and try to understand who they are within the threat landscape. Cool, sweet. All right, so they started from this desktop

machine. They used the admin F account. They moved immediately to the DC sync machine where they deployed the C corridor malware. This was custom malware. They immediately cleaned it up. Moved laterally to AMS machine. After they were there, they used an agent, the agent shim attack is what I'm calling it. We're going to cover specifically what it is in a little bit. And it was a defense evasion technique that was used to try and unhook our processes from or our telemetry from the process, so that their follow on actions would go undetected by us. They then used a lob in RDR, leak, Diag, a living off the land binary or an executable that is signed by Microsoft and

can be abused for things. But it's not necessarily outright malicious. They use it with a the slash one flag, which means, give me all that data back in one minute. Okay, dumping creds. After that, they used the local admin account A few days later to authenticate to the file machine, file server where they deployed the C cord or malware. There was a difference in this attack. It actually began to communicate externally to the messages dot ugaid domain, and then it had a single IP address that we resolved the network communications to. They immediately set up a scheduled task. They had first attempted to move laterally to the hypervisor via remote services. It failed because they fat fingered the keyboard. They gave

me a different account. Different account. That was very nice of them. And then they used a remote scheduled task. After they deployed that scheduled task, it then also deployed C cord, or malware. You'll see that a that means overlap. And something I didn't cover is we have three clusters. And why did I start with Bravo? Anyone notice that, I promise I know my alphabet. It's because, again, we started at the beginning with cluster alpha, and then we said, oh, there seems to be other patterns here. Cluster Bravo was actually chronologically first within the environment. What we identified right here at this time is that actually the day before we saw this C cord or deployment, is that there was a deployment from cluster alpha.

And so what happened is that they actually deployed this and communicated externally. It was set up to be recurrent, but they never did that, and they actually stopped their attack right there. Why, if you've been cleaning up your malware and you've been moving rather quickly, do you then pause? Why do we see a change in pattern of behavior? Maybe because there's de confliction. It's a military term that means, if you see you're going to see, Oh, I see patterns of another group here. I think that I know that I have another person that I know operating here that's friendly, friendly to me. And I don't want to interrupt them in what they're doing, because when they

came back, it was also unique as they moved laterally to end points. Now they stopped attacking servers, so a change in behavior happened post. All of these things are important because, again, we're trying to create clusters, because we have a bunch of different actors, we need to untangle all of that. So repeated this process with cluster Alpha. They attacked primarily from the VPN subnets. They used a different subset of accounts. They attacked a different subset of machines. When they were there, they functioned primarily through the Merlin c2 agent, which is an open source framework kit that they slightly modified. The Chinese are very well known for this. And they used the VM net DLL, which actually took two

DLLs and did some agent shimming with a gadget in between. We wrote a blog about it. Check it out. There's going to be a QR code later, but it was the VM net executable that I showed you earlier. Now, intriguing about this. This is that I went to go on and create and directly write to disk all these other pieces of different malware. There's a lot of capability here. The query and aquarium back door is another thing that's been communicated or published on by Bitdefender, specifically and back in 2017 attacking clients in the Middle East. Rude bird is an MSI 64 both us and elastic labs identified this piece of malware. They only found the memory aspect of it. We had a

more capable sample, and then the pow heartbeat. That's that Sophos UD executable. They also use a different few different mechanisms for deployment. OCI dot DLL is a phantom DLL side load. If you're haven't heard of that, check that out later. Red teamers is the fun way to attack. I promise you, I'm watching those services. After that, there were other instances of standalone malware as well. So there's a variety of ways that these tools were being deployed, but again, they all came from the same accounts to the same machines, and they deployed other pieces of malware. This eager B tool was used as a defensive agent technique distinctly different from the other attacks. Now what's uniquely different,

right? Is that, unfortunately, my map is drawn over there. But what I care about telling you here is that there is an overlap between the hard coded IP infrastructure in the rude bird sample and the eager be sample. So again, we have, bless you, we have multiple pieces of malware being deployed from the same places, and then multiple pieces of malware communicating back to the same things. That wasn't the only overlap, Phantom net and eager B also had overlap. There was standalone infrastructure, but it was all distinctly different from clusters Bravo and Charlie coming together. I hope so. We repeated this, and now cluster Charlie did the same thing. They attacked once from the VPN subnet. They used this account one time, and they

attacked the 365 machine. So this was very analyst because of the time pattern that happened. It was earlier on than everything else that they had done in their entire attack. They also used an account that had been tied to alpha, but alpha only used it one time from the VPN. So it was either initial access brokerage, or it was some kind of handoff, or two different groups just attacking the same thing that's vulnerable. You'd only see one side of the keyboard on the blue team. So they deployed the Poco proxy sample as a masquerading DLL, dot 443, dot text. They then used a different subset accounts. And what was intriguing is that once they were within the network, they actually primarily attacked from

the endpoints. They used poco proxy as well to attack deploying it in both single one off instances, as well as recurrently, it went on to deploy other instances of itself. And then on top of that, it used the max fee file lock, side load. And I think they messed this up, because they spent 12 hours that day like fine, shrink, fine, shrink, fine, shrink, start, start, start, trying to see what their malware was doing. They then, in my opinion, used a defensive agent technique that was mildly successful, but I'm not going to talk about that, and they then deployed the LSA credential interceptor, writing clear text credentials to disk, a UI loader for command and control, again, a different mechanism, and then

almost at exec, a slightly modified version of impact of at exec, the impact module, again. And there's a third domain here that is also has the speed test component in it. And that was another pattern or commonality that we saw from all of this malware posts, right as they're all using similar domain construction to try and communicate back to I really wish that third one was there I promised you. It also says speed test. So there were again, different IP hosting infrastructure from every other cluster. So you're thinking like this, hopefully that this is robust and they're distinct. Now that was how we looked at those clusters as individual capstones, right? Now what we're going to do is look at them

together, and you can see that every while there were overlaps, right? Same server, same network, Chinese working hours. We'll talk about that. They also seem to function. They had a lot of components that were separate from one another, right? They had their own very specific malware. They had different time spans. They're all doing different things. So while there's overlap, I'm not denying that there's also a lot that didn't seem to add up as to being one actor group. Now attribution, threat intelligence is, in my opinion, based on clustering approach. The reason why we use clustering is that we want to be malleable in our thinking. As we gain more data and understanding, we may have made a false assumption. We

want to be able to say, Oh, I now have more information. I'm going to go re evaluate my data. Oh, my word. What I thought was one cluster turned out to be three. So we want to be flexible. Clustering allows us to take these different aspects, the motivation the attacker, that they're using, the attack, that they're the target, that they are the victim, that they are targeting, the tooling that they're using, and then they're like, CVEs. It's all a whole bunch of different stuff. I'm honestly not a threat intelligence analyst. I just play one, and I try my best to hang out with them. There are a lot of smart people that are better at this than I am. But

again, we want to use clustering to try and do that. After we have created our own clusters, we want to attribute this potentially with external or other open source reporting. Use your own data first, please. I beg. View, it is not worthwhile for you to necessarily say, Oh, this is apt 15, because I saw this same thing. It's important for you to understand, when you're defending what your vertical is, and to understand the targets that may be looking at you absolutely but when you're trying to track these threat actors, you should be looking at your own data first to create internal analyzes, right then pattern that out. Don't start from the outside and move in cool. And that's what we So honestly, the parable that

people think of is the parable of the blind man and the elephant, right? Where all these wise gentlemen walk up to an elephant, they're all They're all blind, and they try to feel around on this elephant. They're like, Oh, I feel the leg, that's a wall. I feel the tail that's a snake. Well, it's all the same thing. And to me, this actually brings to point a big challenge in our industry is that a lot of the times when we think about defense, we're not going to necessarily be talking to other people that are in our vertical. So make friends, find those relationships. Because the reason why is that these are actor groups that are have a broad suit of targets in the

same vertical. They're going to go hit every single space agency around, they're going to keep using their same techniques, tactics and tooling to do the job. Talk to the people that you're working alongside in the industry and say, Hey, we saw this. You're going to come hit you next. So we need to be better to as together as a community, and better sharing needs to occur. I'll get off my soapbox. So again, I said time we took all of that activity, every single file, right, authentication, process, creation, and removed any sign of automation, right? Because we want to go down to the hands on keyboard activity and try and look at the actors. Something you'll notice is that it happens to be right, to be nine to five

and by Beijing time. So that's when we first started to really say with confidence, that really these actors, we took our own data and said, Oh, they could plausibly be from China here, right? So again, we're trying to use multiple lenses to create our assumptions and to create our analyzes. What was more intriguing, right? Is, again, we have multiple groups. What happens when we broke them up, they seem to all favor different days of the week. Alpha, Friday, bravo, Thursday, Wednesday, Charlie, Monday, Tuesday, Wednesday, right? Distinct, distinction, distinctive areas of the week where they're operating within the target network. Was there de confliction? Well, I don't know. The data looks like there may have been, of course, it could possibly be different operators

with different tool kits. That's absolutely a another hypothesis.

So, but again, we look at the data over time in a different manner. You'll see right here, it's like it fits right there, like they said, Hey, I need this. I have an action error. I got to be present. Or here I'm going to come in right after you. So there are certain times of slices of time where they seem to work with coordination amongst one another, and they had different activity patterns that they would leverage. There are, of course, times where there's outliers, and there are times where I wasn't able to, necessarily, with high confidence, attributed this to a specific group because of the data that I had available to me. But again, I'm going to try and

set that in its own separate box and say, I don't know what this is. I know it's bad. I'm going to track it. Send me sweet sorry hat timer. I'm going to still track it, but I want to, again, try to make sure that it might be separate, right? I don't, I don't have a hun of confidence level to really say this goes here, here or here. It's something else, okay? Make sure you have the confidence in your own work to do that. So that was the specific actors when they're working kind of together. Now let's look at the operators behind the keyboard, cluster Bravo. They have their own defensive agent mechanism, so these are really expensive to develop. These are highly prized

within the red team tooling, because this is the way that they were going to evade our sensors and capabilities to detect and understand them. Where the way that cluster Bravo works is they use their renamed instance of NT DLL, dot DLL. Think of this as a pipeline from user space into the kernel the R, A, R. What they would do is they would write it to disk, map it into their command and control process, and then delete it rapidly, like 20 times in a single second. Those all three events, boom, boom, boom, boom, boom, boom, boom. And then they would do something else, right? And they would think that we could, they would evade us distinctly. Different from the

other clusters. Command and control. At the time, they were using the C cord, or malware specifically, and they deployed it to very specific users within the environment, which I would have had written out here. Unfortunately, they also used a unique wall bin to capture credentials, so their tooling was separate from the other clusters. On top of that, where do they deploy it? It's written there behind that whiteboard where it's specifically they deployed it to the secretaries within the environment. Why would you deploy stuff to secretaries? Don't we care about administrative privilege? Well, if it's cyber espionage, they're going to see a lot of data and documents going across their desk. So cluster, alpha, precise recon. They were really low and slow, and they would wait days.

They would hit one target, and they'd do a PowerShell evtx query, pull all the authentication, 4624, event IDs back, and map out distinctive administrators and directors of programs so people that maintain administrative privilege within the environment. So. Right from the classic sense that we all know and love and think about, and then on top of that again. But they would go like one they'd hit one person, they grabbed like, four months of data, go quiet for three days, come back him again. So they're very precise. They would only do it a few times. So they were really their technical targeting was very different. On top of that, how did they get around us. DLL, what's their evasion technique? They used a different

mechanism than the other actors, and the way that they did this is through Ed Wow DNS black holes. So they would implant their own driver into the network stack. That driver would then monitor port 53 for subsections of domains of common EDR vendors. When it saw one hit, it would black. It would zero pad bite the packet, essentially black hole in that packet. So if you're a defender and you're looking at your console, the machine is just offline because it's not talking back. It's a distinctly different mechanism than cluster. Bravo. Finally, testing in production multiple methods to reach the same goal as what that says. You remember the first slide that VM, VM net executable it's going to use,

where we first thing we saw was PowerShell a client, right or a scripting interface knocking back to their command and control, and then a few day, few hours later, so feels UD communicated back, Hey, can I get home? Okay, I can. Let's deploy my tooling. Great. They also did this in other, other ways. And if you're a pause, if you're a threat Hunter, I highly recommend that you look for this where a scripting interface, like VBS or anything like that, or PowerShell or command might communicate back to an IP that's rare, and then you see an executable that's also rare communicating that IP. Remember behavior based hunting. It's a behavior to look for. So they also made mistakes. Now, why do we care about it after

making mistakes? Because it means that they have they that maybe not everyone's a crack team, that maybe they're training other individuals. And so what they would do is I would see them deploy like a service, DLL side load to Ike exchange service, and then they would turn it off, and then they would turn it on, and then they'd deploy their DLL, and then they'd turn it off, turn it on, and then deploy their deal, and then deploy their deal, and then it wouldn't work again, and it wasn't taking their DLL up into their service. Two days later, they paused, and they came back and had a script did everything and worked right away. Maybe there was another team that they

went to to go and communicate with and say, Hey, Dad, my tool isn't working. Can you give this to me and make sure it does function? Means that the size of that group is bigger, potentially, potentially, right? But again, we need to look at the mistakes that our adversaries are making, and under making and understand what they might or might not be. This is a plausible hypothesis. Again, cluster Charlie. Their eyes were on the long game, so we'll see later that these are the ones that actually did targeted document capture. These are the ones that actually got conducted cyber espionage. They were the ones that were also they also used poco proxy, a distinctly different malware tool for command and control and

now actions on objectives. They use key loggers. We found an unknown as a time piece of malware called tattletale. Then on top of that, they did their reconnaissance against large swaths of the user base in a very noisy fashion. So this was like, in my opinion, I was like, this is cool, where they took PowerShell, queried those evtx logs hundreds of times over hours, and they would grab all that authentication materials, and then they scraped all of the IP addresses where those users were authenticating from. And then 10 minutes later, down to the second, they started pinging every single IP address. So user A IP address, a boom, boom, boom, boom, right down the row,

and it was 10 seconds or 10 minutes to the second. So a little bit of automation in their tooling there we tooling there we that's what we cleaned out. But they were noisy about how they did this. If we're a defender, we're looking for changes in behavior, changes in patterns throughout the environment. They didn't care if we saw them, as I'll show you in a few slides. Finally, how do they get around us? They use AV vendor drivers to try and disable EDR telemetry that at the time were unknown to be exploitable for this technique, again, and it was signed by another AV vendor, right? So yeah, totally different mechanism for exploitation and invasion. So after this, again, that's how we saw the different

actors as singular operators. And again, remember, we started with the assumption that there's a single actor present. So we took all of the attacks that were there and said, OK, they're just there. They just happened. I've timed them all out well. When we went back and said, Oh, hey, there's maybe three actors, what did it look like when they were all present together? We see very distinct waves of activity where cluster Bravo was first doing all of that maneuvering to those high value targets and deploying their command and control implants. Cluster Alpha came back and did highly precise reconnaissance against the directors of programs and understood the administrative privilege that was over seeing those programs.

Finally, cluster Charlie actually captured, took all that and profited on it, where they deployed their key loggers and actually captured those cyber espionage documents. So again, this is just a recap of what it all said. It's in the slides. So on top of this, I dropped my stuff. That's not the right slide. You'll see that next. So I want to talk to you about the MSS. So think of this as the Chinese NSA. They are essentially or CIA rather. Excuse me, the. This is an overarching body that is part of the Chinese government responsible for External Affairs. There are lots more policy wonks that are much better at this. There's probably some of you even in the room. Please come talk to me about

this. If you have more data on the MPs, I'd love to discuss them as well. But basically, you can think of them as a overarching authority through which they have subcontracted with either cyber mercenary companies to conduct their ends and achieve their goals. There's lots of this. There's a lot better talks on this, and I've attended a few of them, and I'm not going to try and tell you the wrong information. I have a few books, and if you want to talk about that, I'll get to me at the end. So China has lots of funding. Think about this. They're in this environment for what ends up being a year and a half. They're there every single day. They have multiple groups

that are targeting this environment. This environment. Is that cheap? Are you cheap for how much you work? Right? I mean, it's expensive to find good cyber operators. You're going to spend them. You're going to say, OK, go work there every single day. Make no bones about it. China is investing heavily, and has been investing heavily for the past, I would say at least 2015 where Xi Jinping has identified that AI specifically is very important, and their capabilities in the cyber realm is absolutely important. You need to be working hard at understanding, identifying what this is, and it's got even it's even further back than 2015 just saying. But the other aspect to think about right is how they're training

red teamers as well. It's a distinctly different mechanism and tooling than how we train blue teamers. They were investing heavily in these pipelines to create cyber operators and that they are effective. Please understand that this is a growing concern, and if you are not already awake and watching for this, you are making a mistake. So cluster Charlie came back after he swept him off the board. I was really happy. We rolled through and knocked a bunch of their implants across the board all off. And I was like, OK, I can finally take a break. Excuse me, I was wrong. Cluster Charlie came back and deployed a web shell. They conducted their initial reconnaissance in under 45 minutes, moving very

precisely through every single aspect of the IIS web server, understanding every aspect of the web server directories, moving very precisely to exactly what they needed to understand to eventually capture a DLL and exfiltrated from that from that web server. What they did with it, I don't know, but it did start this next phase of their campaign. And what I mean by this is that they really chose to start attacking with some fervor. So what do they do? They again started to capture documents. They capture they deployed key loggers. They started to actually conduct cyber espionage. Black Hat, USA, 2024, if you want the slides are there, just saying, but take whatever photos you want to what did they now? What did they

start doing when they're actually in their system? They deployed a different subset of custom malware to start trying to evade our detections and to start trying to evade me. They tried to use Shadow Copy Service DLLs. The reason why this is intriguing is if you look at the service capabilities of the Shadow Copy Service, or if you look at the permissions for the shadow copy service, it natively has the same set of permissions from the I exchange service, or, well, it has more permissions than the Ike Exchange Service cluster. Alpha in June, modified the registry key the service, high registry key for that service, and gave it those exact same permissions. So whether they're sharing techniques or not, I can't say, but it was a

difference in technique where they knew exactly that this service already had its permissions. Now they did make mistakes, and that's in that service DLL, and like, I feel bad saying this, because they're probably gonna watch this or one of these talks, but they misnamed it a few times. But they did reconnaissance around a bunch of other services. So if you're tracking an actor like this, pay attention to every single move that they do. I promise you, it's going to serve you well, not just here, but other places. And then finally, they took things to the next level. They were actually targeting our Sophos binaries to try and do service DLL side loading of them to masquerade that was great to hand in my

labs team. What did they do when they actually conducted cyber espionage? They were capturing all sorts of documents. They were looking for long run embedment within these systems. They were trying to understand the infrastructure and the maps that were associated with it. They were trying to capture all every single type of key and credential material to the environment that they could that's just the stuff that we as nerds care about. They were also capturing sensitive documents. They deployed an unknown piece of malware called tattletale for for sorry, for keyboard capture. It's pretty cool stuff, and essentially ensuring that they have full access to the environment. This is what they did over several months. They first started their attack here with that web shell I was

talking about. They used Trend Micro binary to conduct service DLL side loading. If you're looking for anomalous patterns of behavior, I highly recommend that you look for AV vendors that you do not expect to be there, especially if they're in anomalous directories, especially if they're renamed, they're trying to use our own defensive mechanisms against us. Please make sure you're looking for this. I beg of you. Let's see. They then started to use Havoc command and control framework. They hadn't been known to use that. So that was particularly frustrating to see they did LSAS dumping a different type of credential capture than we've seen from other tooling prior. They did their target espionage, and they also used a open source, real blinding. Dr tool to try and

evade us. This continued on. They deployed the g2 command and control framework. They did AB testing. They started to target executive branch, external assets from inside of their own government. Who would have seen who would have thought that? So what this really leads us to is Command and Control Framework analysis. I want us to understand what our adversaries are doing when they're put under pressure. Cool. Top left, that pearl bar, that's their custom tooling. I'm really bad at graphics. I'm sorry that you have to suffer through this. I promise I'll try and make this as entertaining as I plausibly can. So let's start at the top left. Go down to the bottom right. Hopefully that makes some sense to you.

I apologize if not. If you have a better way to do this, let me know. What's important here is that they, in my opinion, the reason why they did that is because they're trying to remain undetected. They're going to deploy their custom tooling when it gets burnt. They're going to switch to open source tooling because it doesn't have the same fingerprints, right? These are human beings and human operators. We can track them. If we're paying enough attention, focusing on the minute details, they're going to be different, or you can hunt them. Okay. What do they do after that? We cut after we stopped it. They started using DLL. They use a process injection, running their

Havoc DLL directly from run DLL 32 and why would they do that? That's COVID Strike. Oh. Each color is a different framework. Sorry, Orange is havoc. Green is jibro. C2 blue is cobalt strike. Why, if you have a bunch of functioning capability, would you deploy something in the middle of it? Maybe you think you're going to get caught, and you want to see what the defenders are doing, and if they notice, because you know eventually you'll get caught. And you want to know that maybe you have ways to get around them, and maybe you know that this way doesn't work. Well, that's exactly what they did when we evicted them from this mechanism to stop their process

injection. They deployed three different types of command and control in the same day. The bottom two here, there was the exact same name, DLL. One of them was a havoc Command and Control, or Command and Control reflected DLL, and the other one was a cobalt strike deal, like it's escaping me right now because I'm nervous and I'm on stage, I promise you, I wrote it down. It's in the blog, but it was cobalt strike. The DLLs were deployed to machines that were alphabetically one after the other. They were deployed within the same hour, from the same account, from the same location. It's weird. And then they deployed another standalone instance that same like, an hour

later, a standalone instance to an XE. Why are they doing that? They're testing us. They want to see and understand us as defenders. These are smart individuals. They're trying to build additional ways where they can try and evade us. They're also doing it in production, right? This is a live environment. Do you test your red team tooling in the customer environments. I hope this works. They have a mandatory they believe that they are going to win. Let's pay attention. So on top of that, after that stopped working, they switched to a different DLL side, load libs, theft, dot DLLs. And this is straight into, this was just Havoc straight away. When that, when we caught them, stopped that they switched to a the

zebra command and control framework. So they're modulating a different aspect. Started with process injection from run DLL 32 then did a different type of DLL side load, tried a few different command and control frameworks. Now they're trying a the libsev dll side loaded. They brought their own vulnerable to DLL side loading executable, and then they fired off their own types of DLL or command and control framework into that. There's a lot to say, very quickly, they're modulating multiple aspects of their attack chain very quickly, because they need they have a task and a mission and an objective. We need to keep pace with them. If it's tiring you out, it was tiring to me as well. When that stopped working, they started to

use shell code loading, shell code loaders. There are red teamers here that can explain these a lot better than I can, but that's because they were again trying to modulate their attack their attack chain, to try and deploy their tooling in a different way to evade our defenses. When that stopped working, they used a different type of shellcode loader here. Finally, when that stopped working, they used a different side loader, side load as well, with a different shellcode aspect, consistently modulating their tooling. Finally, where they had we forced them to go back to custom command and control framework tooling back there. That's what this looks like. This is in the blog. I encourage you to check it out,

but that's what they did. It was 28 different deployments over 28 weeks. Little tiring. Let's recap. So what is t2 framework analysis. They're taking a tactical approach. They're going to use their stuff that is, they're going to use their tooling that is, does not have fingerprints. First, they're going to then switch to stuff that's open source. They're going to have a deep tool set. It's a lot of different attack chains to launch off at once. Finally, they're conducting AB testing, because they're trying to understand and get around us. I need to start, oh, I messed up my joke. I'm going to start speaking a little more quickly, because I have less than five minutes. So again, you remember those heat maps pretty cool,

right? I really trust my colleague, Colin Cowie, that built them. And I said to myself, I bet I can predict when they're going to be there. Because again, what matters is trying to capture their tooling, right? We want to capture it. Bring it home, give it to our lab scene, break it all apart, see what's inside of there. It's pretty cool. And. So I said, You know what? I'm going to wait. I know that they're going to be on at 10 APM, local time, all right, fiance, I'm watching our movie. See you later. I got to go hang out, and I would just Control F through the entire environment all for the next two, three hours, they didn't show up. That sucked. They

showed up an hour later, at 2am I'm not going to be there at that point. So I built so this was me, when the threat actors show up and you were wrong in your prediction. So I built this tool to make sure I went off to do that again, the session, process, anomaly and discovery, examination. Tool, spade, because it helps you dig. That's really what I like to call it. What is spade? What is command and control? Command and control is where the threat actor is, the attacker is essentially the real control framework that allows them to move throughout the environment. Now, one aspect in the attack chain is the discovery commands, whom I der like all sorts of things, that

will give them data about what the environment looks like so they can better operate there. We as defenders want to find them that first, because while it is post exploitation, they have cracked through the external perimeter of our environment. They have yet to perpetrate their secondary actions. So on the attack chain, we're attacked. We're hypothesizing an anomaly based mechanism for threat hunting. We want to attack this aspect of their chain. Cool. It's really hard when you have 25,000 clients, when you have 400 million process records over 30 days to say, oh, that instance of ping is bad. Do you use ping for benign things? I do? Do you use task lists for benign things? I do schedule tasks.

Maybe that's useful. Useful for if you have an administrator that needs to do something routinely, these things happen all the time. So again, let's look at things differently. We're going to look at a subset of discovery commands. Discovery commands, 14 of them specifically at a very specific time and again. Because, sorry, I got ahead of myself, because, again, I'm trying to rush. We're going to look factor in human aspects of this attack, the attack chain. We're looking for human operators. So we're going to block time off into two hour blocks, and we're going to say when I see a more than two of these discovery commands in that single session. That's a discovery command session I care about. I'm going to map the

parent process that caused it, because what we're doing is trying to identify DLL side loading of unknown executables as well as process injection that are evading our defenses. And we're going to look at human based behavior to try and identify that or anomalies. So we took those sessions, again, we don't care about automation, so any high levels of process counts, we're going to counts, we're going to get rid of any repeat sessions. We're going to say, OK, we don't care about that. That's probably automation. These are the yet, again, a smaller subset, but again, we have 25,000 clients. If I only have four machines over 25,000 clients, it's like 100,000 things are checked every single day, and I'm pretty tired

already talking about it. So we're going to remove anything that has the exact same set of commands from the parent commands from the parent process to discovery command session, and say that's likely going to be a specific configuration for that environment. Because we don't deploy our command and control infrastructure everywhere, right? We deploy it to just a few targets. So this leads with, yet again, a smaller subset from only our process based telemetry. In my opinion, we want to look at our threat hunts from anomaly based perspective, from a variety of different manners. Process being one of them, or endpoint based telemetry, however you want to think about it, network or identity. Are there other two

major domains? But I got to move fast, so we're going to think about the network connections that command and control infrastructure makes. Specifically, when you run your command and control implant, do you talk to every single IP on the internet? Probably only talk to one, right? So get rid of things, to talk to a bunch of different stuff, and that's how we tracked and identified programmatically, 28 different attack chains with the push of a button across 25,000 clients. I'm giving this away. You'll have these tools. I want you to have them as well as defenders. So I talked about cluster Bravo. They were present for only a month in the environment, they didn't stop. They attacked 11 other organizations over the six

month span from or eight months span from January of 2024 up until I gave this presentation at Black Hat USA in this August, they targeted 11 different other environments. On top of that, they used previously compromised government agencies as command and control relay points. When they attacked other environmental vertical or other verticals, they would compromise other vertical or tenants within that same vertical, and then host their command and control payloads there and then ingress them. Because that makes it really challenging for us as defenders to say, Oh, this government Secretariat talking to that government Secretariat. They do that all the time. Do I really care about that? Do I have the chance to look at

everything at once? No, I don't. Takeaways, no happy ending. I wish very much that I could say that this was all well and good, and the problem is gone and it's not there here, and they are ramping up their capabilities. China has been completing these things for the last 10 years. They're only going to pick up the pace. Start looking now. We need more of our defenders to be aware of this fact. Stop being complacent in their attacks. On top of that, threat intelligence driven, threat hunting is a must with the environment. We need to out think our behavior our attackers. They are human beings and they are fallible. They will leave fingerprints on their infrastructure, their tooling,

and we can trace and track that tooling and identify them as they move through our environments. Finally, laws are cheaper than lawyers. How? Yeah, what that means is, if you're running an IR, you're already paying money the second you call the lawyer, right? Yeah, if you have to prove that your data wasn't exfiltrated, you want the log to say that. Finally, though, I'm begging you, if you have an environment, please, very much. So I need you to deploy your sensors to every single endpoint. I can't tell you how many times the attackers are going to move off to those endpoints because they're looking for the places where we have not deployed things, and they're going to make sure that they are going to reside there.

These people all helped and were phenomenal. I did not do this work alone. That's all I've got time for. Thanks, folks.

Unknown: You.