Operation CuckooBees: Deep-Dive into Stealthy Winnti by Ofir Ozer, Niv Yona and Chen Erlich

[Music] [Applause] [Music] hello everyone and thank you for joining us today we're really excited to be here in these sides uh for me it's the first time talking in a conference so i'm a little bit excited um thank you [Applause] and we'll talk today about the operation cuckoo bees that we investigated as part of incident response last year before we'll deep dive into the operation let us introduce ourselves so my name is niviona i'm working in cyberism for the past five years and i'm the research director right now before that i was in the incident response team with the fear and and five years ago we also spent time together in the israeli air force

and i was the soccer instant response team lead ken forced me to add a fun fact about myself so let's say i love to surf and show off my adopted dog eric hi everyone so my name is khan erlich i'm doing insulin response at nocturnal cyber region and before that i was senior security researcher at symptom and senior threat intelligence researcher at insilo and besides that i like to practice muay thai and mma and i'm doing it for the last 15 years and you can find me on twitter and lastly hi everyone i'm ophel and i'm also an incident response engineer at cyber reason beforehand there was a windows malware researcher at trusteer ibm i'm also a drummer and i really

really like football the right kind of football the one with the round ball and yeah that's it let's begin thanks so last year we got an alert for a suspicious credential theft in one of our customers environments from theirs we kicked in the investigation of months uncovering the full operation of windy that span in three continents north america asia and the europe but before we dive into the operation let's zoom out for an overview of the operation uh with that without this operation cucumbers so operation cucumbers is a espionage campaign um doing this campaign the true apt um sold terabytes of sensitive data from the customers environments and in all of other companies around the world

it was started from an erp exploit that they found in a public in an internet facing a server from there they deployed web shells in this patient zero um we found that they use novel espionage tools uh rootkey that we dubbed the zwin kit and a really cool abuse of windows clfs as i said it happened on three continents asia north america and europe and it was done by uh with the highly confidence as winty which is a chinese nation state apt that is also tracked as ap 41 blackfly and barium the motive behind this attack is geopolitics and economical but before okay and now let's get intimate with the winty the apt group who did this

operation to understand the motives behind this attack uh we need to go back to 2015 when china initiates the national strategic plan that coal made in china 2025. the plan seeks to engineer china from being a low end manufacturer to becoming guy and producer of goods china goal was to gain global market domination by achieving independence from foreign suppliers and other countries to do so the government chose 10 key sectors that we can see over here that they want to do that for example new information technology energy saving new materials and so on four years later in 2019 the fbi published a research describing china's risk to cooperate america in this report the fbi summarized the risks the risk

that china imposes to the u.s economy prove the lacking problem the acting program a few months ago an fbi director spoke in uh um sorry going back um in this report they they mentioned that the annual cost um to the us economy um for counterfeit goods piloted software and theft of trade secrets is between 225 to 600 billion dollars a year and china is a big part of it also in the report they state that in order to get this impact the chinese government is using illegal ways such as trading secrets stealing proprietary data and from foreign countries and to achieve this goal went the acts and we believe that this operation is part of it

over the years the winter group initiated thousands of successful attacks getting headlines all over the world and a few months ago an fbi director said in the 60-minute show that the biggest threat for the american law enforcement is from chinese actors stealing proprietary information according to him the view opens a new china counter intelligence investigation about every 12 hours and we can see his quotes from this show they are targeting our innovation our trade secrets our intelli intellectual property is at a scale that is unprecedented in history they have a bigger racking program than all other major nations so now that we understand the motives behind this attack let's go back to winter and get to get intimate with them

um so as i said winty is a chinese apt nation state apt they are active from at least 2010 um until today and they have free focus areas and they are state aligned cyber espionage ip theft and cyber crime including ransomware cryptocurrency miners um and fraud and they're active worldwide so what is the progression cucumbers operation cucumbers is a multi-cyber espionage intrusion during the investigation we found that the intrusions are there from at least 2019 and they were undetected with the goal of stealing ip from tech and manufacturing companies around the world during the investigation we could attribute the operation to winty the chinese fret actor and we could uncover the latest techniques and their playbook that we

will describe in the presentation we found some evasive and rare techniques and newly discovered malware and rootkey that we will talk about it later so let's go deep and understand how they did everything and now the what is the kill chain in the attack so it's all started from externally facing server exploitation after they got in they moved to install persistence um then they did two phases of reconnaissance that was split into initial and advanced um the next phase was credential dumping after they gather credentials they move laterally between the servers and machining the environment and the last phase was data collection and exfiltration so the entry point the entry point in this operation was scanning when t

scanned the internet looking for certainly facing servers that test vulnerabilities in this case they found their famous erp server and they found multiple rc vulnerabilities that they exploited after they exploited it they look for the specific dll that was on the vmware tools folder that it was loaded by the dropper and was supposed to inject to svc host this dll this is also might be an indication of an older attack that already been there and wanted to check if the dll is already there from there they move to persistence we can see the web shells we found dozens of web shells installed in the in patient zero um and they put a jsp code in the web

shells by looking this web shells online we could find in chinese forum acoust forum and in github the existence of the web shelf from 2006 and this is also one of the ttps of wintey after we started to block them they tried to to try to gain additional persistence mechanism and they moved uh to use the winner m winner am is the windows capability um that's by by enabling that they got remote access over http and https we can see here the command line that they used to enable winram and the last way of persistence was dll side loading they perform the ll side loading to persist through legitimate windows service in this case it's a print notifier service

they siloed into it at dbg help that hen will talk about it later with we called it a deploy log from there they moved to a reconnaissance um as i said with speed reconnaissance into two phases initially coincidence and advanced reconnaissance here the first commands that they use well carry these cos and root print which gave us the lead that they use automated the scanning tools um since this is a windows server and cadit cos is a unix command after they understand they're running on windows they started to run a command such as system in for netstar net user to gain more information about patient zero the next phase of reconnaissance was using a scheduled task the deploy batch

file in this scheduled task that the content changed from face to face and for machine to machine the content contained lol bins living of the land binaries which are windows built-in executables to evade detection and we can see some of the used commands fst like the config netstat and so on

now really understand where well what is the network that they're working on and they wanted to gain a credential so they can move laterally um and we found this first technique they used undocumented executable at the time of the investigation that called mfs dll and in one of this command in the same command we observed loading of mimikaze module which is a well-known tool for credential theft the second way of credential theft was dumping the registry so they dumped the system and some security hives and using that they got more credentials on patient zero the next phase was the lateral movement using schedule tasks so this is the same schedule task with the batch file that

we saw in the reconnaissance phase and now the content change for lateral movement their goal was to find a machine that has domain admin credentials and after they got it they started to move laterally between different regions different continents and different domains within the customer environments we can see the command that they use so they use scheduled tasks with the ip address and username and password that they install of the domain admin they run it only once on each machine they call it test to blend and to sim legitimate schedule tasks then they gave the path to the batch file and run it as a system user the last phase of the attack was data

collection and exfiltration um we find that they use a renamed winrar an archive tool in this case it was a chinese winner that we can see here in from virustotal they enable it to run dll32 to evade the detection and here this is what's the our aha moment when we understand that they are staging the most sensitive data of the customer formulas blueprints manufacturing data and so on now i will learn into a fear that we'll talk about the malwa arsenal that we found in the operation thank you nim hi everyone glad to get this reception so i'm going to talk to you about the the the many different samples that we need to use in

order to infect a single machine okay we will use this uh map this diagram in order to get over the full flow of uh wendy's in infection using this technique uh they were managed to infect many different machines in the different organizations that they were into uh when we go over the different stages we will also talk about the very cool techniques that they've used in order to be more evasive uh and be more stealthy let's go over like this diagram really really really quickly so you have all the first steps that nev talks about talked about during the kill chain explanation uh and as niv said it all started with a batch execution it has the ability to run some

reconnaissance but we've seen that the first thing that's running in the infection chain is the spider loader which brought the first files to uh the machine then stashlog was used in order to stash a lot of encrypted payloads to the clfs log and we will discuss the clfs log in a bit then spark log was the first sample that really extracted information from the clfs log and deployed the persistent part private log to the infected machine private logged in deployed the user mode agent which is deploy log and then deploy log also deployed the kernel mode agent which is winkit now i know it's a lot let's take it step by step and we will start at the

beginning with the spider loader so spider loader as i said before is the first binary that got executed on every machine it got executed by a bad scripts that were dropped using that was executed using the schedule task and we've seen it delivered in two different flavors uh the first one was an exe file the second one was a dll file in both cases what they wanted to do is masquerade as an sqlite three component in the case of dxfi it was very simple very straightforward just sqli3 dot xe nothing really special but in the case of the dll they did something very uh interesting they just kept the dll added it as it is but change the function in

the ordinal number of 138 to carry on their malicious operations uh what it did is loaded uh a file a file that was given in an argument as you can see here if you can see it uh a tlb file which of course is not a normal tlb file but an encrypted file uh decrypted it and dropped the next files that it needed uh to the attack we also seen that it had some rat capabilities we based it on older spider investigations and across different machines we found it in different locations and with different names because you want to be diverse so while we were investigating uh this part we found a very cool uh

anti-hooking technique so first of all let's talk about a hook what is it a hook basically is an augmentation of behavior of any software component by intercepting a function an event or something like that and augmenting its behavior in some way well it's a nice explanation but let's like see it graphically and i did it with animations so it's way better uh so let's say that you want to hook the ldr load dll function for example all right so we have like the first assembly lines you know like setting the stack for further execution everything is fine if we want to hook it and i'll give like an inline hooking example what i can do is just

patch the a targeted function to jump to a proxy function and this proxy function will carry on the different activities that i want it to do in most cases almost every cases in if we're talking about security uh systems uh the proxy function will jump back to the targeted functions and also carry on the first instructions that were supposed to carry on in the overwritten uh the function itself we we know that a lot of edr solutions a lot of security solutions use this kind of mechanism in order to monitor api tools uh api calls so you can see where i'm going with it uh when they wanted to get over those mechanisms so they

targeted those function that are usually targeted by edr solutions and they wanted to unhook him and hook them so how did they manage to do it this is our ntdl that is loaded to memory uh what they have done is they copied ntdl.dll from disk to a different file a different file name that called like tn a bunch of random characters dot dll and load it to memory then it just compared uh the first bytes of the already loaded ntdl to the one that was copied from disk in case of a mismatch what they do what they did is just overwrite the ntdl one that was already loaded and by this way just went over the installed hook in case that it

was there so now that we talk about the spider loader we brought the tools everything is in there let's go on to the next stage which we dubbed as the multi-phased arsenal deployment so this deployment stage heavily relies on the windows clfs mechanism in order to store and extract different uh samples and different payloads in order to stay stealthy and what's not and we'll discuss like the other uh goals later so first of all what is the clfs so the windows clfs is a windows logging mechanism that provides high performance logging for various different goals uh it heavily used in the windows kernel transaction manager both for transactional registry operations and transactional file systems both both

transactional subsystems allow the ability to log different changes uh commit them and even roll back in case of an error or any reason that will be in order to use it windows employs a proprietary file format that calls blf that is not really documented and we haven't found any ready every any usable parsers online in order to decrypt this blf files and it can only be accessed by the clfs api function calls now on this the clfs log is consist of two file types the first one is the blf itself which is the log metadata and one or more container files which contains the different log data if we look at this example you have like

the a blf file of the end user.hive which attends as i said the metadata and we have two container files which contains the information itself so now that we know what the clfs is on a high level let's talk about the first component that really uses it which is stash log stash log basically just stashing old stashing different payloads to a clfs log and in this way just prepares the victim for the next phases of the infection stashlog has two different modes of execution the first one without any different arguments it just create an atom from a random generated guid and store it into the global atom table the global item table is a table in windows that allows the

ability to share different strings using different uh int identifier between different processes and even the same processes but on different times this was like without any argument with one argument it's used to really stores all the encrypted payloads uh but in order to do it uh it depends on the stored atom from before so how is it able to do it first of all the argument given to stashlog is an encrypted buffer that first of all gets decrypted using the guid from the atom then it after decrypting it it checks like the file validity because you need to know that you're doing something in a good way or not then it looks for the destination blf

file in order to store everything which is the uh default user into user dot hive blf file you know if it's not created for some reason it creates it but in both ways it clears it then it encrypt the malicious data it decrypted from before but this time using different machine characteristics and storing everything to the blf container file itself now when we first started uh investigating uh investigating stashlog this was what greeted us on ida pro and as you can imagine like this was our reaction uh yeah what can you do so how did they manage to do it well it was a very cool trick that basically what they've done is tricking these assemblers into uh parsing if

statements as switch case statements with a lot of different potential cases if we look at this example this is a very normal uh if statement on assembly you have like your check and then you have a jump with two different objects straightforward everything is working right well cool uh wendy wanted to be like smarter than this and what they did is uh return the check uh the check result to a different jump list which contained different nodes in the function this way any any disassembler that we try to work with just parse it as a switch case a switch case check with a lot of different options in this example in particular we had 332 different objects yikes

so we talked about stash log we talked about how everything was stored in there let's talk about the first extractor from the clfs log which is spark log spark log very very very simple and straightforward what it does is extracting a payload from the clfs log and set it up to be launched by a legitimate windows service this dll is private log and we will talk about private log in a bit and it's go it's it's set it up to be loaded by either the ikea xt service in windows vista to windows 7. the ikea services the service that in charge of the internet key exchange and the authentication ip protocol in windows and in windows 8

going forward it was siloed to the print notify service which is in charge of all the interactive printing operations in windows so now that we talked about it all i will give my mic to hen thank you fair okay so let's do a quick recap so what we see on the left is the kill chain uh the batch file that is being executed uh can be used for reconnaissance uh but on infected machines it's being used mostly not mostly but all the time for this infection change so it starts with a spider loader some kind of a rat then a stash log is being executed to stash all the encrypted payloads within the cfs log file

then spark log kicks in it's backlog extract private log and then we get into private log so what is this private log so private log is the persistence and privilege escalation component in the infection chain it deploys and then executes deploy log the next link so the first phase is drop in deploy log so it extracting the crypt's deploy lock from the clfs log file and copies a legitimate dbgo.dll from system32 to its newest location under the windows powershell directory then it uses a pretty unique dll loading technique to load and execute deploy log so before we go over the technique let's remind ourselves a bit about ndfs transactions so ntfs transactions are based on clfs

and they enable basically to record a set of actions on the ntfs file system such as edit remove or create files or directories and then commit them at once or roll them back at once the attacker is used in deploy log the the transactional api calls transaction api calls to move on and to deploy deploy log and again stealthiness so it first starts using the create transaction and create file transacted a api calls which create a transaction handle to the new dbgo then they over at the transacted dbgap.dll with a decrypted payload then they create a memory section and load the transacted file to this section in memory and the next step is fixing the section

permissions and resolving the dll imports and then executing deploy logs entry point this loading scheme is pretty similar to the phantom dll hollowing but it's a bit different so after a deploy log has been executed let's move on and discuss deploy log so deploy log was discovered as dbgap.dll and it has two purposes the first one uh is the rootkey deployment it deploys the rootkit and communicates with it the second hat is the user mode agent so it acts as a user mode agent it communicates with the remote c2 and contains holds some red capabilities before we dive into rootkit communication let's discuss a bit uh background on rootkit communication so basically there are a few ways to

communicate from user mode to kernel level uh from a among the ways is using device object and ioctls in windows uh you can use the device i o control ipi code as an interface for ioctl which is an input and output control to basically send control codes to the kernel level the kernel level um gets these control codes acts upon this control code and returns data if needed so the plot log initiated communication channel by acquiring a handle to the bib device if it failed it acquired a handle to the null device both of these devices are used a lot of times in operations that involve rootkits mainly the null device then it sends an

ioctl request to test if the rootkit was already deployed if it wasn't deployed then it starts deploying it so it extracts the decrypted wing kit which is the rootkit from the cfs log file it stops the amd k8 service which is the amd processor a qna driver service this is pretty interesting because um aiming for this service might tell something about the attackers and about the victims it might tell that basically they had some prior intelligence on the victims on the processors and based on this intelligence they decided what to aim for so they aimed for this mdk8 service they changed the mdk8 service execution path to the following path and wrote a winkit driver to this path exactly then

they started the service which executed the driver and restore the service to its default state to enable successful reboot the second hat of deploy log is the user mode agent so as a user mode agent it communicates with the c2 and the receive data from the c2 is intercepted by the driver we'll discuss it later and it holds some red capabilities so it it can execute some cmd commands and download and execute a payload such as the mfs.dll credential grabber that nif talked about before and this gets us to our last step the cream of the crap the king win kit so winky is there a is wendy's rootkit in this operation it was discovered as bqdsp.sys

and it's the kernel level component of the agent receives commands from deploy log and from the c2 and hooks this pip communication by directly talking with the network card so and an interesting thing about winkit is that the compilation timestamp is from 2019 but until we publish the blog post which was in 2022 it had almost a zero detection rate zero percent detection rate on virus total only one vendor alerted the win kit as malicious while others said it's it's an okay it's a it's a good file so it has also an interesting signature so it contains an expired bank u digital signature which was stolen it was expired in 2015 bank you generally is a taiwanese a

manufacturer of electro electronic devices many many displays and it leveraged this digital signature which you can see a bit information about how they try to disguise to bypass the driver's signature enforcement mechanism which has a windows a contains which is basically enforces call dsc basically enforces only a verified signa digitally signature drivers to load in windows and kicked in starting from windows vista so we talked about study characteristics let's talk about execution a bit so it looks after successfully loading it hooks the network communication to do so uh it uses the nd the windows and this api to communicate directly with the networker it it enables the threat actor to hide their network communications so during

investigation if winkit executes if you write netstat for example you wouldn't see the communication it will be hidden and it it can uh receives the following commands from deploy log so the commands are the first command that can be triggered is to hide the driver in memory uh when this command is triggered then winky deletes its headers from memory and removes the driver from the driver's linked list the second ability is to communicate with the c2 directly without going through the process of user mode knl it has also the ability to return the rootkits version to deploy log or any other user mode agent that winkit will deploy and also to clean up so let's wrap up everything

it starts with a batch execution spider loader goes next stash log right afterwards stashes all encrypted payload to the clfs log in a very stealthy way spark log executes it extracts private lock to gain persistence in privileges escalation private log extracts deploy log um in a very cool way deploy log has two hats the first set is deploying wind kit and the second ad is acting as the user mode agent and with winkit being winkit working as a root kit in the kernel level so some detection ideas because we are here for the blue teams so first of all you can use the io season the common lines the from this presentation from the blog post

you can use some vr rules some of them are publicly available to hunt for this operation or you can create them by yourself you can hunt for cmds executed from the jlounch.exe which is the executable that was used in the erp solution and also from svcos you can look for api calls from unusual processes to the blf file and its container files and lastly we can use our old friend volatility using the unloaded modules command you can see if the driver was unloaded in the past uh using this name uh disguising as a benq related driver so the key cater key taker was to wrap up the operation so first of all we managed to discover a worldwide truly global and

impactful um apt-ip theft in alignment with china's notorious made in china 2025 plan and this operation emphasizes the importance of network hygiene so the affected effective victims haven't had any good network hygiene so that's what managed them to literally move to dump credentials to move between continents so they had and patch systems insufficient network segmentation no multi-factor authentication and so on we managed to get also a really a rare glimpse into winter's hacking playbook to understand how they infiltrate how they act how they move what they think and how how they react and we managed to get a deep dive into winter's arsenal so we discovered new malware and new versions of non-malwares and we got a big sneak peek into their

enhanced stealth house of card approach which is pretty interesting i think in clfs so we would like to also acknowledge our teammates that did an awesome work with us on this research and that's it you can follow us on twitter and further in our blog post thank you [Music] [Applause] you