CrimeOps of the KashmirBlack Botnet

thanks thank you john uh we're super excited to be here presenting in b sites so we're about botnets all the time and in this session we'll give you a deep dive into one of them the kashmir black botnet and i will start by talking about botnets in general and describe the kashmir black and its entities dino phil will present the operation and talk about the devops behind it i'm sorry but first let me introduce myself and my name is sarit and i'm a security researcher at imperva for the last 10 years i mainly focus on my application security and i develop algorithms to detect and protect against attacks my colleague of fear is a security

researcher at imperva for the last five years his focus is in database and web application security before we start to deep dive into cashmere black i would like to talk about botnets in general so there are many kinds of botnets buttons that infect iot devices windows machines linux servers some target the endpoint and others target the servers now when talking about botnet purposes there are cryptojacking botnets that infect machines in order to mine for cryptocurrencies other botnets manage phishing campaigns and are used to distribute malware via phishing emails there are botnets that used to perform ddos attacks by sending an overwhelming number of requests to a targeted server or an application in addition there are scraping botnets

that are being used to extract data from websites and even a contact over botnets that are being used to validate combinations of credentials resulting in identity stats and other botnets are offered for rent and may use for any purpose before i dive into the bits and bytes of our research i would like to introduce you with the cashmere blackboardnet so it all started on november 2019 and last 11 months which is basically our research period of time we discovered a botnet that attacks popular cms platforms such as wordpress joomla magento in more than 60 different countries around the world it performs mills of attacks per day on average and we calculated that there were hundreds of thousands of bots out there

participated in the botnet operation those bots utilize dozens of non-vulnerabilities with different attack types like file upload remote code execution and many more this session is a journey into the botnet core from the attacker point of view so let's start security research investigation can sometimes be like a crime scene investigation but our crime scene is spread over the network with nobody in place so we need to collect the clues and fingerprints to construct a picture of the virtual crime as part of a study carried out at imperva we observed around 9 million attack attempts exploiting php unit remote code execution and we were wondering why is this cv so popular among attackers and to understand this hive we started

to analyze attacks from our data lake we saw different ips using the same payload over and over again attacking different customers which remind us abutment behavior so we decided to download the payload and dive in and we basically started the mapping step we downloaded the code and performed analysis we revealed all the entities of the operation and later on we'll talk about them in further detail the next step we took was infiltrate we saw that the budget is updating on a regular basis and we decided to act like a bot and gather these objects for later analysis and finally we played a victim we created a honeypot in order to understand this plus exploitation stage

so let's review all those entities that play a role in this massive operation when looking at a budget entities we can split them into three groups the buttons infrastructure the botnet third party services and the botnet actors inside the buttons infrastructure we have the cnc and repositories amd under the third party services we have github baseband and drawbox that our attacker was using in one hand to camouflage the operation and on the other hand to make the button more flexible and under the botnet actors we have the victim and two types of bots pending and spreading and i will describe the difference between these later on the first entity in the bonded infrastructure which is responsible for the entire

operation is the command control and here we can see the login screen of the cnc the kashmir black cnc is located in indonesia and has three main rules it supplies attack instructions to bot it receives attack reports from bots and it supplies a malicious script that infects the victim server here is a snapshot of the infection script we consider the attacker define a parameter that represents a ground up task this task contains a python script and scheduled to run every three minutes it includes several imports and it uses basic c4 encoding to obfuscate his malicious payload the output of this task will be sent to dev now so no history will be saved in the next code block we consider the

attacker redefines the victim cron task to include this malicious python task and as part of this redefinition the attacker makes sure to remove all main notifications and there is one interesting thing to note here is that the attacker was using the combination of pearl and python which are both installed out of the box in many linux systems and basically this increases the probability of a successful infection let's move to the repositories the original repository a as you can see is a printer component shopping site it was hacked by the attacker and was used to store the communication script file to communicate with the cnc another type of a positive entity in the botnet is repository b

which is um it was classified as an educational institute and was used by the attacker to stop bundles of export and payloads and here is an example of the export payload bundles the attacker files are located under the css path among other css files used by this innocent web server the name of the bundle files start with the in-memory prefix and there are actually zip files hidden with the css extension and here is their modification date and one of the best qualities of this botnet is that the infrastructure is just like plug and play the attacker can expand his target victims and add new exploit by just uploading them here under the css directory and no

infrastructure changes are required and i must add that every file here represents an exploit the target a specific vulnerability here is a partial list of cves the button it uses as part of its operation among them we can see remote code execution file upload remote file include and and many more these vulnerabilities are related to different plugins widgets and themes and some are even a decade-old from 2011 and the conclusion here is that it's not necessary to use an exotic exploit in order to expand a button moving to the cloud-based services another type of entity used by the attacker is github it was used as a version control to store some of his files and when we checked the repositories we

saw php web shells and cryptominers and we can say that by using github the button achieves a layer of flexibility as the attacker can easily update and find this repository without interfering with the buttons activity another entity is spacebeam which is a website that allows anonymous users to share plain text through public posts they are called paste and the attacker use this space as a quick and easy way to access and download vectors through the infection step in the buttons operation and later we'll show how dropbox was used in order to upgrade the botnet hide the operation behind legit cloud services and also to secure the cnc now let's talk about the two types of

bots first the spreading bot this bot constantly communicates with the cnc to receive attacking instructions there those are comments from the cnc telling him who to attack and how this bot is being used to infect new machines and to expand the botnet and a victim that was infected by spreading bot can become one of two a spreading bot or a pending bot now let's talk about the pending bot and as i said before this bot is a victim side that was infected by spreading bot the one that appears here and as a result is under the control of the cnc and it stays in idle mode until the cnc approach and change its purpose and actually this

is why we named it pending bot and i will talk about the purpose in a bit but i want to add that the difference between them is that the pending bot does not initiate communication with the cnc let's move to the cashmere black botnet scope the infiltrate step the best way to learn an organization is to be part of it same for learning about the botnet operation and we call it the infiltrate step so once we mapped all the entities of the botnet we wanted to understand the scope of the botnet its victims the attack and its evolution and to answer those questions we had to take a more active approach to the investigation we learned the communication protocol

between the bot and the scene scene and we mimicked it we infiltrated the botnet by constant communication with the cnc we went undercover and impersonated the spreading but in the botnet and without actually attacking any targets we started to collect information about the botnet victims and here is an example of a request from spreading bot to the cnc asking to get the attack instructions and we can say that there is a special header that without it the cnc will not retain anything this special user agent archer ghost 8 is some kind of a security mechanism to prevent unauthorized access and although this is not a sophisticated authentication mechanism it is a basic security control for the

cnc now as a result to the request we just saw the pending bot will get attack instructions from the in json format from the cnc and the first parameter the script contains the commands that will be executed by the spreading bot first it will run the curl command to download the export payload bundle that will be used to infect the victim and here we can see the file to download and it's it we can see that it's located under the css directory in repository the one that just show i just showed you and the second parameter the payload contained contains a list of victims site that will be attacked by the spreading bot the last parameter is the host name or

ip that holds all those victim sites now as i said before we impersonated a spreading bot in the botnet by sending a request every three minutes to fetch attack instructions from the cnc we gathered all these instructions for further analysis and we use shorthand api to extract the organization the country they ascend vulnerabilities and open ports and they install components of each target then we inserted everything into a database for a detailed analysis so let's see what we learned one of the things we wanted to learn was about the location of the targets to see if there is a specific country that that is botnet targets and here is the distribution of the attack targets divided by

country and it's a bit hard to see but we found 60 different countries these botnet targets it appears that the majority of the targets are located in the us this is the blue part but since a lot of servers are actually hosted by cloud providers the original country of the server does not necessarily reflect on their origin of the victim another interesting question we're curious about is how the attacker find his targets so we believe that the attacker has some sort of a scanner that scans for potential vulnerable targets and this scanner probably uses sheldon or similar service like binary edge to locate for potential targets by searching for specific vulnerability or open ports another method this attacker may use is

running cms vulnerability scanners like cms map or wp scan that helps with finding potential and vulnerable targets so once a target is identified it is being initiated inside a queue in the cnc for a future attack and basically when a pending bot will fetch attack instruction from the cnc the target will will be taken from this queue moving to the botnet purposes so in order to understand the purpose of those victims spending bots we had to become a victim ourself so we created a cms honeypot and attacked it with our spreading bot from the infiltration step we reported back to the cnc of a successful attack and by that our honeypot became a pending bot in the cashmere blackboard

waiting for the cnc to approach so after we reported to the cnc of a successful attack it took the attacker one and a half hours to connect our honeypot which is kind of oppressive as it's very quickly and we had a sort of uh log inside the honeypot showing us which commander attacker did and which files he added or modified and we saw that he added a second web shows a second web shell with command execution vote capability then he ran several commands to escalate his privileges by using sim link and by that our attacker got complete control on the infected server now let's continue to the purposes of the botnet we saw five purposes about the first two

we already discussed those are the pending bot and the spreading bot so we'll talk about the others an exciting purpose we observed is a cryptominer that mines monero coins and as part of the colonizers we did we got access to the hacker's payment address and we can see it's balanced in real time the next purpose was discovered as a result of our cms honeypot it was converted into a clickbait bot when we tried to access the honeypot's login page we were redirected to one of many clickbank sites the last purpose is defacement once we saw the defacement signature we discovered the nickname of the hacker behind the botnet we also discovered it is part of the

indonesian hacker crew phantom ghost by searching the internet we found out more interesting information about the crew like the facebook page and even an online shop that sells the phantom ghost crew t-shirts now after we're familiar with all the entities we will continue and show the entire operation in life thank you sorry and hi everyone so how this botnet works it all starts when a bot exploits php unit remote code execution on a victim server it causes the victim server to download an infection script from the cnc then it will execute it now the infected server will approach repository a every three minutes to download the fresh communication script at this stage we can say that the victim

server is part of the kashmir black apartment the newly infected bot communicates with the cnc to get attack instructions describing who to attack and which bundle to use the bot downloads the bundle from repository being and additional payloads from github and baseball then the bots attack the victim and on successful attack it will become part of the botnet as a last step in the process the bot report back to the sensing now that we are familiar with operation we can move on and describe the stages of the botnet since we discovered it and throughout the research period when we first met kashmir black it had around 10 exploits and only two payloads spending and spreading

it focused only on growth increasing the size of the botnet then infrastructure changes started to emerge to make the botnet more stable and scalable compared to the first two stages the expansion stage is an ongoing process we saw more exploits and more payloads are being added during the entire research period now let's dive into each one of the stages starting with the growth cashmere black has an exponential growth i'm going to show you how we came to this conclusion so we observed 285 bots in our data lake attacking our customers but note that this is only a portion of the bots in the botnets since we see only traffic of our customers so for this example i'm going to round

the numbers a little bit for simplicity and i will use 300 watts so we know that every bot performing attack every three minutes per day it will attack 480 targets the 300 bots together are performing 140 thousands of attacks per day now let's say that only half percent of all the targets are successfully infected it means that tomorrow we'll have 1 000 new bots in addition to our current 300 bucks by day number seven we will have almost half million bots the following chart illustrates the exponential growth now let's move on from the growth to the stability stage in this stage we will describe the evolution of the botnet over the research period and the devops

strategy that enable it to carry out describes remember that the botnet had only one repository a and b once the botnet size increased so did a load on the repositories in addition since those repositories were actually legitimate sites they couldn't be considered as permanent and reliable entities the attacker had to take action three changes were implemented in the botnet infrastructure and the first is adding new entity repository a load balancer expand repository a into multiple repositories and expand repository b there were three main reasons behind these changes make the botnet more dynamic and scalable add redundancy and load balancing the following diagram shows the old infrastructure against the new one while in the old infrastructure every bot will

address directly repository a in the new one each bot will address the load balancer and will get in return one of many repositories to integrate this change into the botnet operation an additional change in the botnet was required we will discuss this change later on when we will explore the upgrade process now let's talk about internal changes that were made in order to secure the cnc and the botnet operation the cnc is the most sensitive and important component in the entire operation securing it is critical let me take you back a little bit to the steps where we infiltrated a botnet and played the victim we created the honeypot attacked it with our spreading bot

and reported back to the cnc we believe that the attacker grew suspicious as he performed two internal changes in order to avoid interfering with the batman reporting everest was changed and both ip tracking mechanism was added the first change is related to the reporting address this change helps with managing bots and versions bots that report to the new address is a new bot second change is within the botnet's communication script it was updated with a bar tracking mechanism a simple architectural change adds the bot ip and country while it communicated with the cnc it allowed the cnc to track and monitor the operation of each bot in the botnet there are two goals behind this

mechanism the first is to secure the botnet and the second is to manage bots versions and upgrades now let's see how it comes to work the changes that we described created a situation where some bots were using the new infrastructure while others were only aware of the old one this diagram described the upgrade process on the left side you can see the old infrastructure when an old bot communicates with the cnc without the ip tracking header the cnc in return sends back attack instructions that inspect the bot to download the upgrade script from repository b once the bot executes the upgrade script it will turn into a new bot that is now aware of the

new infrastructure on the on the right side the upgraded bot will address the load balancer to get to get one of many repositories the script is actually changing the contact job that sarit mentioned earlier now let's talk about migrating the cnc to a cloud-based service there are fundamental problems in the botnet architecture since bots communicating directly with the cnc and the repositories their ip is exposed and security controls may block them an interesting infrastructure change has evolved to solve this problem integrating drawbacks into the operation instead of communicating directly with the infrastructure entities the cnc and the repositories the bots are now communicating only with dropbox now dropbox api is being used to fetch attack instructions

and to upload reports from bots this is a big step towards cam flashing the button traffic securing the cnc operation and most importantly making it hard make it difficult to trace back to the hacker behind the operation when we discovered this change we were very excited since we had the authorization key of the dropbox account of the attacker we thought we thought to ourselves it will be fun to connect and see what's going on we started to fetch all the files from the account and we saw info the structure of the account sorry so the directory is adelia p we believe that the name adelia is significant for the attacker because we saw it in several places during our

analysis it used in passwords and inside urls inside the cnc next we have the payload directory that was used to store attack instructions we found 400 thousands of attack instructions in place next we have the loot directory and it only have one subdirectory called nosql nosql is one of the bundles that added by the attacker to repository bin meant to attack nosql databases with several types of nosql injection inside we found only one report putting it all together the loot directory supposed to store the successful attack reports uploaded by bots splitted into subdirectories for each exploit bundle the assumption is that the attacker was in the middle of transition maybe in the testing phase this is

another piece in the puzzle that helps us to sing to the attacker's world and understand the operation a glimpse into a development cycle in a making by that we close the stabili the stage of stability and now we will focus on the extension stage as i said before the expansion stage is an ongoing process the attacker adds new exploits on a regular basis over the research period the exploits were extended from 10 to 17 targeting new domains expanding the botnet targets from cms that are based on apache to new domains such as web servers that are running over iis and even exploring no sql databases as part of our analysis we fetched the exploits from the repositories and we tried to

map them to the affected platform we saw total of 17 bundles while most of the vulnerabilities are in generic plugins and or component which are compatible with multiple cms platforms sorry or php webshop web web frameworks only one exploits was related to iis we mapped 13 different cms platforms and three php web frameworks that may be infected by the customer black partner the percentage shows the distribution of the vulnerabilities for each platform by that we close the stages overview and now let's discuss some key takeaways botnet deployment is similar to application development process there are some important key features we need to consider in order to create a stable botnet that is here to stay

those are stability flexibility and cicd so in order to create a stable.net we need to take into consideration load balancing and redundancy enabling scalability while growing in other words stability is the foundation that enables the partner to exist but this is not enough the separation of the exploits from the infrastructure enables maximum flexibility as the attacker can add new exploits anytime together those two key features are the basis of the ability to grow and expand on the other hand we have the cicd branch that includes version control and deployment cycles we call it automation behind every massive operation we must have an automatic process to support it expansion and growth cannot exist without a solid cicd

process now let's talk about the insider point of view as a security company we have that of hundreds of thousands of customers where we could see a tax in the wild this is not good enough since our data is biased by our customers here are a couple of advantages we got from the insider point of view being inside the botnet operation gave us the advantage in analysis as we could see the big picture and not just a small portion of the infection we watched the botnet evolution from the first row we saw new repositories exploits and payloads are added in real time and by analyzing the code changes we concluded what motivated the attacker to perform

such changes we had a unique foothold that enabled us to analyze the victims from the attack instructions extracting the country the platforms domains etc the inside intelligence led us to the educated assumption that there is kind of an automated mechanism that searches for potential vulnerable targets and initiate them inside the queue in the cnc analyzing the exploit distribution explains which which exploits are in use the distribution of usage what is the frequency that they are being used and which are more common than others all of this information is accessible only from the insider point of view and it is critical in order to understand the scope of the operation the motivation and the challenges of the

attacker now let's sum up everything that we talked about in a botnet development the attacker wearing multiple heads the attacker is the developer the architect and the devops usage of third-party services are critical part of the infrastructure in terms of how flashing the botnet and bypass security controls and it is not necessary to use exotic exploits in order to expand in terms of research first we need to map all the entities we need to learn the communication protocol and last visibility is essential to understand the big picture so what can we do to prevent infection first make sure to deploy waff in front of your application server invest in endpoint security install antivirus software both on your

endpoints and servers in reference to cashmere black the attacker uploaded web shells that could be blocked by waf or antivirus make sure that your server is behind the firewall and allow only authorized services to connect for example open ssh only for specific hosts to reduce attack surface if you are using cloud provider make sure that your service or host is not publicly open make sure that you are up to date with the latest security patches and that there are no unused or unsupported plugins and features installed that might increase your attack surface and anyway make sure that you use strong authentication mechanism you are probably wondering what is the current state of the kashmir black

partner so when we decided that our research has come to an end we collected ip's host names hosting services and every possible piece of information from bots repositories cnc and entities we notify the owners of the infected servers and hosting services about the malicious activity and today the kashmir black botnet is dead at least as we know it we checked our data lake and we couldn't find any traces of new infections so thank you very much for listening to our talk about cashmereblack.net feel free to contact us if you have any questions and for additional information you can read the two blocks that we wrote just search for kashmir black botnet in imperva's site thank you

so is there any questions yeah let me see um yeah thank you very much uh mr uh shayti and uh miss yatoshami for your presentation uh this is nikhil one of the volunteers so now we do have actually a question uh for you guys uh in the q and a session here um why do you think the united states was a larger target compared to the other countries let me transfer it yeah okay um as we started we said in the slide that actually we believe that most of the servers are um were hosted by cloud providers and let's say that uh when you when you create a uh server in um over aws or uh gcp or whatever

uh most of you just go to the default which is um uh yeah yes or something like that so we think this is the reason why most of them look like uh targeted like uh on the us but it's not reflected on the real allocation of the victim of the of the origin of it it just looks like the us is the most targeted but it's not it's not the the real um like like for us we are located in israel and probably if i will host a server on aws it will be located on on us so awesome uh so the uh sorry about that uh what happens to the next question is what happens to bots

that fail to communicate with new infrastructure do they typically retry later or just stay broken so basically the two infrastructures are living together side by side until all the bots are immigrating to the new infrastructure it doesn't mean that um the the all the infrastructure is having some issues with the stability and scalability because there are a lot of bots but once most of the bots are migrating to the new infrastructure and even if some of them didn't uh successfully uh upgraded uh they keep work they keep communicate with the cnc and fetch attack instructions and attack victims awesome yeah and that's definitely great information and thank you so much for that uh clarity and once again thank

you both for the great presentation and feel free to hang around at the b-sides conference this year we do have a breakout room if you would like to answer any further questions or if anyone has any questions feel free to join us in the breakout room but other than that once again thank you and and welcome to b-sides and enjoy the rest of the conference thank you very much great to be here [Music]

do

[Music]

[Music]

[Music] you