Ebury: Public-private partnership unveils the full scale of a sophisticated Linux threat

[Music] Uh, thanks for coming and um, I think it's one of my favorite uh, researchers uh, that we that we at ESET especially because of the partnership we had and we're going to see that throughout the presentation with law enforcement otherwise we're just showing problems we're not taking real actions uh to make things better at the end of the day and uh just before I uh I start I would like to do some uh quick housekeeping uh first even though I'm the guy here presenting to to you all that's by far not a single person type of task it's actually research that extends for now more than 11 years. So as you can imagine there was uh a lot of

people involved on it. Um and then now the team is actually led uh by Jian Bhen on our uh research office in Montreal and uh but again throughout those years we have many people uh participating on that on that research. Um, another one is um I I especially because we are in besides even though I would like to really go down the rabbit hole and talk about uh the little nitty-gritty of the techniques they they were using uh again because we're talking about 10 plus years of of u um this ibotn net running uh it would be like a lot of a lot of zeros and ones to to talk about. So I'm try to do like a technical overview if

you will but the good part is that we have a white paper that is actually fairly detailed and then you're going to find all the technical skills that we love uh on that white paper. At the end of the presentation you're going to have my contact information. I did that on site besides Calgary uh and other besides I'll just send you the information more than happy to share that. That's the that's why we do this to share uh information. Uh on the same note, no need for photos. You can take it, but no needs because I can share every single detail you're going to see here with more details on that uh white paper. And last u but not the least Q&A

at the end as well. Uh I'll try my best to speedrun that because I love Q&A. So we can ask a lot of questions. And uh okay, I think we can get going. By the way, about acknowledgements on that white paper, you're going to have everybody who participated on that research throughout those years. And see, it's a lot of people. It's not a Lord of the Rings credit, but it's it's getting there. And um one thing that would be very important uh to understand how we were able to run that uh research throughout those years with the amount of visibility that we had. Uh it's related to the company itself. Anybody here heard of ESAT before? Just sanity

checking 20 30%. Which is fair. We don't do a lot of marketing stuff. Um so just to understand again how we were able to track that uh threat specifically uh is mostly known in Europe where if we combine the partnerships we have I'm going to fly by through this just to give you the backdrop. Uh if we combine the partnerships we have with Intel and Google we collect telemetry from more than 1 billion devices across the globe. you name all sorts of devices and again it's all related to the telemetry and the knowledge we use to extract meaningful information out of it uh from those billion you know direct customers B2B customers and then we start getting

on the more interesting part because those are usually the the entities using um I'm trying to be confined here because of the camera they ask me but it's it's giving me the crips a little bit okay uh so this that's the amount of B2B customers and again those guys who have the juicy part for those uh criminal actors and again enterprises they also are heavy users of Linux boxes uh internet facing uh so all that telemetry go to our research centers across the globe of course as I mentioned one in Montreal uh and we again not only with the volume but different types of telemetry we start putting those pieces together to do researchers like the one I'm I'm

presenting now. Uh like I mentioned everything started more than 11 years ago with uh at that time called operation wido where we actually brought the first details of that um first scene large scale botn net uh for Linux boxes and then we published um a lot of things a lot of details on that operation. window. We were hoping that disclosure would actually make it really hard for them to keep uh that botnet alive, but it was not necessarily what uh what happened. And just to also level set a little bit, even though it's going to be more like a technical overview, uh is everybody here familiar with SSH? No. Okay, fair enough. Yeah. So, I'm

going to fly through this as well. basically you'll log in your uh remote Linux boxes. We don't use graphic interface. We don't need that. Uh so that's the main way we interact with those those boxes. Um but then comes Ibery which was um again 11 years ago uh one of the first massive scale uh botn nets we were able to track and they were um just exploiting something on the open SSH um application which is the most common way uh or the common most common application Linux boxes come uh out of the box with that application on it. Uh so they were able to find a back door on the open SSH application that was

basically meant to whenever the real admin of that box logged with their credentials credentials immediately got excfiltrated to the ibery network. Uh so they were able to leverage that to after uh make use of those boxes. uh which was uh really interesting. They started uh leveraging and they were really good on like uh using if I can put this way industry standards. They didn't try to reinvent some of the wheels. They were just really good on uh uh this exploit with the uh on the open SSH application but they didn't they didn't try to invent every piece of the equation. they were just leveraging best practices from other um you name a threat actor criminals and so on. So they actually

started using uh userland rootkit to not only hide files on the operating system which of course is a obiscation technique kind of common and nowadays uh I hear a little bit feedback is it something I can do to get it better or is it okay? Okay. Um but they were also able to run the resource they were utilizing to run their malware application uh on on those systems. So that made really stealthy and really complicated to to companies uh to to detect that and they were just not interested in locking down those systems. they were trying to get as many systems compromised as possible and use those systems to do whatever activities they they were

planning to do. So then as I mentioned we released operation window about 10 years ago. We published a lot of details and whenever any uh cyber security uh research or company they publish those things in in theory you're going to get of course the bad guys also aware of that. Uh and the the idea is that everybody will be able to protect themselves against those known uh ways they try to exploit systems but um that's the like desired scenario. uh but not only uh it's not always the the reality that uh we see. So at that point during operation wido uh the uh was basically leveraging three monetization techniques for their operations. One was um web traffic redirection. Again, not

trying to reinvent the wheel. They leverage a known uh uh malware called a family of malware called Seedor. And uh the other monetization technique was spam campaigns. And that's uh if you guys again I can dive a little bit on it if you're not familiar. Um but it's basically a good way to avoid uh spend filters. So they were leveraging the good reputation. and IP addresses from those boxes and from there uh doing spam campaigns likely fishing campaigns uh using those those boxes. And the third way was utilizing that as like a massive distributed data center like a lot of infrastructure available to do whatever you could do with a normal infrastructure DNS servers, web servers,

so on and so forth. Uh so again we release that to the public in theory is going to help disrupt their operation or in in some cases they are well organized or organized enough to actually keep like as a normal company releasing new versions of their malware and that was what happened. They released version 141 and uh crazy enough uh they mention ESET on their new version because of the hard work we gave them after we released the way they were exploiting those those boxes and mixed feelings with you know proud and holy cow. Um but it again they released that new version and we keep kept track of them. So if we um and then

we started um how can I put this there was one particular and uh so those big operations it's sometimes I don't want to say common but it happens more often than not uh we we were able to pinpoint to an individual that was leveraging uh the botnet and we kept track after he kind of made this mistake we kept track of that individual and um we were in communication with law enforcement in in Europe and when he stepped out of Russia like crossing the border to Finland, it was like super fast for the law enforcement to arrest that person. That was December 2015 and then um the person was actually wanted in the United States. So in the

beginning of 2016 uh they sented to the send that person to the United States to face trial. Uh at that point uh he was pleading not guilty not guilty over and over. So we had a and that's the the the not fun part for technical guys. We had to explain to like uh prosecutor judge how we could track all the technicalities of that botnet and how we were able to pinpoint that to that particular individual that was on trial and it's always if you try to explain you know what you do for a living for your family it's that kind of conversation right so it's it's always interesting uh I I I don't judge I'm not

sure what what happened But the guy got into the prison in Minneapolis if I'm not mistaken. And at some point he decided to plead guilty after one year being in prison. So I'm not sure what happened there, but I I don't don't judge. So I'm I'm okay, whatever happened. He decided to stay in Minneapolis basically. So he got uh the the forced green card and uh he's living in the US now. Um, so that happened, but we had like a fair amount of reasons to think he was now like the head of the Hebrew operation. So we kept track we kept track of the botn net. We were seeing if there was any new activities.

And coincidentally or not, in 2017, one of our honeypotss actually and so the honeypotss uh we were basically excfiltrating the credentials to our honeypotss from our honeypots, sorry. Uh in the same ways I used to buy their credentials from uh info steelers or other uh type of threat actors. So we were kind of excilating that way in hopes that they would use that to log into our uh honeypotss. But in this time particularly they found it was actually one of our honeypotss. So they logged out and uh we were not hearing from them for a few years. Uh fast forward that to 2021 and then another law enforcement u uh reached out out of the blue to us uh

saying that they found a victim of cryptoc currency path uh related to iby. And then if you remember the three three main uh techniques they were using to monetize the the the botnet, it was web traffic redirection, spam campaigns and um infrastructure um u hosting kind of thing for for other actors but never we heard they were using cryptocurrency. So we were like, "Oh yes, let's let's collaborate and and you know partner here to see what we can uh come with that information." I'm going to park here for a second. And that's the really like pinpoint of that uh pinnacle of those 10 years of research when they they reached out to us because they were able to do

something no private company would be able to do. They seized assets. So they actually gave us like servers, physical servers they were able to seize uh on on their side of the the operation and that's the important part like no private company will have all the pieces of the puzzle alone. No public entity will have all the pieces of the puzzle alone. If we don't like combine those knowledges, there's no way we can build the big the big picture or the most complete possible picture is the the best way to put it. Uh so it was insanely valuable when they shared that information with us. The flip side, if you can imagine like a server running from an operation

like 10 years old, they had a massive amount of data. So it was a lot of work to make sense of that information. They have like full list of their uh targets, phone numbers, email addresses, how many times they reach out to do double triple extortion, you name it. So it was like a very uh organized u uh entity behind that. Uh so it was you know a lot of work to make sense of all not everything was like just technical information like hashes or IP addresses and so on. So you had to really understand uh the information we had at hand. So it took us two and a halfish years and then last year we published um the last big

iteration of the ibery uh research where actually before that uh server seized shared with us we were we were thinking we were talking about um throughout those years approximately 40,000 uh Linux compromised across those years but the number was with that server was actually like 10 times larger. But again, not everybody will have uh a single picture with all the information in it. And we thought the Berea was running uh for 11 years at that point. But again after we started collaborating with the Dutch uh cyber crime unit uh we saw like first compromises uh dating back to 15 years actually. So it was way longer than that. Uh one thing called our attention especially analyzing that data. We were

tracking that because it was so large by the month not for like years because it would be massive. But we not we had to create another scale to to actually track some months that had really big spikes. And uh and then we went back to the you know to the to the logs to the data and we realized they were actually now utilizing not only credential simple quote unquote simple uh vulnerability on the open SSH application to steal credentials. they actually had now uh four different methodologies to take control of Linux boxes and again on that white paper there's detailed information of every uh single methodology they were they are using uh currently and they

also changed uh the more oneonone basis type of attacks and that was the reason we saw those spikes. uh we find out they were basically compromising data centers, hosting providers. Um so what we did was we wanted to test that theory. We rented a box u like a raw Linux box just had open SSH on it. Uh we let it run on a uh we thought was one of the compromised data centers. Um, and we let it there sitting there to see how long it would take to get ibery on it or if you want to get ibery on it. So, it took only seven days um to to have the machine compromised. And of

course, it's a it's always that mouse and cat game. We were able to up our uh honeypot game. So we actually made a really good honeypot because we could track uh ib connections across their distributed botn net uh every single day. So it was a um again a way we kept uh the eye on those on those um uh connections not locally anymore like we used to do in the past because they were also prolific in recognizing our our botn nets. Uh so we were kind of a attack in the middle uh attacker in the middle technique but for botn net purposes sorry honeypot purposes. Uh at that point they changed like I mentioned their monetization uh techniques. They

were now stealing credit card information and not trying to reinvent the wheel as well. They were just doing their uh first foothold on the systems and leveraging existing malware. uh three families of mau uh sorry two families of mau in this case elim steel and freezes steel to uh get the credit card information out of those compromised boxes uh traffic redirection and spam was still a thing they're still leveraging that and like you hopefully you remember cryptocurrency that we got first to know with the Dutch uh cyber crime unit uh and that was very interesting uh from a cry especially cryptocurrency standpoint um they were so big from a like a distribution standpoint. They actually

we saw in the docu internal documentation they had on on that server. Uh they actually came up with like a crazy assumption. If you guys are uh familiar with cryptocurrency, they have to have like public facing nodes to be able to do the transactions. Uh and there you have to publish those those nodes. So they came up with that assumption that I'm I'm I'm behaving well here. You see uh the the the assumption that they would likely have uh compromised machines on the same subnet of those public nodes. So they started testing that and they actually found quite a lot of nodes that were actually on the same subnet as they were on those either data

centers as other completely compromised or partially compromised data centers. And uh what they did was and that that's probably the part that kind of boggles me. uh if you guys heard of or know it uh they were doing simply arp spoofing to and that's like a older than me technique kind of thing and worked and then um now you're connecting to your right yeah and and uh and the crazy part is that there's just a vast amount of documentation how to prevent herbs spoofing right so it's almost negligence. You don't have that uh enabled on any almost old switches nowadays or whatever. You can you can do that. Uh either way, so they they're basically

you think you're logging on your crypto wallet, you're actually being because of their spoofing now logging into the compromised machine. and they had like a simple from a attack standpoint technique. So you log there, your wallet gets exfiltrated right away. So uh and that was the case with that investigation from the Dutch u uh cyber crime unit. So and that we also found uh they were using attacker in the middle technique and then not for honey pot purposes but to collect a large amount of uh uh SSA ho host keys and they it was interesting because they got a little lazyish they started replacing the host keys by a unique host key to make their life

easier. But if you know like when you first log into a into a server the keys don't know each other you're going to have a prompt or if somebody changes the host key and then you're going to have this prompt as well. If you are the admin of that box and you know you logged there before and you get the alert this alert what do you do? That's what people do. Hit yes, move on because your ser or your service is down or you have to patch something. People just move on. Nobody the vast majority I would say just hit yes and do whatever they want to do and life goes on, right? Uh so that became like a

really big part uh of the iburi methodology to get uh their operations running and uh but on the on the on the flip side because they're using like for the most part one single host key now we can use our beloved showdown for example to track those host keys. So we started doing that to find ib compromised machines even if we didn't have that on our telemetry. So we could actually go for a hunt of those of those guys. Um, currently we see primarily Iber using still spam apparently it's like fairly lucrative or a lot of profits coming from it. Uh, web traffic redirection. Uh, cryptocurrency was probably a more pointed type of operation. They are not.

so big on it and or at least not focused on it from our perspect as as far as we know. Uh but one of the most damaging things they're doing nowadays is excilation of post requests but everybody familiarish with post requests web forms you name it like insurance you go there I want to quote and you type your life on it and they get excfiltrated too. So they start also monetizing that selling to our uh uh what we what we know as like info steelers. Those guys handling all the or most of the compromised information on the dark web. And um another interesting thing we noticed recently with Ibery because of their very large presence

they started bumping uh bumping into other actors malware and they didn't want that. They want to have like full control of those boxes, not anybody else. And we started seeing on those uh on the on the scripts they were running, on the pieces of m we were able to collect that they were looking for their main quote unquote competitors, in this case big bad wolf. And uh they actually had scripts like to really find uh on those boxes they were operating if bad uh big bad wolf was also u trying to get a back door on those on those uh boxes. Uh this is just you know simple demonstration of that one I was mentioning to you of the post request

xfiltration. So you try to connect to whatever legitimate website actually you get redirected to uh uh the the Iberie server. Your information will go through. You're going to process your payment. You're going to receive the quote from your insurance company so you don't suspect something's wrong. But all your information got uh excfiltrated to those guys. Uh kind of coming to to an end. So, we're going to have some time to uh Q&A. Uh because of the whole operation, including the information we got uh from the partnersh partnership with the Dutch cyber crime unit, we published some detection tools. They're going to be on that white paper as well and uh some remediations for it. Believe it or not, uh, one of the main,

um, things we could do to prevent Ebery are also open source, zero cost. uh some of the conclusions we had across you know analyzing their activities for so many years is that uh and I do recommend there's absolutely and that's sometimes it's a it's a almost like a whole block for victims there's absolutely no shame if you got a Linux box compromised with ibery even super techsavvy victims got compromised uh with it it's just the nature of our industry. You cannot see everything, patch everything and it just happened. Best way to prevent that MFA uh have anybody here set up MFA on CLI? It's it's beautiful, right? It's like imagine the CLI spitting out like a QR code. It's

horrible, but it works. It works. Uh, and again, open source. Google has one. It's on the um white paper as well. And and then you name it if you have the ability to do both. Even better, uh, have EER on your Linux boxes. Doesn't matter which, but pay attention on things running there. Behavior of the box. Again, they started super stealthy, but some of the other things they were running on those boxes could trigger EDR detections even if your EDR doesn't catch the the offiscation techniques IUI itself was using. But then when they install other malware ele so on and so forth then you can catch those and then investigations will kick in hopefully you're going to see

somebody will do like a root cause analysis and they will catch the whole thing happening on that box monitor your Linux boxes and this is my contact information the QR code doesn't have IUI on it just my LinkedIn uh profile if If you want to send me either a message on LinkedIn or my email is over there as well, I'll send you the the white paper. And with that, thank you. We have we Thank you.