Looking Back, Containing Chaos: Lessons from Real-World Intrusions

Well, let's start. He is Abdel. He is going to talk a little about attacks on SPAY, ATMs, attacks of this type on financial sectors in real time. What happens when The institutions where we have our money are attacked. Well, let's talk about that. And give him the welcome, give him a round of applause. Ready. Sorry? Ready. Breathe a little bit. Yes, it's strong. Well, good afternoon everyone.

Let me see this, it's pulling me. Can you help me change it? - You have to adjust it here. - Here? Okay. Sorry, it's my first time. There it is. Very good. What are we going to see today? I think I missed a slide. Let's see. Yes, I think it's not pulling this. Do you see it changing? Or not?

Perfect, thank you for being here. As I was saying, I'm going to talk about four incidents or crises where there was an intrusion or an affectation where I worked before at Vision Fox. I tried to be a little varied, we are going to talk about a case of CryptoJacking, then about SPAY, some of APMs and then Ransomware. At the end we go with the questions and answers. Very well, I introduce myself again, I am Andrés Bolivar. I have transitioned between many cybersecurity areas, mainly in the offensive part. So I've gone from offensive to forensic and again to offensive. Currently, this is how I'm operating in Mission Fox, purely offensive. And I've undergone a lot of transformations. This is the... after 20 years, this

is the art that I sell today. And, as I was saying... What are we going to learn today? Well, I want to introduce you real attacks that affected the business, in most cases. Some of the techniques or tactics that the attackers used. What happened in the incident response? What were the gaps that caused these attacks not to be detected? And what was learned or what changed in the organization at the root of these attacks? So, there will be failures, failures, failures, on all sides. But I would also like to leave a message that in the end crises are good. The truth is that I have been given a lot of cybersecurity crises. As they mention

that they thank developers for having vulnerable systems, crises are also good. and then they give you a lot of things. In fact, that's how I met my wife, during a crisis. And also, friends, be kind, because when you're working in crisis and the other person realizes, especially if it's TI, that you're there at the bottom of the barrel, they end up loving you too. They'll appreciate you. Okay, so let's go to the first one, the cryptojacking. This attack was seen a lot, I think, 10 or 15 years ago. It happened a lot. Now it's more like the ransomware, but before they used servers to mine. A little bit about the story of this attack. I think this attack

was one of the first that I had to review as a forensic analyst. And well, if you notice, what I mentioned, I worked a lot in the pen test part and it wasn't so natural, I feel, that I did forensic. I think it's an activity that goes better with incident response, with SOC, I don't know, but many times it's also the pen test. And at that time, I'm talking about 15 years ago, I had to as part of the Blue Team's functions, define security standards. So, many times there was this battle between IT people, developers, because the standards were very strict and caused the endpoints or the servers to slow down. And this was the case, a little bit. This

case came from a a server, a telecommunications area that was with the performance very high and it was like that for one or two months and obviously who was the first to blame was the cybersecurity area because they are the ones who implement the controls that make things go wrong so who designed those cybersecurity controls? it was the responsibility of my area or the area where I was What happened was that we did a pentest and then, as a result of that pentest, a new cybersecurity control emerged. I'm talking about 15 years ago. Currently we have things like SIS or other compliance systems. So, at that time we designed these cybersecurity controls and we were the first to blame. We arrived

at this server, specifically me, and when we checked the processes, we saw that the Htop was 99% of its capacity. But in this specific case, when we checked the processes, we noticed something strange. There was a specific process that was consuming all the RAM memory. So, the alarms are raised immediately. This is a crisis, what happened there? I think this was, if not the only case that I had, that everything was done under the rule of forensic analysis. We were in that time taking a forensic course, D.C. Consult I think, and here we did, we carried out everything. Alarms are raised, the alert is raised, the whole chain of custody is done, you know, right? The

servers are removed, hard drives are extracted, cloned, returned, and when we do the forensic, as it should be, we begin to find things like XMRig-type scripted cryptomining, right? That was basically it. Following with the forest, we see that the root of the vulnerability had been a JBoss that had a vulnerability associated with serialization. We were able to corroborate this later because the WAF had detected it, but it was only in the advertising mode, you know, it was in advertising mode, it didn't block anything. And why didn't they block anything? Why? Well, there was also this issue that the WAF sometimes erupted in the operation. The fact of suddenly blocking things caused things to fall sometimes.

So there was always this dilemma, this fight between IT and the AI. Well, this is more or less the script, one of the scripts that there was. where it was very clear that they were connecting with a wallet so that everything they mined would go up to that wallet. And now we move on to the Spey case. I'm going to try not to mention bank names, I don't want to offend anyone or sales, but if you hear the name of any bank, make the case.

So, I'll give you a little context of Spain. This is the diagram that you can see in the circulars of the National Commission or of Banxico, where it seems very simple and for the user it is. You want to send money, you do it through your bank, it has a supplier, generally passes through a certification and the space is sent and the beneficiary receives the money and behind this there are many requirements, many requirements the provider I tell you that sends the space must have a certification, redundancy must sign the petitions, a series of things But all those who plotted this coordinated attack, because many banks occurred at the same time, and they were working in the banks, by evidence of what there

were in the banks, in some cases for up to a year, I don't know, doing intrusions, like red teaming, gaining more and more knowledge of the network of that bank, decided to launch this attack one day. And it was very ingenious because in the end, if all these SPAY measures that had to be done so that you could send a transfer, in the end it was a table where you had to insert the movements, the idea was that most of the attacks that occurred in the banks, insert directly in these key tables where the SPAY vendor pulled the transactions and then sent them to Banco de Mexico. So, the truth is that I don't want to think badly about all this, but

sometimes it's inevitable because all the banks sent information to Banco de Mexico about IPs, tables, how the transactions were carried out, and many of them only used two or three SPAIN providers that were certified. And I remember, two in particular, that were the most affected. So what happened in this bank? That in the end it was an attack on SPAIN. Miraculously there was no economic impact. I'm going to explain it to you. Like you lose the special edition.

I'll tell you about it. When the SPI issue happened, around the end of 2018, as a security area, we were in charge of dealing with rare cybersecurity issues. It was already at the second level, not only the incident, but also the forensic. The IT area that was in charge of the entire bank, reports that its main database could not be failover because a query that was carrying massive transactions was blocking the entire database. So what did this mean? That it could not be backed up. That was the alert, right? To give a little context, this infrastructure was very sensitive, it was based a lot on Windows Active Directory, they didn't use any security tools, there was no antivirus, there was no EDR, there was

no network protection, there was only segmentation as a level of protection on the network. So we arrived and the query that was blocking The database turned out to be consulting the transaction database. It was a consultation that was bringing a lot of information and it was left there. Super suspicious. So, the crisis immediately arose. And you know how it goes in the Forensic, right? Every few minutes, there's always the question: "How are you doing? Is the Forensic done?" But in the end, without logs, without information, we didn't know how to know where this attack came from. And we couldn't do the typical case of Forense where the service is downloaded because it was the main

database of the bank and it couldn't be downloaded. So what do we do there as a Forense team? I think I missed something. There it is. Very good. So what do we start using? I don't know if you can locate this tool, Sysmon. which is a Windows service that you can download from the Windows website that allows you to add additional things to the Event Viewer or to the Microsoft Auditor Once installed, we started collecting logs and we started seeing activity like this PowerShells, there are many PowerShells but encoded, in the end they were like stagers like the one you see here If you notice, it's the classic stager, that a file from a remote server is downloaded

and then, using reflection, because it was surely a .NET file, it was executed. So it was very obvious that something was happening there. And part of the information that we started to compile in that incident, in folders that we were finding, tops of the database. So, one of the first conclusions was that they surely didn't know how to make transactions to the main bank table, but they were getting information first, so that later they could insert it. They were seeing that it was necessary, the columns where they had to insert, if there were indices, if there was to sign something, etc. So, the earthquake started to give us traceability. We started to see servers to

which the attacker or attackers had jumped. And we started to go back, back in the network, which was what was happening. And in the end we discovered around 80 servers where the attacker was stopped in some way, leaving artifacts, leaving things and most of these acts we were seeing through the earthquake. But what was happening? While we were still checking the logs, the attacker was still there. It was something we were still seeing. We were still seeing activity, they were still wanting to connect to the database, And in the Inter, we heard from other banks how they were hitting them. "They will lower so much money to this bank, etc." What happened here? We saw that the initial foothold of the

bank network was through a server that had a port 8080, I say, guess what it is, and it had a vulnerability. So, that also came out in the logs. How they were exploiting that vulnerability to assault the bank network. We kept dragging and dragging backwards. So, it was the cat and the mouse. But well, what happened? One day the attacker simply said: "No, I think they're going to give me and I'll stop." So, a lot of the learning from this attack was that Sometimes the detection has to do with luck. In the end we couldn't find the patient zero of the entire corporate network because the attacker stopped acting. Although the security measures were reinforced,

there was no activity or artifacts that would make us understand that there was a vulnerable team or that it was there. What did the bank decide? Well, remove the SPAY providers and make their own SPAY, certify themselves through banking. With more security measures, more reinforced, apart from the SPAY binary, it had to be signed. And well, this left us a lot, the issue of the logs. This was one of the cases where possibly the IT area was better merged with the security area. because being an area where they didn't want to know anything about the security area, from this incident we gained a little respect and the need to have stronger security measures. We move on to

the third one, ATMs and Transaction Reversal Proof. In this incident, In the end, the issue of ATMs was very new to the security area. We focused more on the web, infrastructure, mobile, etc. But this incident arose from the fact that the cashiers began to empty very quickly. It was an activity that lasted longer than the they would put money on it, it would take a few days to empty, and here it would take from one day to another. They would start to empty, empty, empty. And they would specify more some than others. What happens with the ATM? Well, you know, they are a bit like stupid terminals, where many times the docks they generate are not linked

or they are not online, they are not sent to a event correlator, etc. The reasons can be many, or the reasons that they give are many, many times they have to do with not wanting to invest in that part, because the links are in a certain way and they don't want to pay a more dedicated link, etc. So many times there is no traceability online of what is happening until they are making conciliations, they are giving themselves account. So in this case, after they emptied several thousands, after a week they realized. This graph that you see there, the colors represent the four main ATMs that were being emptied and on the left side is the import. The part is being graphed in gold.

You realize that in an hour it was emptied almost 75,000 pesos. Well, reaching a very high part, 75,000 pesos. We did these graphs later, but at the time I didn't know what was happening. So, what was this attack about? There was an attack that started to happen a few months before this happened in Europe. I'm talking about 2020 or something like that. It started to spread a lot in Europe and then it reached Mexico. You're going to see a lot of these messages called E3. You're going to see that they are mistakes, but they are actually messages of the ATM that is sending when something happens. If you know, the ATM has some cashiers, the money goes through the cashiers, it goes back

through a band that raises the money and then it goes up to the top of the dispenser until it reaches the shooter, and the shooter is the little door that opens just that has to be opened right when the money is there. It can't be opened before because other attacks could happen there. So it has to be opened right when the money reaches the end. So, during the journey certain things can happen. When the money is dispensed correctly, the message is an E0, generally. But if something occurs behind the dispenser, it's an E2. If the money couldn't be dispensed because the shooter got stuck or something happened there, then we talk about an E3. Sometimes there can be timeouts, some banks manage, if the

person doesn't get the money, then an E3 occurs. But generally it's because something happened, something obstructed the shooter, so we talk about the E3. For convenience, or to make the experience more friendly to the user or the client, when it happened that the cashier didn't give you money, I don't know if some of you have heard of it or if it has happened, that the cashier doesn't give you money, or the shutter doesn't open, or something happens, sometimes you have to make an explanation. and then that clarification took time and they already repaid you. So many banks started to do the reverse of the money if you didn't give it automatically. That's where this security failure arises,

you know, many times these security failures have to do with something that benefits the user, right? In this case, the reversals automatically. In Europe, they started to realize this, about automatic reversals. So, how does the attack work? You ask for the denomination of bills that could give you more. In the case of some banks, it was around 8,000 pesos. So, when you ask for 8,000 pesos, they give you more bills, from 500 to 200, and so on. I specify that amount because in most withdrawals you see that amount, 8,000 pesos. So I asked for the 8,000 pesos, just when the shooter opens, the cashier is blocked and this generates the error E3, which will return the money to the back or to the ATM return tray

and at that moment they were clicking the money. So the cashier kept thinking that he had returned the money back The E3 was produced and the attacker took the money. As the E3 had occurred, the automatic reversal was processed. This is more or less how the ATM's journal looks like. I tell you, they are like pure messages. They don't have much structure. It can be like pure text. And there you see the message of zero, where it already presents the money and then, as soon as they blocked the shooter, Then comes the cash register, the amount, etc. All of this is stored in the cash register and some banks collect it to make conciliations. It's a pretty good window of time

for someone to attack your cash register and try to make a jackpot. We started to graph those journals. by two brands of local cashiers that had DIVOL, NSR, here I mention the brands because this one does not worry me much about balconizing them. In the world of cashiers, what you are going to see, some who have worked with cashiers, is that many times the sellers specifically do not want to share information, they have many of these security practices by darkness and when one sees how to hack these warehouses, you may have seen it. So, what do we see here? Which was the most attacked warehouse? Because both worked very similar. In the case of D-Ball, where we saw more

reversals over those two weeks. It happened that in one day there could be up to 400 reversals. That's why these amounts are a bit scandalous. where they would go, they would ask for $8,000, they would click, again and again and again and again throughout the day. So it was information that we started to generate, right? There with Splunk, which was the tool we had at that time.

Now, the case of the E3 errors or the E3 messages, you could also see there, very big peaks, because they were generating so many errors in E3. In that part, they were like the top of withdrawals, the circles are cards, debit cards. So, they were like the cards that had the most withdrawals, which is also super rare, and it was withdrawing and withdrawing. There were also more business rules missing, around how many withdrawals you could make, etc. Many of those business rules were just canceled because of applying the reversals, etc. What happened after this incident? Because in the end, we came from an area of security where we did more web sales, infrastructure, etc. and as a result

of this, the need arises to make an area dedicated to ATMs. So that's where there is also a gain, what I tell you, right? Sometimes the crises are good because the areas start to work with synergy, new areas are created. From this we created a laboratory dedicated to cashiers. And we started to make sales dedicated to cashiers. And here, well, I wanted to show you a video, I don't know if it can be played. Let's see, maybe if I go here. There it is. This is a TRF on an NSR, right? So, do you realize? We already had the technique, right? Where to prick? Because in the end, the shooter, I told you it was the door, but on the sides there

are also gears, which are part of the shooter's mechanics. So, if those gears stop, that's where the E3 occurs. This is like an NSR. No, you don't hear the audio, but if you hear the audio, you can hear a "clack clack clack" because the same gear is trying to spin. This is a D-Ball box, open, and let's see if they can help me to reproduce it. To make it work in D-Ball, the TRF was not to use the thumbs, but you had to put your fingers in the bottom part of the door, and there it was blocked, the shooter started to force itself, and that's when the error occurred. Very good. Now let's move on to the last

one, a ransomware. This one in particular is newer. Let's say that here we already had a certain reputation around forensics. So one day my former boss calls me to see if I could dedicate some time to this forensic. I was already here in the company I worked for. And it was like an extra job, right? I don't know if I can say it here, I hope my boss doesn't know. I mean, some people dedicate themselves to Bug Bounty, why not do Forense in your free time, right? But well, you know, Forense is very demanding, right? Sometimes you can spend hours at night ordering pizzas, tacos, and you keep going until the next day, super demanding. So I said, "You know what?

I do have to work, I have some pending work." And he said, "You know what? You can go with some consultants, go just on the weekend and come back on Monday." And that's it. Work on what you have to do. But the option was to do it remotely, because I can't do it remotely. You know, the urgency of the management or who is demanding to know what happened, it helps to make bad decisions. So I said, "Okay, that's fine, they will send me to Atlanta." Because the bank was in Atlanta and there we go, three consultants, Atlanta, we arrived on Saturday, early, we arrived at the offices of the bank and it turns out that they are not in

the offices. I was like, "They're not going to the prisons, right?" "We're going this far." "No, they're in a town where people are really in a town that's three hours away from Atlanta." "Okay, well, rent a car." We rented a car. and to manage the place called Greenfield. So we arrived at the offices and there was no one, even though they were in crisis because they had all their servers encrypted, endpoints encrypted, etc. There was no one. After the day came, someone arrived, but we ended up doing this incident response, or crisis, we end up making it remote. But in the physical facilities of the bank, everything is bad, right? And well, what impressed me is that

on the way from Atlanta to Greenville, they don't know the number of churches and armories that are there. It's incredible, right? So I don't know what's going on there, right? This was a relatively quick process because the idea was to guide those who were going to stay but by Sunday we already knew what had happened. They stayed but I had to go back. So to go back I took a return train. It was a train, a train Monday at 5 in the morning to get to Mexico City on Monday and be able to work. But it was also another experience, right? On that train, I was with my eyes, you know, super awake because I felt like they were doing something to me, right?

Now, I tell you, this was very simple. What happened? Well, you know, from previous experiences, they did have EDRs, antiviruses, but I had not detected them. So, one of the first things we did was the same earthquake, right? In this case, right? and what we noticed in some endpoints is that a specific user had received an email and that email was from a OneNote that had a variant of LockBeat that was spread through SMB. So this is more or less the structure of processes that we get to see, that is, the user executes the Outlook.com, well, your email opens it, opens the OneNote, and then this series of processes are run, right? A CMD, then a PowerShell, and

then the Stagers, until the binary was downloaded, where all the magic was already done. So, what did we learn here? Well, that the logs always give you heart, right? If you don't have logs, I think that's the best message I could give you, When they make a forense, it's between more locks, more hearts, and it gives them a path to go. It doesn't have to be a sism, because there are more advanced things, but it was giving us information to see where we were going. Likewise, many times the vendors sell you things and many times they block things that can hurt you, but You know, the attackers always find a way to bypass everything. And you know, here they will spread

their files. We move on to the final thoughts. Well, here I added this slide with some of the stagers I've seen in the forensics. I uploaded some in my GitHub that is there. If you want, wait a little longer. I think I'll be uploading more this week. Because in the end it's a world that has been interesting to me a lot, especially because I'm there, you know, behind the OZ and then my coco are like these two strangers. What do these attacks give us a little? You know, right? Real attacks do damage, right? Yes or yes, they do damage. Many times one may be wanting to follow the Easy Response process, but in the end you have to be improvising. on the way, seeing

that people can serve you, where to move, etc. And well, you know, right? If you are resilient and have a little humor, then that is the recipe to survive. And well, thank you very much. I don't know if you have any questions. We open the question and answer session. If anyone has any questions about the courses we just saw or about the lessons learned at that moment, remember to raise your hand to answer them with the microphone.

I am a little bit of the same the question is directed to good I imagine that in these situations well when I imagine that for example in the case of the box office that maybe you had to learn new things to know how to find the root of the problem. So the question is that in these cases, maybe when you are in a time trial, but you need to learn to do something, what methodology or what do you do for these new knowledge? Yes, in the case of the cashiers, there is a very large page about the XPS, which is the protocol that the machine uses, the Windows, to communicate with the peripherals. You're learning this, you're going to investigate it,

you're learning it. But the documentation, I think, is the main thing. what does the documentation say? also, the vendors had some documentation that they shared with us so there came the description of the messages because if you saw the journals you only see the message that it brings, right? e*0003, I don't know but you see the journal and you don't know what it is, right? so, the logs plus the documentation gave us a complete understanding of what was happening step by step in the cash register especially if you are going to dispense. That's one thing, and then you corroborate it in practice because you are doing your dispensations and you are seeing what is happening. Obviously, you need access to to the device, to

what you are going to test so that you can corroborate that what you are manipulating is being reflected in the log. So, I summarize it in that way. In the end you have to read the documentation, yes or no, you have to shoot it and you have to have logs to know where or how you are going to interpret them. Very well, thank you very much for the question. I have a question for the banks. If they had losses, I will go deeper into my question. How much did you have to go through the process of attention to the dissident to identify the causes that generated these fraud events? In the case of unmasking responsibilities, I haven't

had to see with facts what happened, because in the end, although We also prepared ourselves for the issue of scams. I haven't had to go through a scam because in the end all these cases, the results were delivered to the security area. But I'm not talking about information security, but physical security, which is the alias of "guacanazos". So I don't know what happened after that, how they used that information. What they did see was that it reflected a lot in current security policies. They started to make more controls, they let the security areas enter their processes, etc. But as for the culprits, many times I didn't have to see if they were taking legal actions or what was going on

there. Okay, next question. We still have space for two more questions. Well, I don't think there are any more questions, so let's give a round of applause to Ben. Thank you very much.