Andréanne Bergeron - Reconsidering Cybercriminal Expertise Through their Behavior with CLI vs GUI

Okay. Um, so our next talk we have uh Andrea with uh attacker behavior. Hello. Hello. Can you hear me? Yeah. Okay. Hello. Thank you for having me today with this presentation. I'm officially the third French accent. Please bear with us. Um so yeah today the first impression that you might have with this title is that it's very niche right but I promise that we will be able to do some generalization about attackers behavior and maybe there's sophistication level throughout this presentation. So what the idea of this talk come from is that in the media and for you know non-expert uh the the actors the malicious actors would uh this is how they would imagine their screen and their you know uh

skills with with computer in generals and with code and everything. So I had this question this is this true uh what people think of malicious hacker are they really good with code line and you know command line and everything or or are they not as sophisticated we as we think so this is what the analysis will a will enable us to conclude today let me first properly introduce myself so I'm beron I am from kebec city of course with the accent I'm the director of research at go secure um and like I have done all my my studies in uh social science okay in criminology so I have done my PhD in criminology um so I have this social

science background behind this this cyber security research um so you'll see that my research is very tainted by psychology and criminology and the law and everything so I keep close tie with the university as an affiliated professor at an invest in moral and I'm also present in my community in the cyber security community as being the co-VP engagement and outreach for Nordsk conference in Montreal in May uh if you're looking for new conferences around Canada and I'm also a board adviser for the Canadian cyber security network. So enough of me in this research uh and throughout this uh this presentation uh we will consider the use of any type of prompt interpreter such as command prompt,

PowerShell, terminal, bash, anything um as a command line interface. So CLI right? So from now from now on I'll say I'll use CLI just to make this presentation shorter. Um, and we will compare this type of behavior like using a command line interface in a position by using the guey. So just clicking your way through uh what you're trying to do your actions on a computer. Um, so at a staggering 87% of people and people being probably non-expert here uh believe that malicious actor pro possess exceptional computer skills. Right? This is where the idea come from. Um, exploring if this is just a perception or the truth is what interests me in this research project. So, experts say that you know

it's possible to become a great actor without coding knowledge but having coding knowledge makes it a lot whole easier. uh but you know the the CLI will provide faster approach to a compli task and and even allows more flexible approach to completing complex and repetitive task and and also from my experience understanding how things work and using the CLI to your advantage is at the core of being a hacker right and and I'm not talking here about malicious hacking but like a hacker in general so in my mind I we are also exploring in fact if malicious aer are also like are similar to ethical hacker. This is who I know like I work with ethical hackers. Uh I I

start to understand how they think. Uh so let's explore if it's the same thing for the malicious hackers. So researchers indicate that because of course I'm a researcher I have to explore what previous researcher have said right just to make sure we are going in the right direction. And so researcher indicate that while there were some common ground between professional hackers and practitioners so I'm not on malicious hackers already um professional you know IT professionals uh professional hacker tend to or professional hackers or ethical hackers whatever we call them um tend to use automated and repeated attack as well as creating new tool using the CLI right while um the practitioner will use multiple task to minimize the effort with the aid

of existing tools and this is in this context that they would use a CLI. So you know their their usage is a bit different here but now if we compare ethical actor or professional actor with malicious actor this is what we know from re previous research. So ethical hacker believe uh malicious actor are lazy, irresponsible and not very bright. Uh and and non-expert for their part will will say that that malicious actor have and I'm not sure that they do the difference between ethical hacker and malicious hacker because they don't know uh but but they perceive malicious actors as a master of their art and highly skilled. Right? So let's test all that. Uh so how do we study the CLI used by

malicious actors? Well, it's in observing their behavior directly. And how we do do we do that? Um well, in this case, we did by um operating high interaction on aut. So we place a real window server, but it was fronted with our monitoring tool called Pyrd. Um it was a research onipot. So the only objective of this onipot was to observe uh actor's behavior. So obviously we did not hide it behind a VPN. We wanted to be attack. We wanted to observe their behavior. So uh so we exposed it on the internet. Um and PRDP more precisely does a lot of thing. Our monitoring tool it does a lot of thing. I will not go

through all that. If you're interested, you can just Google it. We talked about it uh plenty of time in the past and it's an open-source tool. So you can all use it. Uh but the thing that I want you to uh remember from biod is that it gives me two type of data for for research purposes. It gives me data before confirmation. So all the attempt logging to our system I have all the information related to that and it also give me information after comp permission. So once they are in our computer in our system and this gives us a lot of information. So in four year we capture 190 million events. So in other

words log lines and this include more than 20 million attempts login and 3.2 thou two 2.3 sorry thousand RDP capture in video output. Okay, because our tool is very cool for that. It gives us video output. I'll I'll come back in a second but I just want you I just want uh to show you how it uh end up for what we are analyzing exactly today. So from the log line the million log lines uh we have successful loginins. So we will concentrate only on successful login. Um but there's also like um uh replay file that are created from successful login and because we have an epiput with um TLS and NLA uh in TLS like the endshake

is made before so it creates a replay file but it's useless because it's nothing happened and there was not a real connection so um I'll skip the detail for that but in the end there were and sometimes they just connect to the server and and and disconnect for some reason. So maybe a butt was just scanning or something. So in the end I have 454 session with content. Okay, with something to analyze because I was interested in analyzing something. Um so this is what we will talk about and this is what we analyzed. Okay. So I I come back to the video content. Okay, it's really it's truly a video content that we that we analyzed here. So when the

video will start, you'll see the screen of the attacker in action right there. You'll see the mouse movement uh with a little yellow dot. You'll see that our tool provide everything that goes through the clipboard and through the key keyboard. So everything that they type and at the beginning of the session we also see the creds with which they enter our system. So you see that we make their life easy with administrator admin as the as credentials. So, first you see in the clipboard there's our IP address, which probably is how they they found us. Then you see when they type on the screen, we see it at the bottom. And this is very useful when they type

something that we don't see on the screen, like changing a password. And what is interesting with this session is that they kind of connect their C drive with our session which allow us obviously to um grab everything that is on there and then they uh they proceed with their activities which is crypto mining in this case. Okay, they are not doing all the same activities but this is what they do here. So reminder 454 session with content. So session would be like one of the video that you just saw which represent more than than um 100 hours of video footage and all the video happened between January 2021 to June 2023. Uh so the result the first observation

that I have a lot of results for you today but the first observation we made is that how many of them would use the CLI versus the Guey and you see that the white one use the CLI. So it represents only 8%. So we already got our conclusion to our our question uh at first. So uh they don't use CLI that much. But then the like I will not stop here right because it's not enough for me just to know this information. So the first question that come to mind is that how is this group different from this group right? How uh how can we describe better the first group because this is uh the one I'm interested in.

So there's different action done by the attackers in our system once they're in right they add a they add user they change the system password they check for user information check the CPU of the system check our IP address test our internet speed turns out with Windows Defender turns off Windows Defender erase file usually files that were not there first but that they put and then erase file so they erase their traces is download browser, uh, paste files. You know, there's plenty other, but let's just look at those ones. Um, so in blue you have the CLI user and in red you have the GUI user. So you kind of see what type of activity

the CLI user are doing the most uh, in the sessions. However, this graph is totally biased because I just show you what the activities of CLI user are doing in the in the session, right? So, because this is what interests me. So, just keep in mind that there's other activities that they do not perform, but this those are the one who perform. And as a researcher, I am obsessed with statistics. Um so this is cute but the the significant relationship between the variable would be only the one that I circled here. Okay. So CLI user tend to add user change password check CPU checks IP info erase file download browser paste file more than gooey user would

do. Um so yeah I just want to make that clear. Um so there's two trends that appear here at first is that they do reconnaissance activities uh like changing CPU change IP information and we also learned that there's ev evidence of them getting comfortable in our system because they're they they seems to spend a little more time than the other uh because well they add user they change password they paste file so they they're here for for a long time Okay. Um, but so stat I I said that it was statistically significant, but if I go if I geek out a bit on the statistic and I and I swear that it's my only only

slide on statistics. Okay. Uh, but for me like statistically significant doesn't mean that the relationship between their variable are strong. So I simplified this with this graph. We like visual representation, right? uh so the strength of the relationship is very low okay it we have I put the maximum possibility and the minimum possibility here uh and you see that it's very low like it's not it doesn't explain the relationship um uh highly so uh the only thing that we can conclude in fact from those observation is that uh the main activity CLI user would do in our session would be changing the password. So, uh yeah, I just conclude that. So, no trend that is convincing uh enough for

me to drag conclusions. Uh only conclusion is about the activity uh that we see here and it's mostly to change the password. The activity done through the CLI would be to change the password. Um, and I asked myself another question because like based on the observation that I did uh before, do they type in the CLI? Like do they actually know the line of code or are they using pre-written script? Right? Because sometimes they just paste the binary and then launch it and it would count as a as using the prompt but it's they don't necessarily know what it is. Um so and we saw that there is almost 20% of the CLI user who use pre-written

script. So they might be a different group of CLI user within the same group fact and uh what do they do with pre-written script? Uh here are the answers. So collecting information on the system generating proxy and use of API tools. Okay. When analyzing the video, we noticed that some sessions were related to each other. So, and I have a couple of of example uh some uh sessions seem seamlessly picked up at the same place. So, they started to do some activities and then leave but then come back and continue the same activity. Um other than that, there were like the same IP connecting in the short amount of time, two times. So it indicates me

that they might be related. Uh they they change the credential and then come back again with the same credential. So obviously they knew about this new credential. So all all those uh information allowed us to um to merge the session together and we pass from 36 different session to 27 different attackers. So we lost a couple but then the the analysis are um are more precise on the type of attacker than on the type of session. So attacker entered 2.15 total prompts during their session which is not a lot h and 1.2 unique prompts. So uh so only one right uh well almost only one um unique actions in the session 52% of attacker had only one total prompt

throughout their session and among attacker using CLI 33% use it to only change the password while not performing other actions. So this is what we conclude from the first test, right? They use it mostly to change the password. Uh and they do not use it a lot once they they're they're in in there. So then I wanted to see the sophistication level the sophistication we can have a debate on what means sophistication but here it would be just uh are some of them of this group better than other. This is what we tried to evaluate and how do we measure sophistication level? Well, um I explain our our process. So here we will have points for those uh those behavior.

Okay. Attacker does more than 10 action through CLI throughout the session because it was really rare. We would consider that like uh like worth a point. If the attacker does more than four unique different actions in throughout the session um we would add a point and then attacker if the attacker use tools or script uh in a CLI which is a bit more complicated uh we we would add a point we would also substract points for certain behavior some attacker would start something and then was not able to complete their action kind of were not able to didn't know how two or you know uh stopped in the middle. So this is considered a failed

action. So I would uh take off a point and uh if the attacker does one of the following basic action we would take out a point. So add user, change password, user information, IP information. If they do that through the GUI, we would subtract a point just because it's one line of code like we just expect that they would uh use it with the CLI, but if they don't, we subtract a point. Um the so the most common uh sophistication score what was minus one with a downward after that. uh so the and and the perfect score of tree was totally absent. So the what we can conclude from that is that they're not really good. In

fact uh and this result means that malicious actor who are not doing reconnaissance but rather getting themsel comfortable uh with their tools and using them are a bit more sophisticated than those who would do who would perform on the reconnaissance right uh it also mean that having a sophistication score it's kind of worth it to uh to to understand their behavior and their sophistication behavior. So uh so a higher score score score was was associated with using tools and installing tools on the system. So using tools would be associated with um with more sophisticated. So the general conclusions uh would be that a low number of attacker use CLI right this is what the first observation we we made

told us malicious hackers show a low level of sophistication. Second conclusion and uh previous research mentioned that CLI use is associated with professional actors and it might go into that direction. I am not able to conclude exactly that because I was not comparing with behavior of ethical actors here but it it points toward that direction because now I know that malicious actors are not using CLI much. But there's concluding thought that I wanted to uh to talk with you and maybe we can talk more if you have ideas later. Um but you know previous research that we have done have shown that uh attackers might work in group. Okay. And this group and in this group there's

people that might be more sophisticated than than than other. I heard the the term associate uh in this case. So the associate would just perform what they were asked for uh which is maybe launch uh a script like this just they don't know how it works. They're just paid to launch it and and wait. Uh so this uh this is to bear in mind just to understand that in the same team of attacker they might be uh people better than or more skilled than others. Um it doesn't mean that they do not they do not have the capabilities to use CLI. I have no mean to to prove that for sure because uh like uh I didn't like if you

um uh not like if you avoid putting the if you obligate them to use CLI would they be able to do it? Okay, this is something that I didn't test for. So if they had a choice, maybe they could demonstrate capabilities to do so. And I can't I can be sure. Um, is typing command more at risk for detection? I don't know. It's a a question that maybe you can help understand. So is that is that something they took into consideration like they they they are scared that that typing a command would would uh would flag uh would yeah make an alert. Uh so maybe they are avoiding it for this reason. We don't know. And uh don't we all use the

gooey at some point because you know even if you know how maybe you're just lazy and and sometimes clicking is just easier. It depends on the amount of code you need to to perform your action. So you know this this is all what we should keep in mind in the end. Uh so I would be happy to answer your question if you have some. Yes. You listed four or five things that people who use CLI for their common activities. Did you note similar activities with the or were there different uh type of actions that they performed first? Uh I'm not question. Oh yeah. Can you please Thank you. You mentioned there were four or five

activities that were common when a user um performed actions during CLI uh activities. Are there any common were they similar activities if they connected via guey or were there different uh four or five activities that they would perform? So they they will all perform those type of activity like those type of activity are really present using CLI or not. uh but they seems to perform it more the the CLI user seems to perform it more so statistically um so a bit more in this group but it's it's present all over the the group the both group thanks hi um yeah I thought this was really great really interesting um but I guess I was kind of wondering so you said like

in your study design you didn't put a VPC or VPN or protect it really just put it out there so anyone could access it. I was wondering if you thought, you know, it was possible that that might have introduced a selection bias towards less skilled hackers. Like, is it is it possible that the more sophisticated, more skilled people wouldn't bother unless someone at least put in a bit of effort to try to protect it? Yes, thank you so much for asking this question because I love caviarajin research and and this is my opportunity. So um yes, we have to bear in mind that my uh my sample is biased because it's exposed like our our onepath is exposed and it

might attract attackers that are interested in low hanging fruit which can make that they are less skilled than other that would be looking for for for you know um organization that protects themsel better because maybe they have something to protect right So you're yeah on point line with that. Yes. Uh was your honeypot only accessible via RDP or or were other protocols open? Yeah. So I I didn't in face uh put the infaces today on that but yes it was purely RDP. So it there was no other way to get in uh and all the attackers that were able to get in was by brute forcing RDP. Yeah. Yes. Hi. Um, so you only used Oh, hello.

Hi. So, you only use Windows as a as a test platform, right? So you didn't use Linux like anything that's traditionally guey because if I were to like say use your system I would have probably used like L use or MGR as well to change a password over say something like IP config for uh for like say command line just because that is not so much a sophistication as it would be say for me I would think of efficiency as in how quick can I get the action done and I think for that aspect for my own self right you know versus say if you were to go into Linux you wouldn't install X Windows to hack into it. So I was wonder

if you took that um into consideration when you did your sets. So yeah, we didn't because I couldn't. Uh but but this is like one of my concluding thought like to keep in mind, right? Are they are they just lazy? Are they just do they just prefer using the gooey for certain actions? So uh so I was not able to determine this, but it's it's something very important to keep in mind. Yeah. Yes. Uh were you uh detecting any uh script execution like uh probably someone just logged in and executing a script of one one by one commands and they were not typing anything at all. So were you able to detect that or Yes. Yes. Uh so I call

that a pre-written script that so they enter they paste their binaries of pre-written script they just run it. uh they grab and most of them were uh to grab information about our our system uh and then they erase the files and quit. So they would do it on for purely reconnaissance activities but we yeah we witnessed that questions. Okay, thank you so much.