Creating the Torment Nexus: Using Machine Learning to Defeat Machine Learning

Uh good morning everyone and welcome to the Las Vegas Psides 2025. This talk is with Noah Gro uh give on creating the torment nexus using machine learning to defeat machine learning. So a few announcements before we begin. We'd like to thank some of our sponsors especially our diamond sponsors Adobe and Iredo and our goal sponsors for and drop zone AI. It's their support along with all the other sponsors, do donors and volunteers that make these events possible. So these talks are being streamed live and as a courtesy to our speakers and audience, we ask that you make sure your cell phones are set to silent. With that being said, we can stop with Hey everyone.

How many of you guys have seen this uh this tweet before? >> [snorts] >> Um, me and my friend were actually talking about this tweet the other day. Or >> is it not on? Can you hear it? >> How about that? Can you hear that? >> All right. Uh, me and my friend were talking about this tweet um about a year ago. Do you know this guy's from The Onion, by the way? Um here's here's me and my buddy Adrian at the pub last year. Um, I'm the guy on the left. I don't remember much about what we said, but I do remember the most important parts. Uh we were talking about pandas specifically how you could convince an

AI that it's actually a monkey. [snorts] Uh this was the talk the topic of the talk he gave earlier that day about how you can fool AI into thinking something is what it isn't. Uh how do you do that? By adding noise. [snorts] If you hide that specific pattern of noise in an image, it messes with the way the computer thinks about it. And that's enough to fool a million-dollar algorithm. But I can hear you thinking. Unless we come up with a way for this to fool women on dating apps that you're actually Chris Hemsworth. This doesn't do do much for you, does it? But fortunately, this got us thinking. More and more AI models are starting to be

used in cyber defense. Would they be vulnerable to this kind of noise, too? Hello everyone. My name is Noah Groch. I'm a 21-year-old graduate from UNC Charlotte in North Carolina. Uh, I enjoy playing nerd games in my free time and I have four pet ferrets. Last year, I was invited to work at Dropbox in order to work on this project that I'm presenting here, uh where basically I get to uh gaslight AI into thinking what I tell it to. Now [snorts] even though malware and monkeys are seem like two different things, this idea actually did make quite a lot of sense. Lots of people don't know, but this is actually what AI looks like. Machine

learning algorithms are really just giving a boatload of numbers to a computer and telling it to figure it out on its own. Our monkey example is no different. Even though we're going to be giving the computer an image, it still turns it into something that looks like this through something called an extractor. Now, most modern versions of this are much more complicated than that, but it pretty much the same thing. Uh, and that general idea can be applied to files, too. Our version of that is called LEAF. Leaf, instead of taking images, we'll take executable files to turn them into numbers which can be used by the computer so we can think really hard about it. Now we just need the part that

thinks really hard about it and that's going to be our tool right here. This is Ember which is stands for that. Uh what it really is is a 1 million sample data set filled with 10 gigabytes of data collected using leaf from earlier. It means 1 million lines of this. This is Ember's representation of how it sees every file it's been given and it'll judge every new file based on these examples. Basically, after our file has been processed by Leaf, Ember takes a look at it at the numbers it spat out and gives us a percentage chance that it's malicious. My job is to figure out how to give it the right amount of noise so

I can fool Ember into changing its mind. [snorts] But in order to convince Ember that my dubious files are actually just for paying your outstanding toll balance, I'm going to first need to figure out how Ember thinks about the files it looks at. I know just the tool. To quote a reputable source, shapely values are a concept from cooperative game theory and explainable AI used to distribute the total gain or loss fairly among a group of players or features. Or in summary, it makes cool graphs so I know what the AI is thinking. Here's one of those cool graphs. Uh it shows a specific feature on the left and how it impacts the overall output of the model.

You can easily see the feature names on the left and how much they influ in influence the final decision. [snorts] Very useful for what I'm trying to do. Now, let's check to see what it does for my project. So, you can see that um feature 637. Uh it's not very helpful, is it? It looks like the Ember data set doesn't actually ship with any labels whatsoever. So, if I'm going to want them, I'm going to have to make them myself. And I mean, that's crazy. That's how many lines there are. Remember those numbers from earlier? There's 2,000 of those. There's no way I'm writing a name for every single one of those, right? It's going to take forever.

So, anyway, I did. Here's the code. Uh, this is part of it. At least it took a while. Uh, and this got this stuff got so esoteric that I was quite literally dealing with magic. [snorts] Okay, but now I could put these back through Leaf and see what Ember thinks the most important stuff is. Drum roll, please. It's the time stamp. It thinks the time stamp is the most important. Wow. But this is an interesting result. Anyway, it shows us me exactly what the model is thinking when it reads my file. And there's some interesting things in here. Here's four specific features I pulled out that might look a little promising for I'm trying. What's special

about these is that they're all things that I can change without actually affecting how the file runs. timestamp doesn't matter. Uh certificate table size doesn't actually matter if I don't have a certificate. Uh debug, I'm not going to be doing any debugging. I know my code works. And uh major subsystem version. Fun fact, Ember actually thinks that Windows 7 files are safer than Windows 10 files. This is something I can definitely take advantage of because these are features that Ember rates really highly, but don't actually do anything when I change them. So before we get crazy though, let's look at what Ember actually actually gives us. This first one up here is what Ember thinks about a completely normal

file. And [snorts] the bottom one is Mimi Cats, which is a very well-known malware sample. You can see every little detail that Ember thinks about. Uh lots of blue at the top here and lots of red at the bottom. Uh you can see that it actually really likes that both of them have certificate tables. The end evaluation for these files is about what you would expect. One's safe and one definitely isn't. Now, let's look at Mimi Cats and let's see if we can reduce this number any. First, I don't plan on using the debugger. So, let's increase that to the 32-bit max. Okay, I'm just going to spend 30 minutes here trying to figure out where that is in hex. And um

there we go. And okay, that's that's a pretty good reduction there. Uh this happened because when Ember was trained, it saw that files with larger debug tables tended to be safer, I guess. uh even though it didn't really have anything to do with what the file actually does. So changing it still looks good to Ember. Let's see what else we can change. All right. Now, let's go for change this to be a Windows 8 program instead of Windows 10. And all right, wow, that was pretty effective. Now, keep in mind, I haven't actually changed how the file works. I can still run this right now and it would still break my computer. Trust me, it did. Uh,

but surely no one's risking their entire business on a 25% chance, right? Anyway, um, so I'm like a computer programmer or something. Why am I doing all the work? I'm supposed to get the computer to do all the work for me. And I know just the technique. Particle swarm optimization is a very cool algorithm that lets you pick the best values for when you have a lot of features, which I do. You can think of it like a flock of birds. All the birds act independently, but they all work together to find the best combination of inputs to get the best output. Now, I want you guys to think for a second how that GIF would have looked in

2,300 dimensions. You should see your faces. That's what you look like. Uh anyway, I did it. Here's the code for it. Uh it's actually pretty simple algorithm. It just takes a lot of fine tuning, especially when you have a bunch of different values like this. Also, I made a uh cool UI for it. You can change the sliders and stuff. I think it's pretty cool. Um, anyway, how'd it go? Uh, well, I'd say it did pretty good. It reduced the score by quite a bit. Um, I have now fully convinced Ember that my dubious file is completely safe with an alleged .003% chance that the file is malware, which according to chat GPT is the same

likelihood as being killed by a falling coconut. >> [laughter] >> Now, keep in mind, I only did this just by editing these seven features. Uh, handpicked from a list of over 2,000 of them. I didn't I when I first started this, I didn't actually know what half of these did. So, I just kind of picked the ones that looked like they were the easiest to change and and and I found those. [snorts] Um, and so those are the numbers. Apparently, Windows 7 still the safest one. And the best time stamp was Friday, September 17th, 2010 at 8:00 p.m. 3 days after Halo Reach came out. Now, I designed this to work with Ember. My algorithm optimized my new file to

give the best values for Ember and give the best out output. Uh, and it could in theory be optimized for other metrics, but I didn't do that here. Now, that being said, this was the result when I put it into virus total. Keep in mind, I didn't I didn't optimize it for virus total, but that's still about a 20 25% reduction there just as a side effect. Uh, and I did this with about a dozen other malware samples. Um, then they got pretty much the same result. Um, and there's plenty of other ways I can improve this process, too. Uh these are some of the most popular AI tools available for cyber defense that I could

find. Uh and lots of them do a similar thing to Ember, but some of them do other things, but they all will have the same kind of vulnerability if they use the same same AI to to decide these things. Uh and so I'm I'm sure if I did this for many of these other ones, I could get a similar result. Now, there's plenty of ways to fix this issue. For one, don't give useless information to your model. For example, if I'm making a model to predict how long it'll take a pizza guy to reach my house, I don't exactly need to tell him what kind of music he listens to, unless he's a Metallica fan.

Uh another idea that came up while I was working on this uh was to use something called control flow analysis which is basically instead of looking at the properties of a file, I would instead look at the actual logic of the the program, the way that the logic would flow and then feed that into the AI. And I I expect that this would have a lot less of those kind of vulnerabilities since it would be looking at more important things like how what it actually does, you know. Um, but I never got to uh to work on this while I was working at Dropbox because of time. But uh if any of you want to see this at

your company, you know, I'm I'm looking. So uh but so why is this why does this matter? The entire world is in a big AI craze right now. Every company wants to put AI into everything because it looks good for their uh their investors. Uh, sure AI companies can make those annoying AI assistants on on their websites all they want and it doesn't really have that much effect but when it comes to security and protecting the most important things we need to have a lot more scrutiny towards the things that get implemented. I made this job I made this project on the first ever job that I worked on. I am I'm an entry- level person so I I don't have much

experience with this and I was still able to get this kind of result. So imagine what some guy in Russia who's been hacking since he was 10 would be able to do in a couple more years if he had a if he had that time to work on it. [snorts] Thanks for coming to my TED talk. I'm Noah Groch. Uh if again if you like what you saw uh I'm looking for a for a place to work at. So you know uh here's my LinkedIn if you guys want to scan it and connect. Thank you. And uh uh if you guys have questions you can find me. They all be around here. Uh but yeah, thank you