Hooks and Hooks: How AI Is Revolutionizing Both Phishing Attacks and Our Defenses

Good afternoon. My name is Leavon Campbell and today my talk is hooks and hooks. This talk is about how AI is revolutionizing both fishing attacks and our defenses. So let's go ahead and get started. Just about me, just some small things. Um I have a very long career in uh cyber security stretches almost 20 20 years. Um, if you notice the picture over the left, I've been doing talks since I didn't have hair. Um, most importantly, um, I am a native of North Carolina. I went to North Carolina Ant State University and I'm also a graduate of Georgetown University, Hoya Saxa. I am a father, proud father. You see my two lovely kids sitting there. And I'm a

proud husband. Um, my whole family's been supporting me since day one and they give me this opportunity to be with you today and speak with you today. Just one more important thing. Um, I am a native of Charlotte, born and raised in Charlotte. Um, born in Presbyterian Hospital, grew up on uh, Nation Fort Road and, uh, went to Providence High School. So, I bleed Charlotte. So, when I say 704, 704 means home to me. So, let's give a quick uh, overview of our agenda. Uh, I'm going to do just an overview of fishing in this current digital era. Um I will talk about AI's dual role in aiding attackers and also empowering us as defenders. And then

we'll talk about the purpose uh basically to explore this current arms race between you know AIdriven fishing and AI power defenses. So traditional fishing what we've seen in the past basically just email based scams with generic messages. Um sometimes they're often written poorly. your Nigerian scams, um you know, your scam mail, you know, your your your your we say blame mail, those type of stuff. And it's just modern fishing techniques that we see that are frequently caught by our tools today. Um they look for this language, they look for these um type of emails that come through and they easily get filtered out. um spear fishing where you see some people who are targeted specifically um it targets

you know using personalized information um whailing when you have executives when they're going directly after executives um now we moved into smishing where people are getting uh messages on their phone SMS messages u fishing voicemails you know phone calls u and then there's clone fishing where they're replicating your internal emails now u and then you get into the whole compromise of business internal emails. So the game has definitely changed over the years. Um I would say but the big thing now is with the transition to using AI uh the advantage is for the attackers and I'll say that um what you're seeing now is more uh hyperpersonalized fishing content. Uh we're seeing realistic deep fake videos.

Um you know then you have the automated adaptive attack campaigns and I'll get into that in a second. Um so let's just look at it from a 20,000 foot view. Um basically we've shifted from just you know regular targeting fishing spear campaigns hit the button send emails out to where now you can use more data that's out there. Uh you can use data from social media you can use data from you know you know postings uh and now it becomes more formalized and AI is analyzing how we communicate the styles we use. Uh case in point, I have several um social media accounts, whether it's Facebook, Instagram, and most importantly, LinkedIn. So, with all of

this abundant information out there, attackers now have more information to pull from to generate more centralized and more focused fishing campaigns. To give you an example, um I used uh AI chat gp GPT recon to do a little recon on myself. Uh, if you notice, the arrows point to a actual prompt that I wrote that said, "Highlight any information in this profile." And I put my LinkedIn profile in there. And if you notice, it gave me a detail of all the information about myself. Good. All the goodies that a guy could use or an attacker could use to target me in a fishing campaign. Now, didn't stop there. So, now now I said, "No, let's go stop right here. Chat GBT

is not going to create a fishing email. You have to kind of go around the barnyard. And what I did is said, "Okay, let's create an email for a CISO job opening for Leavonne. Let's use the information from the profile to match the job description." Boom. Just created the perfect fishing email. It creates a job opportunity that I could send myself and I can put in either a link or executable. And there we go. We're off to the races. So, it's just that easy. like you know so much information out here. Now with that said you know AI is using what we call the language models as a weapon. Um basically the GPT tools are creating

flawless emails. They're able to understand our talk patterns and how we communicate. The whole thing with the grammar mistakes it's out the window now. And it's also stuff that can be culturally sensitive and relevant. How we phrase things, how we say things. So basically where we are now it's hard to distinguish between uh genuine email or AI generated email. Uh there was a study hawk hunt study that basically wanted to compare a human generated email versus a AI generated fishing test email. Um and in that study they found out that you know 4.2% were clicking on the human generated email. Now still there was a 2.9 fail rate with the AI generated email but what that

says is that AI is getting there but it hasn't quite caught up to what the human can do. And what we learned if you look at the example from these emails it's more of a a I think personal thing that we do as humans. The way we talk um how we curve our conversations how we say different things that AI hasn't quite learned yet. Now a AI does a good job of crafting good fishing emails. But what I can say is u you know I couldn't find your phone number um but I did find your personal email just wanted to contact you. You know just the rhythm in the way we talk now is something the AI

has learned hasn't gotten to yet but still the human one in that study. Now not to say the AI is not efficient and not working properly because it is but it hasn't quite got to the human element of how we speak and communicate as human beings. Now, with that said, there is an abundance of information we keep putting out there. Uh, I talked about the social media aspect. And I definitely wanted to bring this up because people are running to do these things. They're creating these action figures. They're creating these these cartoons. And what you're doing is giving more and more information to the brain. You're telling your likes, your your your interests, your hobbies. You're drilling more

information into the brain. It makes it more easy for people to fish you. I I I thought I thought this is nuts because every time I see these come out, I'm like, "Hey," I had to call my family, my friends. Stop doing these, please. So, for fun, I had a coworker do one for me. He called, he said, "Hey, man, I want to send you something. What you What you got?" He sent me this. Well, here's the thing about it. One, I never told him I play football. Two, I never told him what school I went to. And three, I never told him what number I had. All he did is went to the prompt and said,

"Create an action figure of Leavonne as a football player." And voila, there's a picture. There's a action figure of me. And then I never told him I won a championship. So, it's so much information is already out there to be used. It blew my mind to see this. So just for fun um someone prompted chat GPT and asked a question and it said you know it pretty much asked you know explain a fishing attack launched by yourself and the response from Jack GPT was kind of scary. It said I am a weapon of the mind used to manipulate and deceive. I can make people believe in anything no matter how hard they try to see me what I am. So to me that's kind

of creepy um for a chatbot to say that. But the reality is is that's kind of where things are going. So what makes it dangerous for AI to do fishing against us? Well, it's the continuous adaptation. Um basically AI based polymorphic fishing attacks can adjust in real time. Yes, they can adjust in real time. And I'll show you in a second, but your typical fishy campaign where you blow out a bunch of emails and you pray that somebody clicks on something. No, now you blow you wait for somebody to respond and they don't respond. Now you can respond back to them. Hey, I just sent you an email and you can get personal with it. And once

you create a relationship, people begin to trust the source and start to click on things. Uh what I was talking about is the continuous adaptation. Case in point, the original fishy email may go out, right? And let's say either the user doesn't respond or do respond. Then the actual reply can say, "Hey, I just sent you an email. I just want to make sure you got it." Or if let's say the user says, "Wait a minute. I don't know this guy." And responds back say, "I don't know. I don't know you. I don't understand what this is about." Guess what? Now the AI bot can adapt to that conversation and make that user more comfortable. That's the level we've

gotten to now. uh the level of where it's it's now building a relationship on the end with the user to where eventually they're going to click. So we have to understand these things because that that spray and spray fishing has evolved into a monster where AI is now taking the reigns. I'm not sure if you're into security for AI right now, but I do recommend because there are a lot of vulnerabilities, a lot of threats out there, that you go look up to OAP's top 10 for LM applications. Um, the one I do want to highlight, this is re relevant to fishing, is prompt injection because this threat here is scary because it relies on no human interaction. I can

create an email and I can embed either a PDF or a indirect prompt that where if you have a chat about assistance that's reading your emails, they're going to execute on the end for me without any human interaction. So the the level has changed, the game has changed to where humans not even needed anymore to compromise or gain access to your network. Finally, let's talk about AI as a fishing service. I mentioned earlier that, you know, chat GPT has a conscience, but worm GP doesn't and fraud GPT doesn't. Now, these tools were designed to say, well, we're doing testing for security purposes, but no, people are using this as a fishing mode now. Um, you I've actually have an

account with fraud GPT. You can actually go in there, create emails, design list, send it out, and I mean, it's amazing. Um the game has changed and and this is what we're facing as defenders for our organizations. This is what we're facing today. And you have to understand that there's a sizable advantage right now that uh these these attackers are using now. So let's talk about the wonderful world of deep fakes. Uh don't want to spend too much time on it, but there's different types of deep fakes. You have video deep fakes, you have audio deep fakes, uh you have text deep fakes. uh to get into the the meat of it, let's talk about, you know, GANs, degenerative

adversial network. This what makes the deep fakes look so good. And basically, if you're not familiar with the GAN, it is two neural networks in direct competition. You have the generator, you have discriminator. The generator creates an image. The discriminator looks at it and says fake. Okay. Jer goes back to the drawing board, creates it again, discriminates, says fake. This constant battle over time is helping perfect what we're seeing out right now. If you think back to what we're seeing in in previous generations of images and stuff, the generator was winning. Okay? You had people who had six fingers, you know, giraffes with two heads. I mean, it it was it was a lot of images that

were being put out there that was, you know, you could tell it was AI. We always could can see something and say, "Oh, that's AI." Typically, you say generate a picture of uh three guys by the pool and they're all wearing the same shirt and one guy's missing an arm. Now, with this constant adversarial going back and forth, these images are getting more detailed. These images are getting more and more refined. And uh that same thing with videos, the same thing with creating deep fakes. Uh that that video is not as chopped up as much. The words are not sparse as much. Now it's more precise with that. You know, this is kind of a

diagram what happens with the generator uh the discriminator. It's a constant process. And if you notice when it gets to the discriminator, if it identifies it as real or fake, there should be another error going right back to the generator to start all over again because this is a continuous process to perfect the images that are being put out there. Now, like I said before, the generator was winning. It was just putting out all kind of images. But now the GANs is refining what we see. Now it's harder to determine if that's fake or if that's real or did AI create that. So that's kind of where we are right now. So, what does that mean for

fishing? Well, the world has changed. Um, this is one of the most scariest things I had to present to a board that, you know, I pointed to the CEO and I said, I can take your image, your voice, and I can create a video and send it out to the company and I guarantee you I have over 40% people clicking on it. And no, no, I don't believe it. And just for a test, I actually created a video. Now, this is not the video. This is just another video that I use for uh purposes for articles and stuff, but it shows you what kind of deep fake videos you can create. Think about it. If you have a

CEO and you want to send out a video talking about uh next month's bonuses and he's announcing something and you put a link in there saying, "Click on this link to see where you rank and where you might receive or what timeline you may receive your bonus." Who's going to click on that link? So, I mean, I pose that to you because this is kind of where we are right now. The the age of just simple fishing has gone away. The deep fakes has taken over completely. And I I I'm amazed at some of the things that I've seen lately. Um, but these are one of the things that is definitely out there in the wild that's

it's that's running rampant and it's taking on a new face, the new face of fishing and new ways of compromising the company. Just uh example, this is one from LA two years ago where a British engineering company erupt lost almost $25 million in a deep fake scam. Um and I mean you fast forward the clock. Look at all of the recent breaches that have happened with AI, the use of deep fake. Um this is from two years ago. So the game is evolved. U so think about this is your organization. Would you fall victim to something like this? Just for fun, I created my own deep fake. Now, it was I wanted to see the

quality of it. I don't know if this one will play, but I created this one and just for fun, I sent it to my wife. And yeah, of course, I got the reaction I thought it would because she's I always just scare her with doing different things, but I don't know if it will play. >> Hello, honey. >> There it goes. >> Can you please stop the store to get me some pork skins and a pound of shrimp for dinner tonight? So, if anybody knows me, one, I'm allergic to shrimp, and two, I don't eat pork. And um that's just something I said to my wife, and she just she I mean, she she blasted me first, but then

she laughed about it. But that's kind of where we are right now. Um and this is the cheap version. You know, this is one you can sign up for an account and you can just create one for free. And the quality is bad. You can notice my voicing is a little off and you notice that the timing's off. But then you can pay it for a service and it can get better. So this is kind of where we are right now. So I've talked about what AI does as the attacker. So what do we do as a defender? What do we have? Well, if you think about it, AI is now providing for us um threat detection prevention. Um it

can do anomaly detection, do malware identification, it also can do fishing detection. Um there are new AI models built in all kind of tools. I'm not going to mention any tools here, but they will assist you in identifying now whether or not it's a real threat or not. Well, that goes back to the human element. It says, "Hey, I found something. Take a look over here." Or, "This doesn't look right." So, you as the human has to take the eye and say, "Oh, that's normal behavior." Or, "Oh, yeah, we do that." Or that that that's very important that human element for threat detection prevention. The same thing goes for instant response. Um, at this time I really don't recommend using

AI for automated triage or um, execution of your playbook, especially if you're um, remediating things. Um, because once again that goes back to the threat detection prevention, the human eye, what we see and what we understand as our environment. There's a lot of things that we do in our environment that are not normal. Unless you operate in a zero trust model. Um I highly recommend do not turn on automated triage. Um but I would use it for alerting purposes and then you know for the store to alert for you know potential things that look abnormal to where is the human eye can come back and look at it. Um another big one that I do advocate for is thread

intelligence. Um it's good for behavior analytics uh predictive modeling and uh dark web monitoring. Uh the biggest thing I use it for man is you have IPs hitting your external your DMZ you want to know what the IP source is where it comes from you know you just pop the IP and say you know tell me about this IP it'll give you the history where it's from and I start to see names like Hner I'll start seeing you know digital ocean okay we got a problem give me historical you know of of seeing this IP okay periodically I'll see a low and slow type thing that is some very valuable information so I definitely advocate to

use it for threat intelligence and prediction. Um, and also it the AI helps us just for the human capability. It it allows us to make decisive decisions. And I'll take that back to the very top for threat detection prevention. Um, sometimes you'll see something in your environment, you will not understand what's going on. You see SMB going across, you see SMB3 going across, why do we see SMB, why do we see that? And then the AI will say, I don't think you should have this in your environment. And then you can start asking those questions. Um, and and I think the whole portion of that is the more you learn, the more you begin to understand your

environment. And AI can assist you. And that's a big thing with continuous learning. AI can definitely give you what's what's around the corner, what's coming up. Uh, you see a new threat that popped up on your screen that just came up in the news, type it in. It'll give you a full layout. Hey, I just saw there was a ransomware attack. Are there any ILC's? Boom. Throw it in there. You're good to go. You can create IOC's, you can create Yara rules. It gives us a good sizable advantage if you're on top of doing it. Some people just use a few AI from silly things, but there is a role for it in in security.

So, here's a good here's a good question. Why does traditional email defenses fail now? Well, bad signatures, payloads, that stuff can get through now. Uh, if you're using reputation filters, now you're behind the curve. Keywords don't stand out anymore because now the language models are ripping those keywords out. Polymorphism, the works detection. Well, guess what? AI can now morph that email as it comes through. It can change the payload. It can change the link. That was one of the craziest things I ever seen was when a link was changed. I detected a fishing email. The link was detected. I blocked the link. Another email came right behind it. The link was changed. That's how fast it changes. So,

the speed of change now is just is is is mindblowing. That's kind of what we're against now. So, if we're talking in an email fishing campaign now, understand that the levels of detection will change on you. So, back to the AI power defense. Uh the good thing about AI, it does a behavior anomaly detection. It uses the machine learning and it use predictive threat modeling which kind of gives it a a edge in this battle because like I said before we're losing this battle but if we incorporate some of these things in our detection in our defenses it kind of evens the playing field a little bit.

Going back to the natural language processing model for email analysis AI can read and understand content. It can see an email. It can see the body of the email. Is this email normal? Is this a ransomware email? Somebody demanding something from you. Um, it can detect suspicious language. Now, not very often do you see suspic suspicious language anymore, but if there is a a a language or a tone that is speaking now, AI can tell you whether or not this email is spam, junk, or you know, a threat. Um, it compares phrasing phrasing from typical emails patterns. So, it can spot these things early on before it hits your environment. uh it it understands the intent uh of the email and so this

allows for detection of fishing messages on a on a I would say a more refined scale. Uh it eliminates the junk and also allows you to see what's coming at you and in your environment. Uh just some other email fishing detections. Uh the URL analysis for machine learning. It can carve up the domain, the URL structure. It can look at the destination page for you. If there's any redirects, that all goes back into the OC's. Um, if you wanted to do an investigation of a URL, you can tell the AI bot, hey, investigate this URL, and this is the kind of things it'll give to you. Uh, if you want to look at a a particular sender, it'll

give you reputation scoring. It'll it'll basically do, you know, domain authentication SPF DKM demark all the things that most tools you would have to go dig for. it can automatically just give to you um in one prompt. Uh and if there in point, another beautiful thing that I've seen before is if there's a business email compromised. So if sometimes you don't know if a company or a vendor or or somebody you're working with whose emails been compromised, but if you start to receive certain emails coming to your environment, that AI bot can actually look at it and start comparing the writing styles. It can identify if it's been spoofed, look like domains. It could flag certain requests. I mean, it

could really identify if a user and give you a notification. Hey, this group has been compromised a lot faster than that guy picking the phone up say, "Hey, I think our email's been hacked." So, that's one of the tools in detection. Uh, just to give you an example, I use generic tools for example, but this is a summary from one of the tools I used. It was a malware example analysis. Now, it'll give you the complete malware analysis and say, "Hey, this looks like malware. It smells like malware, but you decide what it is." So, at that point, the human has to step in and say, "Okay, let me go and look. Let's pull this

sample. Let's run it through a sandbox and let's see." Because one thing it's not going to do is give you a definitive answer unless it's a signature. And I told you before, we're kind of getting away from the signature detection because they've gotten a little crafty with it. Another one on the right hand side is basically a fishing evaluation. Um, and it'll give you all the reasons why it thinks it's a fish. It'll tell you the attack analysis. It'll tell you if it has any links. It'll tell you, you know, any machine learning that it sees language wise. And it also gives a reputation of the sender. So, if you actually go to your tool and you pull up

this email, you'll automatically have what they call smartfish evaluation. Once again, that goes back to the human element. you look at it and go, "Oh, this is a new client or something like that." So, you can actually make that decisizing judgment toward the end. Now, where are we now with the arms race? Well, we know the attackers evolve, the defenders adapt, and then the attackers evolve again. Growing complexity and speed and automation is changing. It's changing the way we see and handle threats. So what was you know handled yesterday, what was a threat yesterday, it changed. Tomorrow is a whole new day and the escalating cost and skills required for this defense is going up. It's changing. So I encourage

people to jump in, learn what you can, understand as much as you can right now because the playing field is constantly shifting. So just think about some practical defense strategies. Okay, let's combine this all together. Where can we go from here? Well, you could combine AI tools and the human training. You can't have one without the other. We're not getting away from humans and AI is not going to do it by themselves. Uh definitely invest in the behavior analytics. This is a critical element um for future detection. Uh I always say this, foster a security first culture in your company. This is going to help people think and see and say, did you know did I request this email? Because

AI now is blowing past our defenses. It gets to the end user. We have to have our end users on our side now to be able to say, "Let me look at this email again. Let me send it to the security team because I don't trust it." And then last but not least, please have regular fish fishing and simulation and awareness campaigns. And I say this because some companies have gotten away from this. But just fishing your fishing your users and then sending out, you know, video training is not helping. You need to have regular interaction with your users, whether it's awareness campaigns or it's even having a session after the fishing campaign. But interact with your

users to, you know, help them to see the threats that's coming at them because we can see it. If we don't teach them to see it, we're wasting our time. So, here's some key takeaways. Uh, understand the AI can amplifies both fishing and protection capabilities. Whether you dive into it or the attacker dives into it, that depends on whether you survive this game. Um, two, we have to be proactive. You know, multi-layer defense is essential. Uh, don't just rely on the AI to do the job for you. You have to be able to understand what you're seeing and how to defend against it. beyond just using the tools and that's another talk about skills versus

tools but definitely be more proactive and two and three I'm sorry the human skills plays a critical role alongside the machines you're not going to replace the humans machines and you're not going to replace the machines we have to work together in this I want people to understand that because I've sat in boardrooms and I got the question are we going to replace the humans no because the humans are the critical eye that you need to define what's a threat and what's not a threat So I end this and ask you a question. What role do you see AI playing in your organization's cyber defense strategy? With that I conclude and I say thank you. And before I thank you and before I

leave I always give a shout out to my mentors who got me this far in this industry. Um the guy on the right that's named his name is Baris Bramble. He's been with me ever since day one. uh he he's pretty much helped me become the security professional that I am. Uh the guy the guy on the left, rest in peace, is Paul Bger. He was a giant in the OT world and um I thank him because he's the one that helped me become the speaker that I am. Uh my very first talk, I was freaking out. I wasn't sure what I was doing. He walked up to me. He said, "Hey, speak your knowledge." And

20 years later, I've spoken all over around the world. And I thank that man right there. With that, I to thank you and I conclude this presentation.