Deepfakes & Dark Side AI Attacks

uh really excited to be here besides gallway uh usually I always like to ask who's you know attending for the first time but I would hope everybody would be raising their hand for the first for uh for that question so um this digital doppelganger looking at Deep fakes uh usually when this gets done there's somebody who always introduces me and uh you know introductions vary some people read your bio other times they just kind of make some make some things up I figure if I'm doing deep fakes I'm going to make my own good afternoon my name is Brian H and I'm very excited to welcome you all to be sideway additionally I'm glad to

have James Mig my good friend and cyber security colleague here today as he shares his presentation about synthetic Media or deep bat to all of you James is going to share insights and best practices to so sit back relax enjoy James presentation he probably thanks Brian um I think I've met Brian once in my life I have a couple colleagues who are friends with him so I texted them last night I said you think Brian wouldd be upset if I deep faked him he goes no he's a good guy go ahead and I'm like yeah that's scary in itself thinking no we'll see what happens and you know there was a glimmer of hope that oh maybe Brian I'll

actually show up but no I'll have to go back to Dublin for that one so synthetic Media or deep fakes we're generally recognizing them their social engineering our users family friends to attack organizations steal data money as we heard uh this morning from Bob with regards to why cyber criminals operate why do they do what they do for me the answer is simple they want to make money uh plain and simple uh and so they're going to do anything and everything they can to be able to get that money get that or even that ego uh that goes along with it now when we start looking at synthetic media and and as you can tell

from I'm from the US uh this is a global issue this isn't just stuff that happens in the US uh here we have an artificial synthetic image that was created with uh Bono and Bob gof holding Israeli Flags sewing their support for Israel during the the whole conflict that's been going on this has been artificially generated again with cyber criminals when it comes to social engineering all that comes down to getting people to respond have a knee-jerk reaction and emotions playing a huge part of it some folks may remember or maybe you were there with the Halloween uh issue where there was supposed to be this big huge party parade downtown and it was all fake uh

and I thought that was just it blew my mind it's like oh my gosh so many people went down for this all because of a ad from Spirit Halloween which ironically is an American company in ill the state of Illinois um but they had an ad and it kicked through and it just blew up and people you know responded to that um another Irishman Pierce Brazen you know if my wife wasn't married to me she'd probably tried to be married to him uh but he was the victim in this case where unfortunately you had a art curator who owned a uh an a art gallery uh heard from Pierce and his agent saying hey we'd like to have

Pierce's artwork displayed in your Galler she did a video call with Pierce and this is for the screenshot from it uh she basically was selling the tickets for 500 British pounds each generated about 20,000 lb uh and when she went public with it and started advertising it really uh on a bigger scale Pierce's real actual agent got wind of it they reached out and said yeah no this is not us you've been you know unfortunately scammed uh and she was out about 20,000 EUR uh uh British pounds because of it so we're seeing this happening all around so the question now becomes how do we be able to defend ourselves against synthetic media again for our friends our family

our users and within in our organization all right so my name is James mquan I'm a security awareness Advocate with no essentially my that responsibility allows me to go around to events like these conferences uh provide thought leadership in all aspects of cyber security social engineering uh dealing with fishing and and all that other good human risk management uh I spent I've been with no before now about 5 years prior to that I spent about 18 years working for little German company called zans I had the pleasure of coming over to Germany for a lot of those trips uh I just spent the last eight years being the chapter president for the isc2 central Florida Chapter uh so I

certainly respect and understand what Tom and Scott are going through putting on this event for all of us uh yeah so no before real quick this is the fun marketing slide like what Bob said marketing gives you the slide so yay there's no before we do security awareness training fishing simulations for me one of the the the biggest things is what our mission is is the fact of helping organizations deal with social engineering for me and what has been my personal mantra for the last 15 to 20 years is the fact that I want to help people Empower themselves and Empower others to make those smarter security decisions every day because we're all the we're the Smart Ones

already you know we're we're seeing the different types of attacks and threats and everything going on and we know what to look for in the those fishing emails and when we have those different responses but our family our friends the older Generations not so much overall and so we want to make sure that they're educated they're made aware now essentially what my job is is to go around and talk with people about the importance of securing their dried grapes yeah it's raising security awareness they don't get any better just letting you know now all right so the outcomes for the next 173 minutes and I got about 375 slides yeah that's that usually makes the heads pop up because

they're like whoa hey this was 45 minutes it is um because I know I stand between you all and lunch so I we'll get through this as quick as we can um we know synthetic media it's quick and it's easy and it's getting easier to use basically you know we look back uh I think back to the Jordan Peele deep fake video that he made of President Obama back in 201 15 16 you know that took a lot of Manpower a lot of computers some sophisticated software nowadays you can go on to website upload some images upload some audio and the way ago and you're off and running like you saw that he did with Brian that video of Brian uh

I'm going to have him explain it later on how he did it or how I did it uh but basically it took me about well it took me a little longer because I had slower internet at the hotel last night uh it took about a half an hour to make that video from start to realizing Brian all the way through to the end I originally was going to do the Prime Minister but it the site blocked me because it recognized him as a celebrity so we're going to look at understanding media and how to spot it and then what we can do to prend to defend and protect ourselves against the synthetic media now when we look at the AI playbook for

cyber criminals you know cyber uh synthetic media and identities is up at the top you know zero click rag exploits asy attacks hallucinations polymorphic automated attacks password cracking data poisoning prompt injection malicious AI large language models AI spear fishing looking at agentic AI attacks as we started hear from Bob earlier today and Shadow AI I could do a whole day and cover all these topics and still not get through with it so we're just going to focus on synthetic media cuz that's kind of the big one uh where we're seeing a lot of it already and essentially using artificial intelligence the aspect of large language models natural natural language processing and Gan models gen uh generative adversarial networks it's

basically pitting AI against Ai and what's happening is you have a generator that's going through and creating that video content that image that audio and then you have a discrimin discriminator i AI That's going and trying to determine if it's authentic or not it's the the generator is trying to fool the the other AI in creating the the fake or synthetic media overall so when we look at synthetic text we've already been using synthetic text that's chat gbt those large language models whether it's Gemini grock you know Claud whatever which one you're using you know an AI is learning based off of those models based off the information that gets loaded now for me one of the ear on when this all

kicked off uh in May of 2022 now it's almost 3 years ago uh one of my favorite stories with relating to the downside the biases of it or the hallucinations of it of chat gbt was two lawyers in New York City decided to use chat GPT to look up information relating to a case they were working on cool all right well they asked chat GPT about any other cases similar to this one and gave it the information Chad GPT comes back and goes yep here's six more the pargal were all excited they threw it in the brief they ran it on over to the courthouse and said look there's been six other cases supporting our claim and the judge

looked at it and said yeah these do not exist where did you get this information so they got sanctioned they got their hand slapped now I've been using this for the last couple years was kind of one of my favorite stories and was starting to think the last couple months okay this story's getting a little old I think everybody's heard it now all right well then in Australia somebody did the same thing thing again um and this was last month so it's people are still using chat gbt still falling victim to the hallucinations that we deal with it but cyber criminals are using it to level up I know Bob talked about it in his presentation

where cyber criminals now can write a fishing or spear fishing email in any language and for years the estonians the Japanese they weren't targeted very much because the Cyber criminals out there had a hard time with the language well that all goes away now you can write any pretty well uh any anything on in chat gbt and it'll translate to whatever address um language you want we're seeing a lot more attacks originating from compromised accounts they're not sending out links right away they're building Rapport they're building that trust and then targeting you after that and we're seeing a lot more of the fishing uh as a Services capabilities are providing some type of AI capability

in there we're also seeing we've known about malicious large language models certainly right after chat GPT took off I remember the summer of 2023 uh playing around with warm GPT uh before they ended up shutting down because their product kept getting scammed by other scammers and cyber criminals I mean who to thk no honor among these but we're seeing uh a variety of different large language models available for them to be able to use to write malicious code uh they're not leveraging it for writing fishing emails that went away very quickly they're using it more now to create codes for themselves we see synthetic images generating of images I mean who doesn't love going into dolly or mid

Jour mind Journey or using static Fusion to generate images but what if you could use it for social engineering attacks and scam somebody out of about $850,000 you probably think yeah no I wouldn't who would fall for that God bless the French woman that did because she fell victim to it um she got reached she was reached by Brad Pitt's mom basically saying hey Brad's really you know has come across your profile really likes you and sends a picture then it's Brad Pit reaching out and he's been in hospital and he sending uh images to her and asking for money why Brad Pit asks for money I don't know but uh conned out of 850 all through

synthetic images and some of them you know you look at them and go oh my God that's really really bad and some of them are like that's pretty good I mean they're getting better and better uh striking a little closer to home in the us when we had the the fires out in California and Los Angeles this spread around um the Hollywood image What cracks me up and this this got shared a lot What cracks me up is the fact that they spel it spelt Hollywood with 2DS I mean it's very again a very emotional very reaction uh based Type image that you would have here but you know come on folks so then now we start getting into

synthetic audio looking at how we can leverage audio and one of the big things last uh early last year uh was the whole Biden rooc call um which you know someone got a hold of Biden's audio surprise surprise like there's it's very easy to get um and B and created a uh robocall using uh 11 Labs uploaded the audio you only need about 30 seconds of audio for it to clone The Voice you can go up to about 5 minutes and use the more professional version but just right out of the gate you can use um 11 labs and 30 seconds and Away you go and there's a variety of different tools as well and I'm thinking okay great all

right that was last year people are getting smarter no unfortunately Thailand's prime minister was targeted by an AI phone scam as well so there's they're not picking on one nation they're going around to all of them that they can uh some other examples we had a senior principal analyst over at last pass get targeted through uh text messages and then an audio call from someone pretending to be the CEO a little scarier you had somebody at Pao Alto where they got a phone call from somebody saying that his daughter was in a car accident with him uh she's really shaken up uh and now they she's she owes money to you know pay for the tow truck

and and the damages and everything else while that's going on he's text the AL the Pao Alto employee is texting his wife asking hey can you check on our daughter cuz she's off of college make sure she's okay and sure enough she was at the college this was a deep fake that they were trying to scam him out of so uh he was going out of band and verifying that everything was okay now I don't know if anybody's had the wonderful experience yet of calling up uh like Hospitality change where you call up and you get a human right away it's none of the you know press one for gaic two for or one press two for gaic

and and off you go uh nowadays they're using large language models and essentially the call center software is synthesizing or generating audio and having you can have a conversation with them and they'll direct you to wherever you need to go whether you want to talk to somebody your reservations or you've got a question about your latest hotel bill this essentially call Center Software or answering software we kind of flipped on its head a colleague of mine Perry Carpenter did it and then I kind of followed him on it but made my own and I you give it a prompt you give it a voice and so I snagged one from play HT and the voice that I used is a good friend

of mine Jerry oer or Dr Gerald oer he runs this he every day from for you all it would be from 2 to 3 uh or sorry from 1 to 2 uh here in Ireland he does a daily cyber threat brief and uh I had plenty of his audio and I asked him hey I'm going to snag your audio and what I was able to do was have him call me up and have this conversation I'm going to play it for you just so you can hear what it sounds like hello hey it's me I need your help I've been in a car accident and now I'm being help by the police I don't have my

wallet on me and I need to take the tow truck and start the repairs on the car can you please send $500 c c cost I know you have a crypto wallet so it would be really quick and easy for you to send the money that way I also need an additional $1,500 slow down what's going on I'm sorry just really shaking up I was in a car accident and now I'm being H by the police I need your help to cover the cost tow truck and start car I'm sorry [Music] how F the car car is pretty banged up I need to get it own and start the repairs as soon as

possible I know you have crypto wallet

so where do I send money to you have a crypto wallet I give the money urgently I can provide you with the wallet address and you can send the money as soon as possible thank you so much for helping me out in this difficult situation okay all right slow I understand thank you for your help the crypto I'm really sorry I can't provide that information send you the money I apologize for the confusion but I can't provide that information so essentially I goofed on the prompt and I should have put the crypto wallet ID in there and then I probably would have been able to give it to me now this is a bit tongue and-

cheek especially with the him repeating it and but you re you can uh appreciate the fact that I'm interrupting then he comes back there is a bit of delay but that has improved uh significantly more uh I played around with it for a couple of hours and you know I had that uh I have a my colleague Perry basically made it so that he had a conversation with somebody that was holding his kid Ransom so that was kind of freaky uh on the flip side uh this was over this was done by O2 and I love this and I like to show it but basically they're using an AI voice a similar thing to stop the spammers I'm your

worst nightmare I'm an AI created by O2 the waste phone scan of time so W's then a DOT Tre and then I think ising people right I'm just trying us to have a little

chat a picture of my cat fluffy it's showing you a picture of your cat fluffy stop calling St while they're really talking to me they can't be scamming you and let's face it dear I've got all the time in the world so essentially they're uh intercepting the scammer calls coming into recognize phone numbers of people that are a little more susceptible to Falling victim to these type of phone call scams and so forth uh and so they have Daisy that that helps interrupt that so kind of using it for good in that case all right so let's talk about synthetic video and some examples and issues that we see here now we've been having fun with de big videos for a long

time uh we go back to where we would basically upload an image and it would make like a fun video [Music] now really really bad um but we've been seeing a lot of I mean there's been a huge surgence of deep fake tools whether you're using them on your smartphone we're using them you know high-end graphics cards to be able to process them face swap live deep fakes um you know and so forth and also augmented reality as well from a you know a tool that's available that's out there this is the platform that I used is called haen where you upload 45 seconds of you speaking well lit and it generates that avatar for you and then you just go in

and provide it the text that you want so this is designed to be for training videos but if you flip it on its head thinking okay how can cyber criminals use it Let me Give an

example just the Tex and generates this video see now so that was me in a hotel room uh last year I just type it in and I get that but I can do language translations too

[Music]

so my wife's family is from Lim here in Ireland and I sent this video to her last night and I said you got to get your dad to check it out her response back was like my dad can't believe that you speak that good Gaelic and I'm like it's like no and she goes that's AI generated and I went yeah and she goes okay that's just too scary so even my wife thinks it's scary but it translated and then it moves it has the you know move in the mouth and everything else that goes along with it now for the video that I created the beginning of the video here's Brian to create the C

James downloaded a video of me talking from YouTube and converted it to audio mile he imported that audio file into 11 Labs use samp file to create the de voice with the AI voice generated he typed out Tex one same 11 this created the audio file be used on the AI video generation he downloaded an image picture of me generated the video of Talking various features on face SC so I mean I was able to pull the audio and again with even more soft more audio samples uh could really tune the voice to be even closer sounding um again this took me less than 30 minutes last night download the video pull the uh take the audio put it in 11 Labs I

already have a script dropped that in there generated it downloaded it and then into hedra um like I said hedra blocks if it recognizes celebrity so I guess Brian wasn't much of a celebrity in their eyes and I got away with it um I have done the CEO of T-Mobile out of the US I've done other cyber security folks um but it's very easy to be able to to uh leverage that within the dark web we're seeing plenty of other applications that are becoming available that allow you to use uh to do it with celebrities kind of bypass that security feature uh that are becoming readly available same thing if you're doing uh webcams you can do pick your different

faces that you want you know Robert Downey Jr and I think one of them was uh Elon Musk in there uh but here is uh hater who basically created deep uh deep cam live where you're just providing it the target image and then it totally changes your face around as well now my good puty good buddy Perry Carpenter kind of took it to a whole another level where on the you see what he looks like on the on the right side there but on the left he's basically doing it in real time he put on a wig and then face swapped with Taylor Swift over over his face and as he turned now because he's got the wig uh covering up

the side of his face all it's doing is just focusing on the face and it's very difficult to tell or not tell that it's uh it's him uh and essent because when the Deep fake the face swapping software is working they you usually get stitching along the sides of the face and that's usually a dead giveaway of course with wearing a wig it covers up a lot of those there you can go out and you can uninstall and uh leverage the software you just need a really good GPU like a 3090 24 gigs of RAM um I've got a my the machine I've got here is an M2 Pro and I've loaded Pinocchio onto here and used

face fusion and it can do face swaps uh using the the front-facing camera it's a little sluggish so I decided all right I got to level up my game I've got an M4 Max coming in uh in a couple weeks so we'll see how well that works uh you if you've got uh a spare computer around and you can put a a high and video graphics card in there deep fake OS you load on a bunto server and it comes with a variety of tools deep fake slat uh deep face Lab live live portrait and RDC because the trick now is you want to be able to have the video but you also want to be able to do the

audio of that person as well uh RVC is an app that allows you to do real voice changing uh in real time no doubt that got used for uh that got mentioned earlier today with regards to the um event in Hong Kong where you had an employee contacted by the CFO and then other PE people on the video chat convincing them to send that total of 25 million I found another source today said there were 15 different transfers not sure but I'm sure now we Face the the issue of do I not believe the ignore the CEO request or you know transfer the money to cyber criminals so deep fake scams is becoming a big concern we we're

we're getting a lot of concern from organizations we've seen it already go up 10x the crypto sector is being heavily targeted because that's where they got the money uh again significant impacts in the us but we know this is happening globally the concern with Executives is you know there only about one and four thinking they could spot it and then only about of that a third of their employees might be able to spot it so there the expected loss coming in from research is about 40 billion by 2027 that scammers are going to be looking to get from all of this now as AI is a major part of all this anybody know what kind of music a

computer likes to listen to that'd be an [Music] algorhythm all right so let's talk about defense and protection here so as I said before you've got executive cesos 56% are worried about deep fake fraud and then 75% of the the fishing campaigns there's been a sign you know 92% of businesses have lost money already to deep fakes 50% of them have faced video deep fake uh attacks almost half of them have dealt with audio deep fake attacks um when we start looking about what kind of losses we're talk the numbers are up to about $450,000 for a deep fake attack um 28% of the The Business Leaders lost over half a billion do half a million dollars already with regards

to this um this came out of a CFO uh investigation a CFO survey uh done with people that had experienced um deep fake attacks already so we know it's coming about now you know we want to try to keep up with all the current events that are going on with deep fakes there are two great websites resemble and AI AIC AIC is really good CU they've now created their own taxonomy for tracking these are uh errors Transmissions mistakes that are being made by AI automation that's out there um that's how I found out about the IR the uh the events that happened in Ireland I was able to go through and search I just typed in Ireland and I got five hits um

but you can go in and look at different types of deep fakes audio generative text where there's been errors and issues overall so DET detection and prevention like anything else when it comes to that we're dealing with the type of attacks uh it comes down to our people processes and Technologies because the Cyber criminals have leveled up when it comes to fishing because they can pretty well put this in any language so you know we want to make sure we've got strong communication skills within the organization for being able to verify these digital calls or or uh the the calls or the video calls I know that within teams if somebody's connecting from the outside it tells you they're

external uh but maybe we may need to start to get to the point having technology built into those to determine if it's a deep fake or not um looking at you know regular training improving training that goes on within the organization overall from a technology standpoint trying to get more uh technology to be able to detect these deep fakes we're getting there the kicker is audio right now is doing really really well it's able to go through I plugged in the audio um with regards to Brian's audio that I did through 11 labs and it detected now whether or not 11 labs and play HT are start and those groups are starting to put something in there to make it easier

to detect could be one way um from the video it came back and thought it was legit uh legitimate so our video technology is is getting there it's just not happening in real time it takes minutes and even then we're still experiencing a lot of fakes POS fake positives the haen video one of them out of the four that was done through this website detect oh yeah no that's fake going out doing additional research the 10 best deep fake detector tools I went ahead and checked out all 10 of these um two of them I just showed you the other eight required a demo required a phone call with one of their salespeople first they weren't offering

any free trials so it's easy to go out and make the software make these Jeep fakes it's just a little tricky to be able to properly be able to detect it and we need to make sure that we are not only making that available for everybody but getting it into things like social media and any uh and the video call platforms like zoom and teams and so forth so when it comes to a deep fake what should we be asking do we ask is this as a deep fake no consider these questions and this is the same thing as if we're dealing with email as well this comes from Perry in his book that he he

wrote last year called fake um when it comes to our email you know it's taking that time just to kind of take a moment don't click on it right away here we're looking at analyzing where are the emotional triggers what's the benefits that that's coming out of this see if we can verify the source if it's legitimate um and then staying Vigilant overall when it comes to the uh the Deep fake audio and video that happens you know if you're getting phone calls and it's from a loved one and you're not sure text them out of band connect with somebody else or ask a really weird question or something they may know the answer to

that not a lot of people other folks will you know when you were over on Tuesday night what was that dish you cooked for dinner one of the other interesting things especially with regards to Ai and we've got so many organizations wanting to leverage generative Ai and have it for our users years ago and some of us remember when we had internet proxies first come on the scene getting higher speed internet we didn't want our employees all going out to the sports network or going out to Etsy or PayPal or or whatever else so you had the internet proxy and we want the same thing now start having more technology available for organizations so that we

can monitor prevent the sensitive information being uploaded to a large language model like we've seen with Executives when they upload strategy plans or Health Care uploading patient information to write a letter to a hospital or an insurance agent but being able to go through and monitor and be able to protect our own uh intellectual property within the organization is critical as well as I said before when it comes to email cyber criminals have leveled up so there are three questions that I always go through when it comes to email is the email unexpected nowadays anybody got one of those video camera doorbells nobody couple of us some of us okay nowadays when the doorbell rings at

my house I grab my phone CU if I'm not expecting somebody I want to see who's at the door and I don't want to go to the door and if it's usually somebody standing there with a clipboard it's like yep nope not going to answer the door I'm not expecting it so I'm not going to I'm going to you know take action to either in this case ignore it if I'm not expecting the email most likely either I don't need to read it or I'm going to go through and check some other things is this person a stranger you know if I don't know them I'm going to go through and look then okay what are they asking is it

something urgent is it something you know unusual or they want a quick response and if I'm answering yes to any of them I'm going to go through and either verify the connection and who it's coming from uh double check the message if they're looking to try to start a conversation you know looking and reviewing all of that because um I've already been burned once by no because we get fish twice a week by no when it comes to training essentially we're looking at making sure that we're growing the security culture in the organization we want to go through do the frequent training don't just do training like when Father Christmas and the the Easter Bunny come around like

once a year we need to make it more than that have it regular throughout the year because cyber criminals are changing so much with evolving in their plans as we're advancing we need to make sure we're keeping our users uh up to dat and aware as well any Microsoft employees in the room awesome anybody know why Microsoft employees are so anxious it's because they're on edge how many programmers have I got cool how many programmers does it take to change the light bulb none it's a hardware problem all right some final thoughts and we'll get you all off to lunch 70 to 90% of the attacks that go on that are successful leading to other breaches to

ransomware to business email compromise are a result of social engineering so it's important that we're going through and educating our users not thinking of them as the weakest link thinking of them as a layer of of the security in our organization getting them educated getting them aware when I have security people come and tell me ah humans are the weakest link what are you doing to be able to fix that we need to make sure that they're educated they're aware and I know we can tell them and just because we're aware doesn't mean we care it's like driving down the highway when the speed limit says 120 and you're doing 150 you know what the speed limit is

you're just hoping you don't get caught same thing applies with we see with a lot of the users they know they're not supposed to click the links they're in a rush or whatever we want to make sure that they're being mindful of it so we know AI is an incredible tool we want to ensure that we're educating everybody it's okay to be politely paranoid we want to be aware of the AI hallucinations the biases the Deep fakes we want to trust and verify those the fishing game hasn't changed we need to be aware don't rush and be skeptical it's okay to be skeptical so some questions to consider as you go back to work on Monday does

your organization have a protocol do you have a way of verifying Executives or people in your organization if they're calling you if they're reaching out if the policy is the CEO will never call you then that's what the policy is but we need to make sure that folks are aware it goes back to the gift card issue scams that we were seeing years ago they've just leveled that up can your sock team detect a real time AI deep fake attack before it succeeds and how do you defend against social engineering now that AI is being involved for years when it came to technology you could keep up read some news articles read some magazines a

couple times a month with AI it's a daily activity for me personally I know it is going through Reading the different news newsletters because we're hearing so much and so much is changing almost on a daily basis um one colleague who does AI for their living they're reading and keeping up to date hourly there's so much changing and going on overall so there's some newsletters and podcasts you can definitely check out to try to keep current and up to date on all of it now I know today I've given you some dad jokes in my presentation but does anybody know why DNS jokes are so hard because it takes 24 hours for people to get

it so I want to thank you all for your time and your attention here this afternoon

yes that's a QR code it goes to my LinkedIn profile um I do have stickers I brought stickers with me I know that everybody likes stickers i've got some fun no before ones uh if you've ever heard the phrase don't click on I've got some of those stickers too um yes I do have a YouTube channel for my dad jokes and yes I do have a way of keeping track of all my dad jokes it's in my database so thank you very much enjoy lunch