On Your Ocean's 11 Team, We're the AI Guys (Technically Girls)

welcome back everybody uh we have a half an hour break after this session and then the closing ceremony so please stay around after this talk um right now we have a great talk by Harriet foow and Chantel R vesa uh on your Oceans 11 team where the AI guys technically girls let's welcome them to the stage

someware group is taking credit for a massive cyber breach MGM Resorts International says it has been hit by a Cyber attack TVs were out in the casino couldn't gamble at this point we're wondering what are you paying for it's September 11th 2023 and over at the MGM casinos in Las Vegas something just isn't right all of a sudden guests begin to notice that their room keys no longer work the casinos computer systems are completely offline their ATMs and slot machines are also broken and as a result the casinos begin losing millions of dollars with every passing day rumor has it the attackers behind the hack were none other than security startup Founders Chantel Reva and Harriet farlo

they really do mean it when they say startup life isn't easy Legend has it they're targeting Singapore canra and Sydney's casinos next so the rumors are in fact true that was Harriet and myself fortunately for us scattered spider and black cat and a number of other APS took the blame for the attack so here we are one year later with our next victim Cambra Casino only in cyber crime are people actively willing to take credit for crimes they didn't commit for those of you I haven't met my name is Chantel RVA and in addition to hacking casinos which I clearly love to do in my spare time my passions and expertise lie in the people's side of cyber security which as

I'm sure all of us know regardless of our disciplines or the cyber security controls our organization puts in place uh we're only as strong as our weakest link which is why the user is so so critical and that's what I do in my company cyber uh where we deliver tailored awareness programs as well as ENT assessments so that's really what I'm going to focus on in this presentation today uh and a lot of the presentations have been highly technical so hopefully I can bring bring a little bit more of the psychological as well Harriet I'll hand over to you thank you Chantel hello everyone I'm Harriet Faro it's such a pleasure to be here I'm

actually from CRA and this is my first time presenting at bides CRA so it feels like a very special crowd um I've been working at the intersection of AI and security for almost a decade now I originally studied physics but soon found that artificial intelligence and data science was my calling I worked in Consulting for a while on defense projects moved to the USA to work for a startup um came back to work for the Australian government for a certain agency you know we're all one of those kinds of people here right and then while I was doing that I started my PhD in adversarial machine learning which is basically the ability to hack AI systems

and I realized that there just wasn't a lot of resources in the outside world for companies to be able to secure their systems so I started my company maleva security Labs about a year and a half ago with that goal um the last time I presented my part of this talk was at Defcon in August which was amazing but also terrifying and I'm not usually scared of public speaking but I think that particular talk was by far the scariest thing I've ever done so it's really nice to be at a a friendly crowd uh like like here in CRA although speaking to sort of friends and colleagues and past colleagues is in some ways a bit scarier I think but I

remember for the death con talk because I'm knew but had to do a shot on stage which may or may not have like been to my benefit or detriment but I do recall telling everyone that if that event were in Australia you wouldn't be doing a shot on stage you'd be doing a shoy and I don't know if that's a legacy of Australians I'm proud of sharing but um there we go that's what it is so we're all familiar with the concept of a of a heist right um the origin for this work started with un wanting to understand how AI secur like the impact of security of AI systems and AI for security impacted different Industries and I

thought what better industry to look at than the casino industry not only is it an industry where there's you know really um High demands for companies to be secure and trustworthy because people are sort of losing money there but because the companies have such a you know an an onus to handle that that data sensitively and especially in today's economic climate that that's even harder um and of course because defc in Vegas I thought the casino element might be really nice so a lot of this re the research actually started with me interviewing people at casino Cambra right around the corner from here so thank you very much to those folk and casino CRA might not be exactly what we

imagine of you know the kind of Casino in in Vegas right but I really wanted to look at the implications of AI insecurity in this context through a heist and you know Chantel and I we're both startup owners uh life is not easy in the early stages of a startup we really need to get some cash so we decided that you know thinking about that from the perspective of criminals trying to uh steal money from casinos through some of the the recent hacks that we've seen would be a really cool angle to it even if um we we don't see as many George cloony as we'd like but I think we all get the premise of a heist

right so these are our objectives for today so we're going to rob the Casino um first of all Chantel is going to be conducting some ENT and socially engineering our Target to elicit the information that we need uh I'm then going to exploit that information and hack the facial recognition AI of the casino um using an attack that I created and then Chantel is going to um look at the the implication of deep fakes in the setting as well not just for casinos but for lots of Industries uh and then we're going to profit of course but also we should really talk about what your organization can do to protect itself uh so there's a few disclaimers

obviously these are real attacks we're talking about and real attacks that could be implemented um but this presentation itself is educational and fun um we're not showing you exactly how to implement the attacks um although there are lots of sort of resources that you know we've put online or other people have put online that show you how to do that but we're ethical hackers uh and casino cber was very generous with their time we had permission to do this kind of research um and we're very grateful for them so something that you might not be aware of is the incredible uh increase of AI security related incidents uh in companies and organizations um on the

whole like I've been working in this field since 2021 and I remember when I first started and I'd come to these kinds of security conferences and I was the only AI person there and I was sort of that you know it was a bit novel you know it was a bit random um but now as as of 2024 77% of organizations have reported AI related security incidents in 2024 um and that sort of covers you know ai4 security and security of AI but even in um you know my field it's it's greatly increasing it's really important overall so this is why it's important um but I do have a fun question for you because I have some maleva challenge

coins to give away um because of course you have to have a challenge coin right we're all from government um so the first two people to come up to meet the end of the presentation with the correct answer to this question what is the percentage increase from 2023 um within 5% error um can win a challenge coin okay thanks so I'm going to start off by setting the scene of the MGM hack because that really was the context for how we decided to Target Casino Cambra so as you would have seen from the video earlier it's been over one year since the MGM fell victim to a Cyber attack that was of of course perpetrated by har

and myself uh so leveraging LinkedIn we were able to identify an IT specialist who became our Target and we gathered as much information on this target as possible enabling us to call the MGM help desk and socially engineering them into resetting this employees password and the only information that was needed was the name employee ID and date of birth which I'm sure all of us can agree that with a little bit of effort that information can be so easily sourced so from that we were able to gain access to the mgm's network and our victim was actually an administrator with Advanced privileges across the MGM systems so that gave us access to mgm's OCTA and

azur tenant environments so some Reports say that we stole 6 terab worth of data including personal information about customers so names uh email addresses passport information social Security numbers the list goes on and on uh regardless of that the MGM refused to pay our Ransom and when they detected that we were on the system they shut down their OCTA sync servers and essential infrastructure which is what actually caused all the disruptions from the digital reservation systems not working to key cards and so on so here we are one year later ready to Target our next victim camra casino and the reason that the MGM hack is it's not unique in the slightest it really does

sit at the Nexus of several cyber secur Trends and that is the increasing targeting of the user which is why education awareness and the human element of cyber security is so important and 98% of all cyber attacks are the result of some element of social engineering so these are the three stages of our attack today uh I'm going to start by infiltrating the casino's Network and gaining initial access and from that I'm going to escalate my access by gaining more credentials and then ultimately exfiltrate data and hopefully demand a ransom make money from the attack so reconnaissance reconnaissance is such an important stage when it comes to cyber attacks because we know that with a lot of sophisticated threat

actors like APS reconnaissance is a huge component because social engineering is really the Cornerstone of their attack operations so osin is what we're using today for our reconnaissance and osin is or open source intelligence is the process of gathering and analyzing publicly available information either online so through the surface web or the dark web or offline to form conclusions uh and answer specific questions there are multiple tools that I when it comes to ENT uh we've got government records we have internet search engines social media the dark web the list goes on and on so why ENT traditionally used in law enforcement ENT has really expanded expanded in its use over the past few years and is being used by a number of

other Industries and organizations so 's a really powerful tool in identifying vulnerabilities being able to identify potential security threats uh and bolster a an organization's security and it's used for a range of different purposes as well from executive protection which is what uh we use it for at my company to cyber threat intelligence supply chain risk management uh the list goes on and on and recent report came out actually this year that by 2033 they estimate the oin industry will be worth more than $58 billion so it's only be becoming more and more important but of course the Dark Side of oan is that anything that we find as cyber security practitioners can of course be found by attackers as

well and that is exactly what we're using for today because our purpose is to infiltrate cber Casino so similar to the MGM attack I'm starting with LinkedIn to be able to identify my Target and fortunately Casino camber has a much smaller security team with only 12 people so I've selected my target Henry Goan he is the director of security and surve surveillance now the points of data that we look for when profiling a victim as an attacker is of course passwords email addresses phone numbers where they live their friends and family so our aim is to gather all of this information by using osin so from Henry's LinkedIn account we're able to within a matter of seconds

find his email address and his phone number by using contact out which is a data broker uh so that's a tip for you guys today go home and contact a number of data Brokers and request that your data be removed from their from their platforms now of course once we have a piece of data like an email address we can very simply find a password I'm using a platform an oent tool here called Expos Ed which is very similar to have I been pwned in the sense that it tells you what data breaches uh your data your email address has been a part of but unlike have I been pwned it actually tells you the exact passwords

that were breached so maybe don't tell too many people about this tool cuz that could have us in danger because attackers don't even need to purchase the data breach they can just use this Tool uh and visibly see your credentials the first is to compromise Henry's home Wi-Fi so because I'm here in CRA I know exactly where Henry lives I've actually driven to Henry's house I sat outside for a few hours and I'm executing a home network compromise attack so how did I do this well to execute this attack I'm using what is known as an air crack NG suite and an alpha network adapter so that I can sniff all the traffic within my vicinity

but I only care about Henry and his Network because he's my target so I've been able to identify and locate his BSS ID which is his Wi-Fi network name because Henry uses his family name as his Wi-Fi name so what I want to do here is I want to crack the four-way handshake and the four-way handshake is what authenticates your device to your Wi-Fi network and how can I crack that four-way handshake well one way that I can do this is by using something called a deauthentication attack so I can send an authentication frame to the network which will force all devices on that Network to De authenticate and then reauthenticate and in the meantime I've

been able to intercept and capture that four-way handshake which is that encrypted password then I've gone home and I've cracked the packets and I've been able to identify what Henry's network password is and I've cracked his home Wi-Fi there are a range of things that I can do now that I'm in Network um I can sniff all the traffic on the network and decrypt everything I can run a man in the middle attack and I can compromise every single device on that Network so from his daughter's devices to his own device I can uh intercept his iot devices and persistently listen to everything going on in that house the list goes on and on I can also steal his credentials as well

so his passwords and his session cookies and the most important thing is that I can impersonate Henry and access the casino's assets so when it comes to home networks home networks are not monitored networks which means they're so much less secure and possible to hack uh and when we think about hybrid ways of working which almost every single organization Works in a hybrid way it poses a serious risk to an organization's security and of course Wi-Fi credentials I mean how many of us have gone and reset our Wi-Fi username and password probably not many of us and they're generic and that means that they can be cracked in a matter of hours now attack Vector number two say I

am not in Cambra and I can't sit outside Henry's house and crack his home Wi-Fi well I can actually send him a targeted fish and given everything that we've learned about Henry so far through our reconnaissance we are going to socially engineer him in the most targeted way possible so we know that Henry loves caravanning and from his Facebook he actually recently sold his Caravan so he could be on the market for a new Caravan at the right price so I've sent him this fishing email that's beautifully created about secondhand luxury Caravans here in Cambra uh I could there are two options here I could attach a malicious attachment which when the macros are enabled would deploy malicious code

however given that micr Microsoft now warns users when they're enabling macros it's a lot less stealthy so I'm going to go for the second option which is to send a malicious link and when Henry clicks on this malicious link oh I just also wanted to say that URLs are now the most popular delivery mechanism for cyber crime as well which is probably for this reason but when Henry clicks on that link he's going to be redirected to this page and in the background a JavaScript exploit will be deployed which will basically give me access to Henry 's browser uh and I can control and compromise his entire browser and as we know browsers have so many permissions

which is incredibly dangerous because it means I can do so many things I can move laterally from the browser uh I can again install malware onto his device I can same process those last two I can steal credentials passwords session cookies and I can of course which is our priority impersonate Henry and access the casino's assets either apps and data and then exfiltrate data and demand a ransom uh because depending on what sort of threat actor you are depends on obviously what your intentions are but in this instance we're financially motivated so that is what we want to do but I'll hand over to Harriet now to talk about the AI component thank you Chantel so

cool okay we're going to switch gears a little bit and instead of looking at um some traditional cyber security um like incidents or ways that we can use AI for those uh we're going to look at um the security of AI themselves so I mentioned before that I started this work with Casino CRA because I had a few assumptions I guess in how AI was used at casinos and I thought well Chantel and I need to make some money so how can we try and exploit those systems I assumed that AI would be used for things like surveillance game monitoring facial recognition person detection but I thought that game monitoring would be one of the most important forms of AI

use in casinos um I was actually really wrong it turns out that facial recognition is by far the most important use of artificial intelligence in casinos um because it turns out that even though casinos obviously want to U you know monitor the games that is very much still dependent on humans to do that through security cameras to then identify the people that they need to be fed into the facial recognition system so that they prevented from in in future so I understood that facial recognition is a really uh vulnerable form of artificial intelligence it's something that we definitely want to try and exploit so let's do that now so to clarify we're not looking at AI for

hacking we're looking at hacking and AI system itself and this is actually quite a new idea for for a lot of people it's definitely changed quite a lot over the last few years in terms of recognition and and how seriously it's taken so can I get a show of hands people in the room who' consider consider thems an AI person you use AI quite a bit the lights are so bright okay um so that's not many people so so that's sort of what I expected so it's traditionally not been really seen as part of the Cyber and information security world I guess to consider novel attacks on AI systems but it's becoming really important and it's

moving from just an academic field into something we're actually starting to see in the real world right now so the the field that it's come from is this academic field called adversarial machine learning and this is basically the idea to um like hack AI systems to manipulate them or disrupt them so they do the wrong thing basically so this example here is of a computer vision machine learning model looking at two stop signs one is normal one is coated in a special adversarial paint and for the one that's in this with this adversarial paint it's misclassified as a sports bow with 80% accuracy even even though to to a human that's still clearly a stop sign so we can imagine

how this kind of uh implications um would be really significant for computer vision problems and things like autonomous vehicles where they need to be able to recognize stop signs this is the original adversarial machine learning example I guess you could call it so the idea was that you have a clean image and then you add specially crafted adversarial noise to that image so that a human can't really tell that it's there but it's able to disrupt a model really considerably so we add this special noise to the panda image and then the model misclassifies it as a gibbon with over 99% confidence and this isn't just random noise this is noise specially crafted based on the

Target Model uh that you're trying to disrupt or some kind of surrogate or proxy this kind of attack um I guess was proposed in sort of 2013 and the the field has moved a lot since then when I first started my PhD and I told my supervisors I wanted to look at adversarial machine learning they were kind of like are you sure I don't think that's ever going to be a real thing like you might be shoting your career in the foot well I proved them wrong unfortunately because now there are lots of AI incidents and companies are really starting to uh to struggle with how to actually defend against these kinds of attacks this is an example of some

really popular kinds of attacks if if you can't read the words very well the point I want to get across is that for all the different kinds of machine learning models and systems you have at the top I have an example of a computer vision model which is a convolutional neur network and then on the bottom a natural language processing model which is a Transformer which is the backbone of things like GPT um for all of those different I guess attack surfaces you could call it there's over a 100 different kinds of different attacks now that you could Implement on models like this they're different to traditional cyber and information security attacks and they occur at all different stages

along the sort of machine learning life cycle uh from training to inference and it's something that's that's really important and it's it's growing way instead of talking about all those you know 100 plus attacks um a way that I like to describe the attacks is by the 3D model um can I get a show of hands for anyone who saw Tanya's presentation on on Thursday the one about Tik toks and adversarial Noise Okay so that's a lot of people so she did a great job fortunately of explaining adversar machine learning so now I don't have to um but for those of you who didn't know I will still um go into it but basically we co-authored

this paper that proposed this this model as a threat Centric way that sort of complements the CIA Triad but for adversarial attacks so a good way of grouping them up is by thinking of them as being able to either disrupt deceive or disclose so if you're disrupting an AI system you're just not making it work at all if you're deceiving it you're not making it work but in a very specific way so if we think back to that stop sign example um a disrupt attack would be if a Tesla doesn't recognize a stop sign and then a deceive attack would be if it recognizes that stop sign instead as a speed sign we've specially um we've

chosen that that Target classification and then a disclosure based attack is just being able to leak sensitive data about the about the training data or the model information itself now the idea of hacking an AI system has existed as long as as AI however you you know decide to Define it so something like um algorithm hacking has existed since the the 1990s and some really interesting examples in the casino context um I'll I'll talk through very quickly so the first one is uh from the 1990s and uh a man who was employed by the Nevada gaming commission or authority board something like that um he was able to get access to the seed of the random number generator that

generated the numbers in a Kino game and he was able to make you know $100,000 um from Kino games by exploiting that information to guess the correct number so that's an example of hacking that random number generator algorithm without necessarily having to do adversarial machine learning another example that I think is really interesting is actually um card counting and blackjack um if we think about it the Blackjack algorithm itself is one where you know you can exploit it if you play Perfect strategy for example you know the way that you can minimize the Dealer's Advantage um any of the games in the casinos have an advantage to the casino from you know 2% to 25% the the

25% top end would be something like the slot machine and the only reason it stops at 25% is because that's by law by regulation um but a game like Blackjack can be as low as 2% if you play Perfect strategy and then if you implement card counting on top of that um you can make that even lower so that's an example of of algorithm hacking as well by the reason that adversarial machine learning as a field is quite different is because it exploits architectures inherent to machine learning models and what's really important about the machine learning training process is the idea of minimizing eror through many many iterations so I think most of us are

familiar of machine learning as a concept you basically have your input data uh some sort of output label or prediction and then over a training process the model is able to map that input to the output and the way that it's able to do that is by tracking the the loss so the error um and it does that by a gradient function so we'd probably remember the concept of a gradient from from math it's sort of like the the slope of something basically we're tracking the the slope of the algorithm as it goes from high error to minimum error and the way that attackers can use that is that if we know the information about a model and

how it was trained and how it was trained to minimize error we can exploit that to maximize the error or at least maximize the error within a specific uh Epsilon value or small amount so it's just at a level that it passes over a classification boundary um but not at a level where it's really obvious to a human and so if we think about the machine learning training process of something like a convolutional neural network where you go from your input images through the training process and then you end up with a probability of of who you're looking at at the end um it uses that gradient function and and we can exploit that as an

attacker so I mentioned before that I used to work in the government and something that I was really interested in um for my PhD in in adversarial machine learning was instead of having to create an attack where you basically perturb the target object or the entire entirety of the image frame you know the entire picture what if we could only perturb small regions within it and beyond that not actually have to perturb the targ object itself but perturb regions around it so I'm imagining that if you have um a computer vision model pointing at I don't know me for example instead of recognizing me I could have small you know physical objects placed around me that that basically disguise

me or can act as camouflage and because I was in defense I was sort of interested in applying this to um Urban Camouflage environments I think I need to play the video here

there's always a tech issue that's okay so it was just a quick clip of um showing the ship moving with these what I call distributed adversarial regions around it so this is a still of it so basically if I could place these um these regions these adversarial regions that are distributed around an object to sort of camouflage from from detection so this uo would instead be classified as a ship and you could make these sort of dynamic they could move around um I was I was interested in exploring that and the the technique um called distributed adversarial regions or odar um I found to be really effective for object detection so being able to disguise things like ships or cars or

boats um you know or or planes that that kind of object it was it worked quite a lot so I thought if we want to make money from the casino and we know they that they use facial recognition how could we apply this to my face so I can walk into a casino and it doesn't recognize me even if if I've been logged as someone that they might want to prevent from walking

in is it not

working maybe it's the internet I um because demos never work I did uh download the video here so I'm going to play it

locally thank goodness for fail safes um so as I as I talk through um this I'm I just put it on two times speed so it's not quite so quite so long but basically this is an example of like implementing this attack uh I know that watching Code might not be all that interesting for some people all of this is on on get so if people are sort of interested you can go check it out later anyway but basically the point is that you can implement this very quickly um for this uh one you know image example that I'm demoing here um so the idea was that if if I want to add sort of regions around

my my face to prevent the facial recognition what kind of object could I put around my face well I decided on jewelry so if if you're a man and you're not wearing jewelry yet well it's 2024 so you should really consider it especially if you want to walk into a casino and defeat the facial recognition but basically by being able to um exploit the internal gradient function and the parameters of the facial recognition models that I was testing as part of this research I tested a bunch of different open source facial recognition models um I could optimize for the ideal area within sort of a range that makes sense for some jewelry uh within the image frame and um

optimize the specific sort of pixel values that I want to have as those pieces of jewelry and then do things like add different colored filters so that it's not just random sort of pixelated regions as jewelry um and I found that it worked quite well at being able to disguise me or or somebody like me because at the end of the day it's just an optimization problem thank you Chantel and what I'm really trying to do is just move is is add data to you know me or to this image in such a way that it it um causes the model to move up a classification boundary for example so it's going from recognizing me to not

recognizing me so I can encode information in these sort of pixelated regions and then turn them into jewelry the the the screen probably isn't um you can't see it well enough but um behind the filters there's lots of you know different pixel values that are sort of encoding that information um and so this can be used as both a disrupt attack so it just doesn't recognize me but it can also be used as a deceive attack like I can encode information about specific people like our Target that Chantel was able to elicit information about um so that they could instead be recognized as well it's working so this this is an image of me walking up to a facial

recognition camera it was a day where I really should have done my hair but it's camber and very cold so that's what I have um and I can walk up and the facial recognition doesn't um detect me so this is an example of a disrupt attack do that again you can see that even though it does tend to use a a facial sort of boundary box um a lot of the time it doesn't work um and so we can encode that information with say Henry Goen um so that there is a match found and that's an example of a deceive attack and then of course we're able to profit now we can use an attack like

that to be able to um get into the casino um and um do do whatever we like this is an attack that we can use very flexibly so I I tested this on lots of Open Source models like I mentioned um and the reason I did this instead of relying on proprietary facial recognition models was because you know as a research you don't always have access to that um even if I did I wouldn't necessarily want to um exploit that um you know in front of a big crowd but most of the time from my research I found that a lot of companies do tend to on open source models anyway and that's a really big challenge when it comes to

the field of machine learning because of um because of the optimization that's inherent in machine learning and in the process even if an organization were to create their own custom machine learning facial recognition model they're likely going to be training on exactly the same data set as these open- Source uh models anyway or at least information like it and so the thing about machine learning models is that they tend to converge so I was able to use just these open source models really effectively and this is sort of the number that I got now um during Ten's talk she she talked quite a lot about how AI creates a new attack surface when when you add it to a cyber

system like there's there's new attacks that you can Implement against an AI system that's different to traditional cyber Information Security based attacks and so when I talk about a number like 40.4% that's something that's quite different about machine learning because it is inherently probabilistic so it's not like this attack worked 40.4% of the time this 40.4 represents the decrease in classification confidence for a model when I feed at different images so this means that if as a data scientist I set the threshold between maybe yes it recognizes me and no at 90% um if the original image had a classification confidence of 95% and then on average I'm able to decrease that classification confidence by over 40% that's enough to

push it down the boundary so it doesn't recognize me even if it still you know thinks it's me at 50% um that doesn't necessarily matter because of that um because of that threshold that I've coded into the model so this is really important because so many organizations are relying on these kinds of models now and at the end of the day this is sort of lwh hanging fruit really like this this attack isn't that difficult or complicated to implement um and often I get asked if people you know well if criminals or threat actors would really go to all the effort to deceive an AI system um which cyber professionals is ridiculous right because we know just

how much money and time um different thread actors will spend on trying to um get into computer computer systems so we know that they're actually trying to do this on AI systems now too and this is a really important consideration Beyond casinos um and Beyond us just trying to make money because there is facial recognition being used everywhere for for surveillance for airports for being able to unlock our phone and so making sure that these kinds of models are really robust to even you know basic attacks like like a Dar um is is really important and something that I urge you as cyber professionals to really consider how you can contribute your skills here but without that we want to

move on to the next stage of our [Applause] attack thanks uh so a lot of the discourse around deep fake attacks is very much leveraging deep fake technology to scam to clone someone and scam either a friend or family member or organization out of millions of dollarss and we saw this earlier in the year with a uk-based engineering company where a finance worker joined a teams meeting with who they thought was the Chief Financial Officer and a number of other colleagues and then pay wide across $25 million us to several different accounts and the reality was that he was in a call with every single person in that call was a deep fake but what I wanted to talk about

today was actually leveraging deep fake technology and the threat that deep fake technology poses to disrupting political and economic Landscapes so for us we don't just want to hack camber Casino we want to ruin camber casino's reputation because our next step is to actually Target the casinos in Sydney and Melbourne so we hope that by ruining Casino campas reputation um more people will go to those casinos spend more money that when we target them it will be more profitable for us in the long term and for a lot of the organizations that a lot of you work for and for myself having worked in banking before this that's really the threat of deep fects that we're seeing being able to

clone a high-profile executive and say something incredibly outlandish that could impact the market value of that company making the Share value drop that a malicious adversary can then go and purchase shares at a cheaper price so this is one of many reasons World economic Forum ranks AI generated misinformation and disinformation as the biggest threat to humanity over the next 2 years even surpassing global warming which is massive so oh I might have to do that as well um is it this one the bottom yeah so that being said I've deep faked our CEO of CRA Casino Nicole here and she's got a message that I'll then stream online and impact hi my name is Nicole and I'm

the CEO of the casin

Frozen I think the laptop's Frozen the laptop's Frozen oh no and who doesn't want to see Nicole Kidman on the screen maybe Cino camera is hacking us back maybe oh anyway you get the ID um oh uh are those train

screen that's okay I can just talk to it but basically with one video of Nicole Kidman I was able to clone her exact voice and image and produce that video with my own deep fake tools where she's saying to not spend money at camber casino and that cber casino is not above board so that's the general gist but I just wanted to I guess shift the discourse around deep fakes and the threats not just on scamming companies but actually on how that can impact uh economic Landscapes and market value of organizations so moving on to how you can protect your organization there's so many things that we can do and I had a slide on it which I'm not sure if it's

going to come up now is it still frozen can you keep talking yeah yeah that's okay I lucky I remember um the first is of course education so tailored education for different groups of people and if we think about the MGM hack where the attackers socially engineered the MGM help desk you know having only three points of data that you need to verify and not being trained appropriately as a help desk on how to identify social engineering are just a few of the issues at play and that's why having education programs that are very tailored to different groups because everyone has a different relationship with cyber security and I think for a lot of us we're so ingrained in the

industry that we forget that a lot of people don't have nearly as much knowledge as we have I reached out just last week actually to one of Australia's largest private Hotel owners and this man is a billionaire and he's also a doctor so he's an incredibly intelligent person and I said to him oh Jerry I'd love to deep fake you for my LinkedIn for the purpose of Education and he replied to me and he said absolutely if it helps the cause but what's a deep fake and I think that's just such a reminder that so many people don't know they don't have this knowledge so it really is up to us to spread awareness and we do have the responsibility to do

that the next is of course be careful what we share on social media which I think is a no-brainer but we can all do with the regular reminder um in the in in law enforcement overseas not just running training for employees but they're actually starting to run training for family and friends of employees because often it's not just what we share on social media that can jeopardize our security and our organization security but it's what our family and friends share about us so I thought that was a really interesting Trend that's actually going on in uh the UK which is pretty cool the next is to of course not trust Wi-Fi even your home Wi-Fi I personally never do internet

banking on my home Wi-Fi and I recommend the same to you and always update your browser a vulnerable browser I don't know if I even mentioned it earlier but Henry had a vulnerable browser which is what enabled us to run the man in the browser attack and a vulnerable browser means that it's so easy for someone to hack into our browser and once they're in the browser they can sit there forever and we wouldn't even notice so that's another really important tip and the last is to be aware of tailored fishing campaign specific to you I recently last week Harriet and myself were in Singapore for bside Singapore and the Formula 1 which was really fun

and I dined at a restaurant and then the next day I received an email to say thank you so much for dining at our restaurant here is a link to book again we're going to look after you next time you come and it looked like it was from the reservation company that I had booked through and it wasn't and it just goes to show that a lot of these platforms there either already compromised or they sell our data or they do both and we really need to be aware of tailored fishing campaigns because they are increasing it's so much easier to do now and a lot of what we share on social media you don't even

know who's following you who could leverage that information to Target you so yeah those are some of my tips and I'll move on for you thank you um I also love that restarting a computer is still the number one way to get out of a tech situation um I would agree with Chantel I've seen so many like such a change over the last few years in terms of the recognition of AI security but there's still so many people where I I mention AI security and they assume that I'm talking about ai4 cyber security right um and people have never really considered AI systems as an attack surface in their own right um it's it's definitely changing but it's really

important that as you know organizations but you know in your roles as cyber and Information Security Professionals as well um that this is something that we were being aware of and there are more resources now that help us be aware of it but especially for those people in your organizations who aren't as Tech literate as you guys might be um the other thing is to take it seriously so just as we have you know a really mature security ecosystem around our computer systems I would argue that we need to be adapting all of those traditional cyber and information security principles into the world of AI like AI security and adversarial machine learning is a traditionally academic field for data

scientists and as a data scientist you know I I learned to prioritize efficiency and accuracy of my models and sometimes you know we have to consider bias depending on the circumstance but I was never taught about security this is a really new and emerging threat and uh and it's changing so all of the things we do around risk managing our cyber systems we need to apply to AI as well that includes all of Technology people and process SL ah there are more resources out there now um it's definitely maturing as a field um there's an o wasp ml top 10 maida Atlas is the repository that has ttps for adversar machine learning attacks you'll be familiar with M um attack um there's

now an artificial intelligence security Center as part of the um the US government through the NSA and there's lots of private companies now that are dealing in AI security um like my company I guess but it's nice that this is really increasing and the different kinds of help and services and products um are really out there um there's also educational resources I I don't want to talk too much of a a shout out but um Harriet hack is my YouTube channel and podcast that I copresent with Tanya so if you want to learn more about a security you can listen to that as well um but beyond that please do keep in touch with both of us it's such a

pleasure to be able to present to all of you on something that we are really passionate about and for all of you to to come and show your support means quite a lot um especially you know coming back to CRA I remember the first bsides I went to was 3 years ago I think and I think that was the the first time where I was meeting a group of people who were so willing to collaborate and work together on something that you know I thought was important AI security and it was definitely one of the reasons that I felt sort of confident enough to take the leap and start my own company and I know Chantel has experiences like

that as well so thank you for being the kinds of people who will come to a besides and to you know contribute and learn and please keep doing that do stay in touch um but we also have I think some some time for questions as [Applause]

well a great presentation are there any questions in the audience we have our Runners willing to get to you if you raise your

hand I think there's a speaker at yep

Henry I don't know maybe not after this presentation I'm in I'm interested to hear about your uh concerns about WiFi considering you're going from wifi to the internet surely the internet is more insecure uh I think that unfortunately anything is insecure right if someone wants to Target you there are so many ways to do so even I was doing some research and they were saying oh a VPN will protect you from a Wi-Fi hack but that's not true it makes it harder for an attacker but it doesn't protect you either so yeah so I'd be happier running through my home wifi than being on the internet surely sorry can you say that again you you're making comments about Wi-Fi being

insecure but I'd say the internet is more insecure surely I guess it depends on what you're doing on there but yes if someone wanted to Target you and in this instance you know as we know sophisticated threat actors they they want to if they they financially motivated so they'll do anything to Target someone so in this instance being in camra and that was our Target it was very easy to do so I actually did it at home I have the alpha network adapter which you can purchase off Amazon for $100 and I did it to my own home network and it's it's incredibly easy it's pretty scary and there's actually a $30 if anyone's heard pornog GOI which is

this little device it kind of looks like a Tamagotchi maybe that's why they got the name um and it it makes wi-fi hacking even easier and it's a tiny device $30 you can walk around and it will sniff all the networks within within your vicinity and it has a little face on it where it's like a sad face and it means it's not scanning any Wi-Fi works so you need to take it out and charge it to to change the face to happy which is pretty interesting so yeah uh I'd say definitely both insecure but it depends on in what context we're talking about but yes Wi-Fi can easily be hacked but surely on top of that the protocols

will be encrypted your TLS hopefully not SSL yeah TLS 1.2 1.3 so therefore if you sniff it it'll just be encrypted surely given there's a break after this as well we're happy to the break sorry it's a bit hard to it's a bit hard to hear over the microphone but yes we can have another chat about it Che bonu is really cool they also um communicate via a mesh and share some of those creds so if anyone's got them watch out but uh have you played with uh so two-part question uh have you played with n shade yeah actually poisoning some of the pools to all the training data for for another form of a back door

and before you answer uh if anyone knows the artist that made hyperace can you please get them to reach out to me uh cuz he's really cool he does some of the same stuff you're talking about but yeah Nightshade mainly have you have you played with it is is it a vect sorry I couldn't quite hear all of that question but you're talking about Nightshade so for for other people in the room Nightshade is a uh one of one of many kinds of tools where they're implementing these kinds of adversarial uh the adversarial noise to different images to either like protect it from from copyright claims or reverse image searches or those kinds of things

there's there's a suite of Technologies um what what exactly was the the question sort of how effective those those different tools

are uh sorry I couldn't hear that either but I I think I got the gist so when I was doing this research um and sort of that the same as T we found that there was quite a lot of tools that will address specific parts of the problem but they're not necessarily generalizable to all kinds of models or all kinds of contexts and like we love the work that those tools are doing night shade Forks um there's lots of really good good use cases out there and I think maybe the um the challenge now is being able to mature all of those tools to being um I guess cool um I don't know academic add-ons that

that's not a you know um not a dig at them at all but I think the the problem that I see is that there's no mandate for companies to have to consider those kinds of defensive measures you know if they want to add something like N Night Shad or Forks amazing that's great good on them but there's no um there's no standards uh few standards few regulations that mean that companies actually have to integrate those kinds of security measures into their AI systems right now although that is changing over the next couple years we might have to stop it there if you've got any more questions please hit up the speakers uh during the break let's thank

this uh these amazing speakers one more time thank you thank you