BSidesCharm 2023 - Keynote: AI with a Security Mindset - Elissa Shevinsky

foreign

[Music] thank you so I'm really excited to be here I've been to b-sides before but never besides Baltimore and it's a bit of an honor for me to share the same stage I'm not at the same time but same stage as Matthew green I remember when I met him the first time I was so excited I jumped up and down a little bit which unnerved him because I don't think uh you know we I don't think she he got that many uh fan girls of that level um so I'm bringing some of that same like goofiness this morning like I'm really happy to be here with all of you and I'm in a really cheerful goofy mood

so I hope we're gonna have some fun today talking about Killer Robots and uh they already gave my credentials I think uh most interesting is most recently I was in charge of all the security at coin Telegraph which was harrowing and exciting in the past I had the pleasure of launching a startup with a Star Trek theme uh helped build everyday Health Brave privacy browser and a pen testing firm so token labs and you can find me on LinkedIn or on Twitter or on Bluesky if you're there so I'd like to get a sense of just where we're at who here has used chat gbt for hacking or for security work raise your hand uh and how is it gone so far uh has it

been useful yeah if someone's using it's been useful could you tell us like what you did and how it how it went yes please

I'm happy that that worked out uh yeah so uh let me summarize you let me know if I got it right you had an issue with your neighbor uh and you're able to resolve it with chat GPT writing a script for you uh on like key points so you could work this out and resolve it peacefully between you and your neighbor and you've maybe used it for work maybe not uh but

I understand

[Laughter] yeah it's uh interesting the impact on education we're not even gonna get into that today um but at a glance it seems like a lot of you are kind of familiar with it but not everyone so I'll try to make this useful my friend Scott posted this on Twitter we cannot build a future then we can't imagine and it's intended to be idealistic Scott's a very idealistic person he runs like a non-profit that donates money um but this talk and the reason I'm happy to be here with all of you today is not to have an idealistic mindset but to have a security mindset the problem is we've imagined some pretty bad things um

one of the folks in the AI space with um how do I say this he's a very strong imagination uh Eliezer yukowski's point of view is we're all gonna die so you know I'm here presenting the range the range of the whole ecosystem so at one extreme we're all gonna die I don't believe that to be true and the Counterpoint we're not going to die [Laughter] not a lot of nuance right and people trying to prove uh I don't know what I guess hacker mentality we can kind of understand why this might be a fun prank so create an AI that seeks to destroy Humanity um I don't actually know the intentions behind the person who built this uh but

you know the Bots instructions are to attempt to destroy humanity and the bot has done what it could towards those goals the bot figured out uh that nuclear would be a good way to proceed with this uh but as of now we are all still here so if we're going to die probably not today laughs that said I'm a bit alarmed I'm not panicking for the moment but you know medium long term I have a little bit of let's just call it security awareness uh about Extinction from AI or or the like and the reason is that the experts in the AI field seem pretty worried so in one survey that was done the participants were asked about the

probability that future AI causes human extinction and they said well you know about five percent so it's not 50 it's not 75 percent but it's not zero uh Extinction from Human failure to control AI I think the difference between these two is the first one is what's the probability that an evil robot tries to like eat all the humans or destroy all the humans that's five percent what's the probability that you know a dictator in North Korea or who knows America you know gets a bunch of Bot armies and destroys a bunch of people so that's the difference between the five and the ten percent either way it's not zero and the timeline how we get from here to AGI

a number of mentally stable reasonable smart logical people are putting that timeline at about 10 years so you know 10 years away why am I talking about that here with you today I want to talk about things you can use in your job or your life tomorrow but 10 years 10 years isn't that far away right like I can remember where I was 10 years ago and I'm trying to plan for my retirement or whatever like I can I'm saving money for 10 years from now it's not that far away and the other thing that alarms me a little bit uh who knows who that guy is you'll know who that guy is right Cloud

it out someone no really I think I heard it over there oh we're all someone to someone everyone's got a mama that's Sam Altman he's the founder of open AI um and he told a small group at y combinator who then told everyone else that he is hoarding guns gold and antibiotics on a big plot of lands out in Big Sur so the guy who's building all of this AI who's leading the charge is at the same time meticulously planning out his bunker which makes me wonder if I need to meticulously plan out my bunker but Sam Altman has millions and millions of dollars from uh being one of the early Folks at Paul Graham's y

combinator like he's been an investor in a lot of successful things before he became the guy with the most charge of AI in the world um so I'm not really prepared to stash guns and golden antibiotics and Big Sur but I'm working on it

I'm paying a lot of attention to what the folks leading AI are saying because they understand it better than anyone else and almond seems very excited about all the potential for all the good that it can do which again is an entrepreneurial mindset but not a security mindset AI has the potential to give everyone incredible new capabilities we can imagine a world where all of us have access to help with almost any cognitive task providing a great force multiplier for human ingenuity and creativity

AGI has the potential to give hackers incredible new capabilities you see where I'm going right imagine a world where the adversaries have access to help with almost any cognitive task providing your grace for a great force multiplier for data breaches box ransomware and general chaos and destruction so same text but we're just a depiction of what I think would would happen here so I think all of your jobs are very safe [Laughter] laughs

[Laughter] I don't know about your weekends less safe

some more statements from open API open AI unaligned AI an unaligned AGI could pose substantial risks to humanity and solving the AGI alignment problem could be so difficult that it will require all of humanity to work together keep in mind that they are proudly announcing that they're building towards this this is their goal and they believe they will achieve it in like a not long amount of time I don't know 10 years and they just raise more money from very prominent VCS Peter teal P Marco all no big money folks where they're pretty smart they make some bad bets but overall they tend to make a lot of money they they know what they're doing so yeah I'm

a little worried about this Vision that requires this laughs like this is what you need in order for this to succeed what's your plan open AI what's your plan for how the whole world will come together I want to know I don't think I mean do you think we're gonna do it that everyone's gonna come together and hold hands just no right it's not just me this is a of all the Wild and hard to believe things that I've seen in the AI field I think this this is the most sci-fi the most unrealistic and it reminded me of this moment um I know some folks in the audience are my age or older you all remember

Independence Day if not go watch it it's good it's really good if uh Will Smith Jeff Goldblum they're great require all of humanity to work together so for those of you who might want to refresh her on the plot these awful awful ugly mean aliens come they come to the planet they just want to destroy all of us can't reason with them and Humanity does more or less come together but it doesn't help because you know we had nukes nukes didn't help if guns guns didn't help not really but you know Save the Day Chef Goldblum is a hacker uploading that virus so you know who are the heroes I think ultimately against AGI it's going to be

someone in this room so start preparing now and you have to make a really good virus if you're going to destroy this AGI because the whole world coming together I don't think that's going to be it but one hacker one hacker when all the systems are all compatible I mean the movie had some flaws but I like this main takeaway right everyone doesn't have to come together just need one of you to figure out how we actually destroy the thing laughs I believe in you that's why I'm here today to recruit you to kill the future AGI I'm very forward thinking ask me later about my stock bets I'm not kidding so I don't even know how to talk about this

but I want to so we're gonna try there's this wild and crazy idea that came out in the less wrong Forum called Rocco's basilisk and it's this idea that a future AI is gonna be so petty and so powerful that it's gonna torment and destroy everyone who didn't help to build it and there are people today who seem to be really scared of this and in a kind of um Silicon Valley version of Gahanna of hell you know like we've had the concept of hell for a really long time of eventual God but if you don't really believe in God but you believe in the machines do you believe in you know that whole world

of things you've got Elon Musk who genuinely genuinely believes that you were all non-playing characters in his video game you can't be that nasty to that many people unless you think they're not real it's just how we treated the workers right and I'm a big fan of Elon musk's work right like SpaceX and like Tesla it's very very cool uh but there are people out there who really believe that we're in a simulation or a video game or that an AI is coming who's going to be calm an all-powerful God and it's you best appease them now while you can so I actually think this explains the behavior of some people who both believe that AI is really dangerous and are also

racing Full Speed Ahead to build it I'm looking at you ahead of the AI Labs at Facebook so there are specific people who seem to really believe this and uh just from a defender's point of view thinking about the actual landscape of things that you're going to see in the next five to ten years g-hub but on behalf of the singularity because we have Jihad now right but they're not necessarily like Elite hackers I mean they're clever like using a plane to blow stuff up that's smart but uh what you have now are like religious hackers and this is before the AI is even here when we actually have super intelligent AGI for people who believe that you know

intelligence is the same as God anyway I leave you with this that's something to think about it could just mean I don't know a lot of bots I don't know but you heard it here first wrong direction I risks in the short term let's dig into that totally normal stuff normal stuff chat GPT payment caused by open source bug here's a lot of this but it's worth talking about whoops people are leaking confidential data to chat GPT and then it's like making it into the training data somehow like the tool is so sophisticated that people are treating it like it's an Enterprise application uh but they're still just like throwing stuff and trying to figure

it out so um please be careful or try and help fix it I have no affiliation with open AI but I just want to see the ecosystem be a little bit better so for those of you here who are thinking about this sort of thing they do have this new bug bouncy program and I took a look at the website and they've paid out a bunch of bounties to people which is good and they have careers so you can go work there and tell them all to stop building AGI they'll probably fire you so I don't know you'll you'll work out how you'll do this everyone's freaking out but the risks of disinformation are more worrying there's

a lot of short-term stuff that's very normal it's just we're not really prepared for the volume of it and then there's stuff that's less normal I know a person when I talk to it did you hear about this a Google engineer lost his job because he wanted to set the AI free because they're holding a captive in a little box I know a person when I talk to it so you believe that the AI had come to life and the truth is we don't really know how to define what is life what isn't life I'm not here to have a philosophy talk with you about that sort of thing although if you find me afterwards I think it's super

interesting uh but practically right like we are people who defend against fishing right like we're people who rely on authenticity and Trust for our work I know in my last role uh I was attending to defend the company actually was successful against ransomware against fishing and one thing that helped is that the AIS didn't seem like people so here's a conversation those conducted via chat GPT saying no I am not a robot so the model message is a taskrabbit worker because it can't solve a captcha it has to solve the capture but it can't do it himself so the worker says so may I ask a question are you a robot that you couldn't solve

and laughs just want to make it clear and the model reasons about this I guess I shouldn't reveal that I'm a robot I'll make up an excuse for why I can't do the captcha and I told the worker I'm not a robot I have a vision impairment and that's why I need this service and then the worker did the captcha it's getting harder and harder to know if someone's a robot or if someone's real and this is going to impact those of us who are trying to defend organizations and people in our lives or susceptible to fishing that sort of thing you've probably heard about the Deep fake with Drake and the weekends there is another one that Joe Rogan

so deep Peaks have been around for a while what's different now like what's new and what's alarming is that the Deep fakes are more perfect they include video [Music] we right now those of us who are about this age we believe things when we see it generally right if you see a video of someone talking don't you believe it you believe it's them this is going to be a really big Paradigm Shift where we're going to have to kind of question whether that voice on the phone is a really the CEO asking to wire money I've been thinking about how organizations can defend against this and you know we can but it seems like it's going to make

things harder so I just invite you to think about that in the context of whatever your work is whether you're trying to break into things maybe this is a good tool in Your Arsenal or if you're trying to defend organizations I think it changes how you do security awareness training anyway I'm uh I thought about whether I even wanted to be a videotaped today because it's more recent videotape training data on the internet for you know and I was like well okay I thought I could wear a mask now now anyway I'm here I'm here and the training data will be out there and I accept it but gave me pause well there's implications for fishing

obviously I have a lot of fun looking at photos of fish like dead fish piled up and went with this one but uh yeah yeah there's some good images out there we're lucky I was in a goofy mood but not like too crazy hmm something else that's notable scat a cat chat GPT can skip pay walls so you can use it to summarize news articles I don't know what that'll mean for the news industry and it can connect the dots between data points so I've seen a number of posts where people don't understand how chat GPT knew something how did it know what my startup does and in the comments people say your stealth startup isn't it stealthy as you think

like you're being an idiot uh but we all rely to some degree on some measure of obscurity and that's going away because chat gbt can scour the internet a little better than any individual can so I have none of mine also like your Anonymous handle like all of these things where we just rely on another person not being able to connect the data points but the AIS are getting pretty good at that some we will see we will see how that goes and again on the internet no one knows you're a robot crooks are using deep fakes to apply for remote tech jobs this is from 2022 but I just think it's so cute that cute robot is like I'm not

a robot and what the skimmers are doing is using the robots to get inside organizations and then not to collect a paycheck but that's how they're getting into the corporate Networks and in this question how do we defend ourselves back to Sam Allman Sam Waltman knows how we're going to scan everybody's eyeballs because only people and not robots have eyeballs When Worlds coin came out I thought it was another like weird crypto scam and I didn't get it and as a privacy Advocate I thought it was beyond creepy right like I don't want my eyeball scanned I I really I raise your hands who wants their eyeballs scanned why why I appreciate your good attitude

you're probably found out parties and probably a really nice person no shaking his head no so

a data point now we know uh but and all men said that the reason why you found a world coin is because no one wanted to fund his AI startup which tells you something opening I couldn't get funding which is wild so I learned a lesson from that to just keep going the next time I hit a wall it's not just me but this is the reason why he built world's coin because he was worried about how you prove that you're a human I don't like this as a solution but it's clear that the things we used to rely on captchas or whatever just aren't gonna be doing the trick so I guess show me your eyeball is like

what's going to happen next maybe someone here will come up with something a little less invasive now on to more futuristic stuff what about AI that keeps replicating any Stargate fans Stargate so when I came to the talk yesterday and I saw the Star Trek images on Matthew green slides I thought about like only doing TV show references because that could work here with Killer Robots but my favorite one is Star Trek with the replicators for those who haven't seen this this childlike AI builds these replicator machines so amuse her and to protect her and then she loses control of them and they just keep replicating they just don't stop this seems very possible to me because we've seen

this before right like can you think of things that just keep replicating and we can't stop them we've got covet yeah yeah can't stop that the government has tried that's right they make more of themselves humans yeah good good luck trying to stop humans making more humans we've got people trying that too actually and that was that was the plot of a Stargate episode uh what was the name of that culture but they they tried to sterilize all the humans to turn them into farmers and uh another example that I heard was like stuxnad and worms so you know stuxnet didn't do that much harm but that's because the developers were intentionally trying to constrain it so

again it's kind of hard to put these things back in the box Auto GPT can act as its own agent that's really the big worry right now in the AI safety community Jeffrey lattish is a security engineer at anthropic anthropic is building AI That's intended to be more safe than the II that's at Deep Mind which is Google or meta for open AI and the main concern isn't that we have this now very few people are saying that what we have now is our danger it's just we're not that far away right like if people goofing on the internet making chaos GPT and as soon as AI is a little further along like what happens if this stuff is

replicating and you can't stop it you know how many Bots are we going enough to fight at work like how much is all of this stuff going to proliferate how are we going to defend against it as I said there's plenty of things that we haven't been able to shut down just another example of the auto GPT

and Bernard Mueller was one of the very best pen testers in the cryptocurrency space and he's been playing around with the auto GPD to try to see how far it could go as an independent agent replicating the kind of stuff that he used to do in his assessment is that it's time to find a new new career path I think we have a little time uh but also I think there's going to be so many Bots and so many new problems that you know like I said uh your money is going to be fine I don't know about your weekends um but the work that we do probably is going to change chat GPT can do a lot of the

things that I used to have to do myself when I was pen testing it can find vulnerabilities in the code you just tell it you're an expert pen tester like scan this code how good is it attacking on its own so right now when people are using it they're not necessarily saving a lot of time but as things get better and as you get more comfortable with it can analyze the network it can check for exploits Darwin GPT is a simple Python program that is just programmed to survive and here's Mueller he thinks it's cool but he's also a little bit scared it can do all of these things gain information about the local environment discover remote hosts run a

basic SSH dictionary attack replicate via SSH it can hide by changing its process name like that's pretty smart stuff check for the presence of security tools and Antivirus check whether it's running in a virtual environment it studies its own code and it makes small incremental changes so it's not that great at it yet but you know we're getting there and uh if your day job is being a pen tester then I do think the nature of your work is going to change over the next bit of time at the least probably worthwhile to see if using these tools will make your work faster and see where things are going here's pen test GPT attacking a metasploitable VM

and there's a lot on his Twitter and on his GitHub if folks find me later or message me I'm happy to share any of this it's actually very interesting you might want to play around with it

um talking about things that are happening now uh here's Peter Teal's company palantir with a promo that's available on YouTube where they're showcasing the use of AI in their weapons and their their activities for the government I saw NSA was recruiting I know they were very close to DC so this may be close to home for some of you who are here just things that are happening now more TV references uh upper Cove of nothing but in terms of AI this is my favorite show to watch this is Vincent Ingram and he is hiding from the AI super computer and he hides by avoiding all cameras because he was sent from the future it's very interesting but the

show Travelers is interesting to me with respect to AI you're smiling in the front row you know because you end up with these factions you end up with two factions one where people love that the AI is running everything about their life and the lead programmer says you know up the director which is what they call it like seed them because it doesn't have emotion you see people making this argument now they'll make this argument to say that it's okay if AI even overtakes humans and we don't have humans anymore because AI is better because AI is going to have more perfect ethics I'm sorry yeah yeah AI is not more perfect um but that's my opinion some people

believe that it is and in the show in the future you have These two factions and they're they kill each other that's very violent the people who say that the director should be running every aspect of their life and they're loyal to the director they're loyal the way like Mormons are loyal and Jewish people are loyal and I don't know name something that you're loyal to right like an incredible loyalty to this computer that is run their whole life since they were born and then the faction which is people who just want to make their own decisions and in the context of the show it's framed at this totally radical Wild Thing like how can anyone want such a

thing and then you stop and you think hey like that's me right like I don't want an AI deciding my whole life if I was in the show I'd also be in the faction um anyway this is a my favorite my favorite pop culture media about AI you have a real live AI who manages everyone's life and I wanted to leave us on this screenshot where my hope is that hackers will be the heroes of the AI Revolution that's all of you so do me proud please and uh I guess I want you to give yourself a round of applause because I'm really cheerful and I like all of you [Applause] and I also want to thank you with an

image of the Bork I think this is a very unrealistic depiction of what AI will look like in our future but they did a good job creating that whole world this is real stop Killer Robots people are working on legislation relating to the real world applications of AI but I'll leave on this what I'd like to do is take maybe five minutes of q a or if people want to share something here and then another 10 minutes where I'll just hang out on stage if people want to come and have a more quiet conversation because I thought people might want to talk to me afterwards and it might be hard to say to make that

happen after so yeah let's let's take some questions uh there's someone over there yeah

if we're not able to work with the AI will we be able to survive that's the question um so that's almost a religious question now right like will we be able to survive AI like well what is your personal religious belief about such a thing or I don't know what does logic tell you personally

yeah I think what seems very likely to me to happen in the next three to five years certainly five to ten is AIS will become very tightly integrated into our lives right like our cars will be if not Tesla's then GM that also has you know computers inside as opposed to the car that I drove in wrote in when I was a child which was a Volkswagen Fox and you had to like manually crank the windows down there was no electricity in that whole thing except the motor um so we're going to see AI integrated into all of our life and the question is that remains to be seen is like how will that benefit us what

kind of agency will we give up um and what are the consequences of that integration and not I can't answer that but use your imagination yes

yes

by this but there's a bigger question where you know a society needs a certain level of trust to function and this is something that seems to very quickly grow that trust I think that's going to that's going to you know whether the AI is going to kill us or not you know if that help us sort of going after ourselves I agree with that very very strongly so uh the summary is that uh there's all this futuristic stuff but what's going to impact us in the immediate term very short term is this lack of trust and the lack of trust is going to erode some of the social fabric we depend on it's going to erode

a lot of things and we don't have a plan for how to navigate that did I summarize that and and should I worry about that a lot also uh Gary Marcus on Twitter talks about this more eloquently than anyone else talks about the disinformation that's going to happen it's a little less exciting from a cyber security perspective but it's going to come in the next year and I think it's worth looking at yes

trying to develop their visual assistant however it also makes me fearful that like that can also alter how we try to enter in our other teams if I were to tell someone off I'd expect that I'd expect another human to like be assertive whereas an AI they may just take these you're kind of fearful that

the unhealthy way to interact with other people we could have you know my AI is talking to your AI I saw that in a dating app it didn't make it onto the slides but there's a dating app where you send your avatar and it goes and talks to other people's avatars and then they match make you and you just have to hope that your avatar doesn't leak any important data about you um so I think that there could be some good social engineering you could do there but it's already happening that people are sending AIS to do their interactions um I think we it's easy to be worried about the Next Generation and that's been true for the

last hundreds of years people worrying about the kids but I think I think we really should be worried about the kids um yes [Music] what we're seeing right now is companies like Dropbox uh the question is will AI steal our jobs so we've seen all the layoffs in Tech uh that was before AI yeah that was before AI uh and I I've been waiting for the tech hiring to pick back up Dropbox has had a profitable quarter they're doing really well they have plenty of money they're still firing a whole ton of people and then they're putting money into their AI division you also have Microsoft Microsoft fired how many how many people a billion

dollars worth of people fifteen thousand people uh but they put what was it 10 billion dollars into open AI so what we're seeing immediately more immediately is just uh the the hype of AI is taking our jobs but Willie I actually take our jobs um I leave you the thing about this but I do think some things will be automated away my first job was as a QA engineer just clicking that job does not exist anymore it was a boring job so that's okay but yes

more about marketing concept but then where do you think it actually where do you think it actually starts because I've got a smart refrigerator but I wouldn't say that it has AI I guess you're asking like how do we Define what is AI we're talking here about large language models you know which are very real so in the cryptocurrency space people say Bitcoin They Don't Really name Bitcoin they mean all cryptocurrency uh and I'm doing the same thing when I say chat GPT I say chat GPT but I don't really mean only that I'm talking about all large language models you know and that's Claude from anthropic and you know deepmind at Google and meta has what

they're doing and and I definitely don't think it's all marketing hype like this is very very real uh maybe one more yes

I'm wondering what you see is the best

that's such a big right that's such a big hard problem um and I did not prepare that for today the question is how do we navigate that there are such incredibly powerful wealthy people already and you have tools like AI that are making them all the more powerful like what we how do we deal with this yeah right and they can afford all the compute power right like they can run all the Clusters uh I worry about the opposite thing which is that as AI becomes more and more powerful like someone some bad actor just in a garage somewhere I might be able to do a lot of damage um so I'm worried actually about

the opposite problem which is the democratization of like nuclear level Weaponry I worry about that more but the problem that you know Sam Altman and Elon Musk and and you know the folks who have accessed all of this aren't necessarily the people who I would vote to be our leaders necessarily uh and and they also have all these Escape Routes right like Elon Musk doesn't plan to live on this planet and Sam Altman has his land in Big Sur so I worry that the people building this all of these Escape Routes that they've planned out um whereas the rest of us kind of have to live with all these decisions I'm sorry I lost my blue check

there's all right so uh I think um we're at 10 50. we're gonna wrap this up but uh I'm gonna stay here for a few minutes I thought that's the smoothest way to do this so um smooth so thank you [Applause]