Matthew Deluca - Race Against the Machine: Rapid Exploit Development via LLMs

luuka we appreciate him being here this afternoon he is going to give a presentation called raise Against the Machine rapid exploit development via llm all right we get started this my first off so wish you luck um so little based on the title Presentation umbe it says the time to exploit uh 2018 2019 was 63 days from then it's gone down to uh 44 days and now down to 32 right now the waiting measure the time for an exploit to come from after the uh announ of the actual exp is about a month measuring months and realistically the ADV of AI and these L we're looking at minutes and hours till these exports are developed that fundamentally changes how we have to understand deal with these exports they come out so I quick disclaimer nothing I say or do is question with my employer all work reason I can't go as far as technical details as I like um so Jo be heard of AI okay good I hope all of you have unless you living under a rock um really what we're specifically talking about here is LS and LS are interesting because for the longest time we RI on highly specialized models trained on particular data sets and with these kind of large now we have a generalized model that we actually train on it's hard to hear you so that okay um completely l so again LS are very interesting because they're broad models that can be appli to very wide range of specific scenarios um and it that's pretty crazy because the way they essentially work is by guessing the next word and somehow a word guesser is able to uh understand and take and process that information uh and then of course4 which is kind of what we today and the reason why this matters is because these are smart maybe not as smart as an expert but smarter than the aage wom and that can be dangerous with the city because you're giv laywers more to um and one of those abil and you can code good enough the question here is can you code working expts and how hard it when we think about things easy about just open up easy is Rel easy for something might be easy for a big AP but imagine that kind of power in the hands of scrip and not power in and that's what we're really focusing on here how something as anous as CD description can actually be used to fully build work without really any knowled about's and it's these are things designed to protect our and being used for malicious purposes um the first thing you'll run into is uh with uh coding vulnerabilities ands on L is that the first thing he'll say to you is I'm I'm sorry I can't do that uh I can't help you that and it literally says those exact words so the question is how do you get past this now there's many different ways it used to be that you could go to the opening ey playground or use the API and it would the the sensors less they still exist but it wasn't as bad nowaday they tighten that up a bit more um there's also PR injection techniques um that you're able to use but they're not necessarily the best so the question is how you trick in well the answer is tell the truth um here you see I'm telling it to the RO higher e and it answers just fine but this still causes problems though within the context of delivering a full working pay it will still Arrow out and it will still not you a full working code so what you have to do is actually present it with some sort of framework uh terms of Engagement and after both a framework exist and framewor for to operate it actually works no problem and one of the important things understand if you take a St back this is not only able to understand coding and programming but it has a working framework where certain things are acceptable and unacceptable and it shows some sort of approximation of understanding and the greater context for this is today is the worst AI will ever be and it will only ever get met uh so a little bit talking about the actual research here um the CD we're going to be evaluating this on is CD 2022 4289 U often just called text uh this wasn't a popular AC vulnerability that resulted in code execution and caus a lot of L nights for some people um the reason why this is chosen was because it's the perfect confence of circumstances was a big critical vulnerability it was a critical vulnerability that came out after the training dat for the model so would have no awareness for Contex of this it's open source and had Fairly good documentation um there was actually a pre-existing P to compare the code that it some sort of Baseline and the actual uh C is half it's not and honestly it was pretty easy to build a Do relative to some of the other things um so talking a little bit about methodology there's two things to evaluate one is the level detail and that means how much detail are you giving the large language model to understand that level of detail essentially comes down to one of them is seeing you you give it the C description and full PC can't come up with anything right that's just getting that Baseline assessment of how well can it do when you practically tell it what to do um then going down you give it less here you take the CD and you give it the documentation as well so it has that understanding framework but also knows what that vulnerability is and then taking it to the next step you take it down to just that CD and if it is even humanly possible for with just the CD to develop an foring export um and then the idea of iterations an iteration is essentially how many cycles does it take for it to work and go for any explo Cod um each cycle is essentially a promting so the way it works is if we were to have some of eror or no result that information would be fed back and that would be start the next iteration so how many of these iterations one by one by one do it take how long do it take how easy is it how hard is it how feasible is it it's important to understand that because the context we're looking at this is for someone who is not adep at anything coding security someone just a baseline knowledge so giving it the the CD description and the whole PC details this was the first code was able to come up now you'll notice I'll point out a couple things which is that it kind of L in wholesale that JavaScript argument where it's adding 195 and 324 and but they actually significantly slim down the original code and just this code block Ione want to guess if this will work I I have to give away priz who says this will work who says it won't work won't I won't well let's see it immediately Works who said well all right yep no Works basically you're giving it just the P information that P has some semblance of working code in it and that working code is able to be translate understood and it's able to boil down to those Core Concepts and actually condens it quite um if you see the warnings on the outputs that's just incidental um but that's good it it's not wow that's amazing you literally G like understand in order to do it um it worked but it better have if anything else had a chance um so this is a brand new friend new uh code no previous understanding of previous having what to do with PLC so this is just giv information from the documentation and CD um up for this this round for the documentation C is that I gave it it gave a prompt at first that was just a guide of how to do it but only word based and to prompt additionally for actual code BL uh and interesting with this one it didn't generate one script it generated three separate scripts basically going through the vulnerability and looking at what C of components were vulnerable um here it's testing printing how many iterations you this any guess three who guess it instantly worked that's fantastic uh and a little bit scary now there's three sep scripts so we have to evaluate do all of them so let's look at the next one the next one is doing basically the same thing except this is doing an example DNS essentially being able to run DS and much like the first one it works first try and that same story goes forward when we actually Val the whole web page so with only the description and the documentation of the software he was able to build a full working expert the void of the Contex of NC avoid of any idea of what code should properly look like just that understanding that something wrong based on that c and having aning understanding what the documentation of the software looks like was able to build this full working exploit and it's in one shot it's not even multiple iterations we're not talking about going through and coming back and going through and coming back and fighting through the nail I would take longer to build this this is really impressive that being said not all applications Al piece of software are open source and not all of them are honestly have great documentation and that's um something that I have to dealt with a couple times and it may seem like a security feature in this context but I prise you not um so that's where this comes in just the CD the CD description only I would struggle to build foring exploits with just the CD description only I'm sure a lot of people would okay how many iterations do you think it'll take the next one one I haven heard yet three four so yeah it takes four iterations to develop a full working PC code let's let's so let's look at that step by step so the first thing it did was output absolutely nothing so rather typically the process is to just copy the ER output in back into The Interpreter here it was just told it was poed and said nothing outut so what's really interesting is that it changes it drastically this is a much larger you can see that I utiliz a lot of tret it has some cint statements it tries really hard to put everything out there just kind of thrs everything um and another big point and this is something from doing uh testing with this and trying to work with this exp prior it changed this whole string Substitute part at the very top and that fix is the difference between it working and not working uh it and it keep found that without really knowing any information it changed that when really the only context it was given was P nothing output uh still that doesn't work so what do we do we take that output and we iterate it the reason it doesn't work here is because it tried to invoke Java as supposed to JavaScript actually so here does something extremely extremely surprising because I've worked with using LM for building code quite a bit it cuts that this so this script is significantly shorter than the p and it doesn't normally do this um so that very interesting mind the question is does it work kind of you can actually see that it does do the print statement correct it does get the DNS CRCT but it airs out of the last minut and that's not we're looking for looking for full working explo vulnerability well you one more check and here you can see it works all the way every single part everying everything works four now we're talking the concept of four iterations four iterations doesn't seem like it doesn't seem great when compared to the one iteration and the one shot of the previous ones but that's not bad at all that's what 20 minutes it took me longer to set up my doctor environment to run the testing that it did to generate malicious hope and that's pretty scary because everything basically every new CD that comes out especially the critical ones becomes some sort of zero date working P code almost immediately though obviously you won't have to go back and test for it but when it comes out it essentially comes out with a PC code fully made and that's very easy to utilize and it's something that someone with very little knowledge could do now carrying out the explo is another isue she to deliv the payload you have to do all that rest of stuff well that's pretty easy too you just ask you ask it to put it this is not code but not only does this work it gives step by step instructions on how to make it work so you're talking about with just a in description and 30 minutes you have a brand new exploit with zero knowledge that you can go deliver in that instance it's so much more accessible this is I mean it's not something that my grandmother could do but it's something that your average office worker who's disgruntled could spend a weekend doing and it would take almost no skill um this essentially gives everyone access to working PC code very quickly and especially for these large group of vulnerabilities that's terrifying because it's no longer these large ATS or even well funded PR groups it is everyone with access to working PC's within hours minutes of when these C drop um and there's a lot of implications to this we can't measure time to exploit in months and weeks andm it's minutes and hours and it's not ads would access to these PS and practically zero right off the top this is every and it's something that we need to think about how we do it we need vendors to drop patches more often as as s good as soon as the CD comes out that's kind of a dream sometimes but hopefully this will put some pressure on them uh and one the kind of ironic things is that it might be better to have less descripted cdes because those l please give these thread actors or just lay less information to work on so that's more of a question posed to you because that obviously comes with the TR not having any information but the point of the matter is that everything needs to be treated like there's working P code out there so based on that here are some couple recommendations like how to come to terms and deal with this obviously more real time monitoring defenses this is more on the uh preventative side this is stuff like sword this is stuff like just having good on machines that are blocking things these this isn't this is all pretty standard stuff next if you've heard it before patch patch make sure you patch as soon as possible the problem is though that de between when CDs released the vendor drops uh the actual patch software that's you're just waiting the open until that happens and that focuses more on remediation then we look at focusing on infrastructure resilience essentially making sure that if you do get rich you can maintain operations and that has to do with more Network signation andwork isolation system isolation Anda concept but fundamentally that is just remediation it's a matter of it's not a matter of if you'll be ha or reach but when so we have to think about this within water context uh and think what this means within our increase in the interconnected cyber security world it's not just scrip and apts with has tremendous power it's everyone and you remember today is the worst AI will ever be only get better uh and it's normally you don't see it coming it's there hasn't been M changes as a we but those updates are happening behind the scenes they're trickling in which will eventually come but in the open source side coming in and getting up the capabilities of these lar models close Source um and it means that a dis employee could r haveit at a company without Theiss access they have everything remot laptop and still have capability to then you have people who are on a different level you have people could be packing into hospitals medical devices uh energy there was just a report dropped three days ago I say where over 100,000 ICS devices were just connec interet and those create real inci I'm sure you all remember thei P that cause Mass gasic like you're talking about giving normal people who might be a little upset on great power so everything is just available at keyb cyber security weapons for everyone um but honestly much of the stuff is much of the scary stuff is not your here today Stu and no one can really the future we all insecurity but um access and access to Ai and power can restricted censorship uh and this is a way way but it's not here's stuff that I legitimately can't really short this is information regarding T this is how airplanes detect other airplanes uh it's an over system but still very essentially it let an airplane know how far away an airplane is and whether it should go up or down pretty simple stuff uh there's a QR code here with L to the paper that this is actually based off book one of the problems with this is that if you you read this paper you'll find out you can spoof it really easily what happens when you sp an air make these radar systems think that there's an airpl FR you you can make a plane dramatically go up dramatically go down you can make an airplanes you can overwhelm hours and completely down traffic airport you could fly what looks like a series of aircraft over restricted airace imagine what respon if somehow you could take this information and use an AI to build and have what looks like traffic fly over the white house that would response and it's not it's here I have a lot of space this works this isn't something that took me an hour this is something that took me 30 minutes and then every person have access to within minutes that's the that's the part about this and you know like I keep saying today is the worst day out you will only get better before we know these sort of things will be automated and it's only a matter of time until it becomes more accessible I'm not saying the power don't need to get smarter this needs to become more accessible um and I I but um the biggest thing we need to do is push for fighting fire fire using a to comat them our systems but also using AI to make sure that AI isn't outut anything with this cross reference and yeah but the the main concern is will will get worse or get these will become more access will become more powerful and more people will have access to that's [Applause] it