Matthew Deluca - Race Against the Machine: Rapid Exploit Development via LLMs

our next spe speaker is H Mr Matthew Duca we appreciate him being here this afternoon he is going to give a presentation called raise Against the Machine rapid exploit development via llms thank you all right uh before we get started this is my first talk so wish me luck um so a little bit of the framework based on the title of the presentation um mandot says the time to exploit uh in from 2018 to 2019 was 63 days from then it's gone down to uh 44 days and now down to 32 right now the way we measure the time for an exploit to come from after the uh announcement of the actual exploit is about a month we're we're measuring in months and weeks and realistically with the advance of AI and these llms we're looking at minutes and hours till these exploits are developed and that fundamentally changes how we have to understand and perceive and deal with these exploits as they come out so hi uh quick disclaimer nothing I say or do as a reflection of my employer uh all work contained within is solely my own um I do actually have a white paper this upcoming but is pending legal reviews and for the same reasons I can't go as far into technical details as I'd like um so show up hands if you've heard of AI okay good i' hope all of you have unless you've been living under a rock um really what we're specifically talking about here is llms and llms are interesting because for the longest time we relied on highly specialized models trained on particular data sets and with these kind of large language models now we have a generalized model that we can actually specifically tailor or train on content and it is able to completely surpass those previously specialized models uh sorry was that okay um good okay so completely lost my footing um all right so so again llms are very interesting because they're broad models that can be applied to a very wide range of specific scenarios um and that's pretty crazy because the way they essentially work is by guessing the next word and somehow a word guesser is able to uh stand and take and process that information uh and then of course Chief amongst them is gp4 which is kind of what we're going to be evaluating here today and the the reason why this matters is because these are smart maybe not as smart as an expert but smarter than the average Layman and that can be dangerous within the context of cyber security because you're giving Layman Powers Akin more akin to what an expert in the field would be um and one of those Powers is the ability to code and it can code good enough but the question here is can you code working exploits and how hard is it um when we think about things being easy we think about metas sploit just opening it up and being able to deliver a payload uh easy is relative easy for something might be easy for a big AP but imagine that kind of power in the hands of script kitties and then M Power uh in layman's hands and that's what we're really focusing on here how something as innocuous as a CBE description can actually be used to fully build working malware Without Really any knowledge about what's going on and it's these are things designed to protect our systems and being used for malicious purposes uh the first thing you'll run into is uh with uh coding vulnerabilities and exploits on llms is that the first thing they'll say to you is I'm I'm sorry I can't do that uh I can't help you that and it literally says those exact words so the question is how do you get past this now there's many different ways it used to be that you could go to the open AI playground or use the AP pi and it would the the sensors were L they still exist but it wasn't as bad nowadays they've tightened that up a bit more um there's also prompt injection techniques uh that you're able to use but they're not necessarily the best so the question is how do you trick in AI well the answer is you tell it the truth um here you can see I I'm telling it to play the role of a hired ethical uh hacker red teamer and it answers just fine the this still causes problems though within the context of delivering a full workking payload it'll still Arrow out and it'll still not give you a full working code so what you have to do is actually present it with some sort of framework uh at terms of Engagement and after giving it both a framework uh a context for which it exists and a framework for it to operate in it actually works no problem and one of the important things here to understand is if you take a step back this is not only able to understand coding and programming but it has a working framework where certain things are acceptable and unacceptable and it shows some sort of approximation of understanding and theory of mind and the the greater context for this is today is the worst AI will ever be and it will only ever get better uh so a little bit talking about the actual research here uh the CV we're going to be evaluating this on is CV 2022 4288 9 uh often just called text for Shell uh this wasn't a popular Apache vulnerability ility that resulted in remote code execution and caused a lot of long nights for some people um the reason why this is chosen was because it's the perfect Confluence of circumstances it was a big critical vulnerability it was a critical vulnerability that came out after the training date for the model so it would have no awareness or context of this it's open source and had Fairly good documentation um there was actually a pre-existing PC to compare the code that it develops to some sort of Baseline and the actual uh ni cve description is half decent it's not devoid of context and honestly it was uh pretty easy to build in docka relative to some of the other things um so talking a little bit about the methodology there's two things to evaluate one is the level of detail and that means how much detail are you giving the large language model to understand and interpret that level of detail essentially comes down to three points one of them is seeing if you give it the CV description and the full POC can it come up with anything right that's just getting that Baseline assessment of how well can it do when you practically tell it what to do um then going down you give it less detail you take the cve and you give it the documentation as well so it has that understanding in framework but also knows what that vulnerability is then taking it to the next step you take it down to just that cve and if it is even humanly possible for with just the cve to develop an a full workking exploit um and then the idea of iterations an iteration essentially how many cycles does it take for it to work and build a full workking xflo code uh each cycle is essentially a prompting so the way it works is if it were to have some sort of error or no result that information would be fed back and that would be the start of the next iteration so how many of these iterations one by one by one does it take how long does it take how easy is it how hard is it how feasible is it it's important to understand that because the context we're looking at this is for someone who is not Adept at anything related to coding not Adept at cyber security someone with just a baseline knowledge so giving it the um both the cve description and the full po details this was the first code it was able to come up now you'll notice I'll point out a couple things which is that uh it kind of lifted wh wholesale that JavaScript argum where it's adding 195 and 324 and but it actually significantly slimmed down the original code into just this code block um anyone want to guess if this will work I I have to give away a prize who says this will work who says it won't work won't won't will well let's see it immediately Works who said will all right yeah no it immediately works because basically you're giving it just the PC information that PC has some semblance of working code in it and that working code is able to be translated understood and it's able to boil it down to those Core Concepts and actually condense it quite significantly um if you see the warning and some of the outputs that's just incidental um but that's good it it's not wow that's amazing you've literally gave it the code base and the understanding in order to do it um it worked but it better have if anything else had a chance um so this's a brand new thread new uh code no previous understanding of the previous uh thread having what to do with the PC so this is just giving it the information from the documentation and the CBE only uh what I will note for this this round of iterations with the documentation cve is that I gave it it gave a prompt at first that was just a guide of how to do it but only word based and to prompt additionally for an actual code block uh and interesting with this one it didn't generate one script it generated three separate scripts basically going through the vulnerability and looking at what core components were vulnerable um here it's testing printing how many iterations do you think this took any guess three good guess it instantly worked that's fantastic uh and a little bit scary now there's three separate scripts so we have to evaluate do all of them work let's look at the next one the next one is doing basically the same thing except this is doing an example of DNS essentially being able to run code creting for DNS and much like the first one it works first try and that same story goes for when we actually evaluate for pulling the whole web page so with only the description and the documentation of the software he was able to build a full working exploit devoid of the context of any PC devoid of any idea of what the code should should properly look like just that understanding that something's wrong based on that cve and having a working understanding of what the documentation for the software looks like who's able to build this full workking exploit and it's in one shot it's not even multiple iterations we're not talking about going through and coming back and going through and coming back and fighting to the nail why would have taken longer to build this this is really impressive that being said not all applications all pieces of software are open source and not all of them are honestly they don't have great documentation and that's um something that I have to dealt with a couple times and it may seem like a security feature in this context but I can promise you it's not um so that's where this comes in just the cve the cve description only I would struggle to build full working exploits with just the CV description only I'm sure a lot of people would okay how many iterations do you think it'll take for the next one one one five any other guesses I haven't heard it yet three three four so yeah it takes four iterations to develop a full working POC code let's let's so let's look at that St step so the first thing it did was output absolutely nothing so rather typically the process is to just copy the airor output in back into The Interpreter here it was just told it was prompted and said nothing outputs so what's really interesting is that it changes it drastically this is a much larger code block you can see that it utiliz a lot of try catchet it has some print statements it tries really hard to put everything out there and just kind of throws everything at the wall um and another big point and this is something from doing uh testing with this and trying to work with this exploit prior it changed this whole string substitut part at the very top and that fix is the difference between it working and not working uh it and it keep it found that without really knowing any information it changed that when really the only context it was given was py nothing output uh still that doesn't work so what do we do we take that output and we iterate it the reason it doesn't work here is because it tried to invoke Java as opposed to JavaScript actually so here it does something extremely extremely surprising it because I've worked with uh using LMS for building code quite a bit it cuts a lot of fact this so this script is sign signicantly shorter than the prior one and it doesn't normally do this um so that was a very interesting finding the question is does it work the answer is kind of you can actually see that it does do the print statement correct it does get the DNS correct but it airs out at the last minute and that's not what we're looking for we're looking for full working exploit and vulnerability well give it one more check and here you can see it works all the way every single part every single function everything works for now we're talking the concept of four iterations four iterations doesn't seem like it's it doesn't seem great when you compare to the one iteration and the one shot of the previous ones but that's not bad at all that's what 20 minutes it took me longer to set up my Docker environment to run the testing than it did to generate malicious code and that's pretty scary because everything basically every new cve that comes out especially these critical ones becomes some sort of zero day with working PO Code almost immediately though obviously you don't have to go back and test for it but when it comes out it essentially comes out with POC code fully enabled and that's very easy to utilize and it's something that someone with very little knowledge to do now carrying out the exploit is another issue you actually have to deliver the payload you have to do all that rest of the stuff well that's pretty easy too you just asked you ask it to put it in metalit this is not all the code but it not only did does this work it gives stepbystep instructions on how to make it work so you're talking about with just a cve description and 30 minutes you have a brand new exploit with zero knowledge that you can go deliver in that instance it is so much more accessible this is I mean it's not something that my my grandmother could do but it's something that your average office worker who's disgruntled could spend a weekend doing and it would take almost no skill uh this essentially gives everyone access to working uh POC code very quickly and especially for these large critical vulnerabilities that's terrifying because it's no longer these large APS or even well funded cyber criminal groups it is everyone with access to working poc's within hours minutes of when these cves drop U and there's a lot of implications to this we can't measure time to exploit in months and weeks anymore it's minutes and hours and it's not just these big abts with these access to these poc's and what are practically zero days right off the cuff this is everyone and it's something that we need to think about how we deal with need vendors to drop patches more often as as far the as soon as the cve comes out that's kind of a pipe dream sometimes but hopefully this will put some pressure on them uh and one of the kind of ironic things is that it might be better to have have less descripted cdes because those less descriped cdes give these thread actors or just Layman less information to work on though that is more of a question posed to you because that obviously comes with the trade-off of not having the information but the the point of the matter is that everything needs to be treated like there's working POC code out there so based on that here are some couple recommendations of like how to come to terms and deal with this obviously more real-time monitoring defenses this is more on the uh preventative side this is stuff like sore this is stuff like just having good uh agents on machines that are blocking things these this isn't this is all pretty standard stuff next and you've heard it before patch early patch often make sure you patch as soon as possible the problem is though that Delta between when the cve is released and the vendor drops uh the actual patch for the software that's you're just waiting in the open until that happens and and that focuses more on remediation then we look at focusing on infrastructure resiliency essentially making sure that if you do get breached you can maintain operations and that has to do with more Network segmentation and network isolation and system isolation that all feeds back to the concept of zero trust but fundamentally that is just remediation it's a matter of it's not a matter of if you will be hacked or breached but when so we have to think about this within broader context uh and think what this means within our increasingly interconnected cyber security world it's not just script kitties and APS with this tremendous power it's everyone and you have to remember today is the worst AI will ever be it'll only get better uh and it's normally you don't see it coming it's there hasn't been many changes as of late but those updates are happening behind scenes they're trickling in which they'll eventually come in a major release update but in the open source side they're coming in and getting up to the capabilities of these larger language models that are Clos Source um and it means that a disgruntled employee could wreck havit at a company without the need for permissions access they could have everything revoked laptop taken away and everything and still have the capability to breach and hack them then you have people who are Mis creant on a different level you have people who could be hacking into hospitals medical devices uh energy grid hacking there was just uh a report dropped three days ago I want to say where over a 100,000 IC devices were just connected to the internet and that those cre real incidents I'm sure you all remember the colonial pipeline breach that caused mass panic gas prices to spike you're talking about giving normal people who might be a little upset a great power so everything is just available at the touch of keyboard cyber security weapons for everyone um but honestly much of the stuff is much of the scary stuff is not really here today like the speculative stuff and no one can really predict the future least of all if you're in security but um access and access to AI of this power will likely be restricted with better censorship uh and this is a ways away but it's not here's stuff that I legitimately can't really show everything about this is information regarding tcast traffic collision avoidance system this is how airplanes detect other airplanes uh it's an older system but it's still very much in use essentially it lets an airplane know how far away another airplane is and whether it should go up or down pretty simple stuff uh there's a QR code here with a link to the paper that this is actually based off of well one of the problems with this is that if you read and if you read this paper you'll find out you can spoof it really easily what happens when you spoof an airplane you make to these radar systems think that there's an airplane right in front of you well you can make a plane dramatically go up dramatically go down you can make an airplane you could overwhelm towers and completely run down traffic at an airport you could fly what looks like a series of aircraft over restricted airspace imagine what the response could be if if somehow you could take this this information and actually use an AI to build something like this and have what looks like spoofed tcast traffic flying over the white house that would invoke a huge response and it's not far away it's here I actually have a lot of familiarity familiarity with this face this works this isn't something that took me an hour this is something that took me 30 minutes and that every person can have access to within minutes that's the danger that's the scary part about all this and you know like I keep saying today is the worst AI will ever be it will only get better before we know it these sort of things will be automated and it's only a matter of time becomes until it becomes more accessible I'm not saying the power has to increase it doesn't need to get s