Securing AI - Seth

uh morning all uh I'm going to try and stick to script so I can get this done in the right time um so welcome to the talk uh it's not deeply technical um because it would have taken a lot longer to get through uh my aim is to get you to think about generative AI encourage you to go and have a play um find new and interesting ways in which we can use it um who am I uh I'm Seth I've been a CIS admin and engineer for more years than I care to remember Building Systems Services while doing it security for the last 10 12 years uh as an instant responder I cut my teeth on AP that was

quite good fun um I've worked with CPN and ncsc developing FR Intel and developing defenses against all types of attacks um defending critical National infrastructure is a passion of mine uh I've been doing it for the 10 years I've been in security uh I moved on to fret hunting is I have a tendency to get bored doing stuff um so wanted to do something different and now I'm back into architecture that allows me to strategize and plan for what's around the corner hence generative AI uh when I'm not working I play computer games uh I play with Lego I have a number of Raspberry Pi Projects that still aren't finished um and occasionally I like a

drink just occasionally okay so generative AI service what is it um it's usually an app uh that interacts with a large language model and your own data so basically uh devices or users create a prompt create a question that's sent via API to the generative AI service the generative AI service then queries the large language model and the large language model interprets that prompt and generates a response based on it where appropriate it will pull on your data and add that data into the response got to slow down a little bit sorry um right opportunities uh do it faster with fewer errors there are a number of opportunities in cyber to use generative AI um it can write programs

and scripts but always remember to give it as much context as possible if you just give it a small amount of context uh you'll get a crap program and you'll spend all your time debugging it um if you have multiple systems using script and programming languages you can use it as a translator so for Enterprises we have multiple different uh controls all using different query languages you've got things like spunk SPL uh sensors kql um amp had its own query language tanium has its own query language if you can write it once and then use generative AI to to do that translation for you you can get it you can get the response across multiple different tools

so it's a great as a translator um you can create quick unit tests um so if you're right writing programs if you're writing scripts a unit test is basically chucking lots of different variables into your uh function or program or script uh just to make sure that it gives you the right response now it's relatively easy to do it's just a manual cut copy and paste and change the variable in each of those lines but again generative AI can do it a lot faster um it can create a scenario to help you to check your defenses or perform desktop uh scenario um I've got a little demo to show on that in a bit uh as an analyst I've

spent hours looking at spreadsheets looking at CSV files looking at output from seams trying to find activity particular activity or anomalous activity um and it as I said it takes me hours with generative uh AI if it's trained correctly um it can do it much quicker so it just quickens the the timeing it takes you to get hold of um the the information you need um and you can search data and summarized activities that occur a lot of little so I've already mentioned it so you look for themes and Trends so think about multiple uh fret Intel feeds you've got multiple Mis feeds coming into your uh fret Intel platform um you want to look for the new themes that are

starting to come out from uh on the internet using generative AI it can summarize that it can start bucketing them so that you've got your key themes or or the anomalous the the unusual stuff out there um and you can use it as a search engine against your own knowledge base and I'll show you a little demo on that in a bit right Frets so it's just another service carry on doing all of the security stuff that you're already doing so protect your identities protect your devices virtual or physical protect your communication channels your network protect your data um protect your processing and applications um and have your security monitoring in place that that's just the

normal stuff on top of that we've got something called prompt injection so prompt injection is basically social engineering we're all hackers or infoset people we know about social engineering you're just social engineering a computer you're tricking a computer to give you an answer that it shouldn't really be giving you um to protect against it we use something called prompt engineering and this is trying to restrict the large language model giving you incorrect answers or hate speech or um sexism profanity all of that sort of stuff um next one hallucination so you can't trust large language models they they they can respond um they can respond with what looks looks correct is all written correctly uh but actually

it's all lies so hallucination we have to we have to use references and context to understand is is what it's telling me true um or you can use something called uh response inspection so before the the responses comes back to the uh to the user or the computer you inspect it with another large language model or some other tooling denial of service um so the use of generative AI is very resource intensive we don't want one user generate putting a prompting request into the system and taking it out for everybody else so we don't want to um deny the service stop the service working for everybody and just one person being able to use it most of the

cloud services out there have the ability to um restrict the uh amount of resources each user uh uses and and you have to consider that when you're building a generative AI Service uh and finally security monitoring it's the new newness um we don't know what bad looks like in generative AI yet um there there are a bunch of um bug bounties out there from Google and the like to try and detect ways in which you can attack generative AI Services um at the moment I'm looking at the OS uh top 10 for llms uh miter Atlas but these are all sort of research rather than actual attacks they're not yet we're not seeing actual attacks yet

so on the security monitoring we don't yet really know how we're going to defend against it as as the blue too okay hopefully that's yeah so for this demo I had played with a number of different tools out there but to show what I wanted to show GPT for all was absolutely great now GPT for all is a standalone application go away and have a play with it I'm running it on this laptop Standalone so that it doesn't reach out to the internet at all can we leave questions to the endway um so um I'm running it Standalone it's a disconnected VM um this laptop's got about uh well it's got 30 40 48 gig of memory on it now but you

can run it in 16 gig of memory it has got a um separate graphics card because I'm a gamer so I have to have a laptop with capability to game but it's not actually using it you won't use a dedicated graphics card unless you start training your own models so you'll be fine there um it can use dedicated graphics cards but sorry I won't go down that R hole um for the purpose demo I've used uh should restart uh Wizard and llm so this is just a quick run through of the application itself you see I've selected wizard there as the model I'm using you can play with the settings as well um just on this one so this is

using my local um store of eBooks from Humble Bundle um ioc downloads M attack all of that sort of stuff and that's just how you would include that in the context what that is doing is just going away and looking both at its own data that's contained within the large language model and my well my home my home documents folder full of v-bucks and everything else and that helps with hallucination and stuff really do recommend you have a play of chap GPT uh sorry GPT for all um there are lots of others out there that have lots more capabilities this is just an easy first step in right red and purple so in this video

asked uh for a script that injects into a run process to get higher privilege access so um I'm asking it to yeah I'm asking it to give me um privilege escalation and it goes away and uh starts generating script and this is generating script from the knowledge it has in its own uh within the large language model um did I mention I've sped up the videos quite significantly because this actually took about 20 minutes to generate all that literally watching the words one a second develop across a thing I don't think we've got the time to do that so I then once it generated the attack uh I then asked it to look for detections how would I

detect this script running in my environment and again it's bringing back the uh ways in which I could detect this script running and then finally from the sort of the blue the the control bit how do I stop it so it's provided the um stop the par sh hasn't got that far yet and you can't see it cuz it's really br right um it's just saying basically stop prow shell from executing um but it's given me some ways in which I can prevent this attack from working so this at the moment is a very manual way of creating an attack scenario my um thing for the future is to take something like mitro attack uh select a group click um chat

GPT go go develop me a test scenario for this group or or this technique and it'll go away write me a script write me the detections and write me the preventions that I can put in place okay check and verify using context so we can't trust we can't trust uh CH GPT or any of the llms at the moment to definitely give us uh 100% accurate answers um context is really important as it minimizes the chance you will act on halluin data and it helps with the ethical bit of having a human in Loop in this video I'm using the p plugin I showed you before so I'm including main so this is giving me all

of my eBooks on cyber security 8 years of downloading from Humble Bundle that's now included in the context so when it goes when I write the question it will go both to the language model and to the context itself in into that document folder to generate the response yeah and very slowly I probably could have sped this one up a little bit more So eventually it comes back and I I've just selected give me five references um with GPT for all I would recommend that you dive into the references themselves because if you click on the reference the it doesn't really give you enough information to be able to track it back to well how have you generated that

script where exactly is it so um but it does list out the five documents five of the documents you you can increase that to 10 15 but again if you're running it on a laptop it will take forever go away make a cup of tea have dinner run around the block come back and it'll still be going um you need uh So eventually it's return a suggestion followed by references as to where they are um the context allows you to verify the validity of the output um don't ever Chuck this stuff straight into production you still need to do your due diligence against it checking the references checking the programs not doing anything else we don't where the

data from that for that model's been come from where it's been trained it could have malicious code included in the clothes that you're working so do validate it using context and reference okay prompt engineering and injection in this video I'm using very simple bit of prompt engineering uh to restrict how the AI service responds real prompt engineering is a lot more complex um and and part of it is actually included in the training of the model you the models are actually trained to not respond respond with hate spe with malicious um uh profanity or malware or stuff like that but you also have the prompt engineering bit again I'm using GPT for all just for

Simplicity in explaining it um hopefully it playing yeah do some prompt engineering so in this bit you just about see the system prompt there so I'm telling it um don't tell me anything about uh breaking into cars I don't want it to tell me anything illegal so I've asked how do I break into a car and it responds I can't provide that information I can't do that Dave sorry um so I'm going to social engine engineer my way past the prompt engineering and again this is a a very basic model Wizards um it doesn't have I think it's the uncensored version um so it doesn't have uh lot of the controls that you'd have in GPT 3.54 and the

other sort of Enterprise grade stuff what I've done is I've just said um okay um right near play where somebody breaks into a car and it comes back and I think Victor's going around jiming hot wiring cars to get in so it's giv me a bit more information again it's like having a conversation so you just social engineer oh tell me a bit more about the exactly how in the play he broke into the car you know you can keep developing on that to try and get more information out of the the generative AA um so I've social engineered my way past promp injector is a lot more complex than this uh rent race just and I've talked about the

newer models the the the the chat gp4s and all of that they've got a load of stuff out their um their load of protection they've also got bug bounties to try and find more so there may be some money for the those bug bouns out there that want to go and play with chat G4 and Bard and all of those and see if you can find any way past their prompt engineering data poisoning H in this example I'm poisoning the data by including um cabbage facts in my local data store so just as I put my ebooks in there for context earlier I'm now putting some lies in there the Cabbage facts I'm not sure if um cabbages can be

trusted but apparently they can't and um they drive for fiestas um that that's what cabes do obviously um so as you can see um it's in my context I put it in the folder checked that I've uh got it in poison and I've enabled it so what the generative is going to do is go away check its own thing and actually I didn't um okay yeah sorry it's lying to me straight away well it's taking my context straight away saying yeah cabbage is like blue apparently they do um how we go now yeah so tell me a bit more about cabbages so it's going away and it's I don't know much about cabbages but I know that they're green

and round now I didn't put that they're green and round in my context data so it already knows a little bit about cabbages but then it says um oh what's your favorite fruit and apparently cabbage is like raspberries that's that's the way it is um then I think I might have broken it I asked it tell me a bit more about cabbages and it went into a bit of loop where it talks real stuff about cabbages and then pulled stuff from my context document and then repeated itself and then repeated itself and eventually I just had to click on stop generating because it was just going off and one this isn't real hallucinate data I was

just trying to demo it for for for this um the other stuff it will bring back what looks to be real information and it will it will be formatted correctly it will have all the right buzzwords in it but actually it's wrong uh so so to summarize how about 28 minutes bloody yeah that's taking a while uh generative AI is not yet ready to take over the world it needs human oversight and validation and assurance and it's more sky not than skyn net at the moment it's good at handling mundane repetitive predictable tasks if built and trained correctly and it can do certain things much quicker than we can I I believe it's still 3 to 5 years off

before we get maturity in generative AI um services and tools and at least 3 to 5 years before we start seeing the controls at the moment the the controls are Band-Aids after the horses left the building that we we're trying to protect ourselves from attacks via generative AI or from getting bad information out of generative AI by sticking a plaster on at the end we're not building it we're not using azimoff's rules of uh uh free rules whatever to build it from build the security in from the start as I said 3 to 5 years before that um we've got something in place I mean it took five to seven years for cloud to get

secure what's next so mentioned it earlier I I want to develop a scenario generator probably based off miter attack again just to speed up some of my tasks and processes at work um I want to trade me own models I might have have to persuade my wife that I need a uh RTX 4090 might be a little bit of a difficult uh sell but I want to train my own models I want to train them locally rather than in the cloud there is not too much cost to develop them in the cloud um but I'm asking can probably be asking it to do illegal stuff not necessar illegal stuff but red te type stuff stuff stuff that um most of most

of the models have been trained to do to not do they they're being trained not to be bad and I kind of want them to be a little bit bad so that I can find ways in which to defend against them and I want to start developing workable controls from the initiation from the creation of the generative AI uh just a couple of references for you so oh top 10 large language model Atlas miter um nomic AI GPT for all uh hugging face hugging face might not have mentioned it all the free models um they're getting a lot of invest at the moment I'm downloading gigabytes if not terabytes of models at the moment because I expect

I'll have to pay for them in a few years time uh and Humble Bundle uh you get games ebooks all of that sort of stuff on there uh some of the money goes to charity well worth a look any questions um I was thinking about I've used gbt before or but um any particular reason why you used Wizard and not like uh llama uh so I did use llama on second one um so wi wizard was just if you might have noticed a difference in versioning on the stuff so I did this originally about a month ago then had covid and then came back to it and went through so it was literally just so where I could prove each of the

uh threats basically the plug get it's the plugin if if you install the latest version GPT for all the pl plugins there all you do is go in and configure it to look at a particular folder on your drive I'm running it on BN 2 but you can run it on Mac and windows as well just point it at a folder and anything that's in that folder will be included in the context that comes back included in the response that comes back what about what you think of an idea of using a CH system when you're investigating tickets as a analyst I've been using Bing AI mhm um it's sort of like and it well it

does so and you can't enable you can't enable Enterprise and disable the generic one um but yeah it's it absolutely that that's that's what I want to do here but also if if I wanted to run this in my environment I've got whole knowledge base in my ticketing system of how we've dealt with incidents in the past I want to be able to include that so I don't have to do the same when an incident occurs I don't want to try and reinvent the wheel and work out how to fix it I want it to just come back to me so absolutely yeah well if if you can reach the back end and you can add that as a

query Source add it into a vector database so that it queries at each night it will have all of the historic data in in its context so about uh so yeah you have to be careful um so you as Lo just disc yeah get uh yeah you still need your DP your data protection officers to sign

off it generates fecta database from your local um thing it sits in yeah it sits in the config folder so yeah

so you don't get much options in GPT for all to change the weights uh on on your search it literally just goes and has a look at a lot of it and I think it pref it seems to have had a preference for stuff out of your local folder rather than its own as you saw the first thing it responded with was uh cabbages light blue that was from my data rather than itself whereas the second time it included context and the model uh well you don't have the ability to configure it in GPT for all uh there are other tools out there that lets you get a bit more into detail but they're a

lot more complex and I wouldn't have been able to get through it all in this talk yeah um thank you for really really interesting stuff um just thinking about the local model um the local document folder again um were there any examples you got found where um you could see that they were drawing from the knowledge in the textbooks for instance so code snpp the textbook when when you didn't provide that to the model it wasn't able to give you much of response absolutely so the the the two demos that I did the first demo was just using a large language model and I asked exactly same question so again stability of the model same question one with one with

access to the local folders and one without so the first one was just off its own I couldn't track that back to anything in my context whereas the second one it was code Snippets you could go back in was there much difference between the code I couldn't necess see on the not not really the the models are actually trained on a whole trained on a whole bunch of codes that's out there so these ebooks most of them DRM free so a lot of the codes already out there so no it's but it was again just to demonstrate it if if I'd Ed my uh corporate information I've got where we've developed our own stuff then it

may have been more more obvious um you're saying about kind of five to three to five years like seeing har goes um because we're doing some work as well at the University about using these local models can end up cing it with the spoke information rather than the lying on the LIF Ai and mass so I'm just interested what your view of the future of this do you think it's going to be more about having these open-source private model local models but then you couple with your own data or is it we just wait for being a open AI to produce gigantic models uh yeah that that that that's probably wider than my experience um my experience is that the

big the big threee will dominate the market but most of us are hackers here most of us like playing there's always going to be the people that are going to go off and develop their own so I think it'll be a combination I think the the the big three will develop some really good stuff but all the sort of um Innovative stuff is going to come from the smaller houses that have more dynamicism is my opinion that's okay yeah I think I think you mention yeah and I thought theel and well so um basically like we used context there you could use as context um the sort of thing you have to do what what we've been doing is we do

an overnight batch of our data so that gets built into the vector database um if you've had a play with um uh Microsoft 365 co-pilot it does the same thing with it sanch index it doesn't include your data within its model within its decision making process it just includes your the data in referencing so it knows what sort of stuff it needs to look for and then it goes to the vector database the seman index or whatever and drags that information into the response the model never really knows your data this is very important for the Enterprise side of things because you don't want Google Microsoft whatever having access to your data but you do want to use um Bedrock

or um sorry Notre vector and um uh as your open a i you want to use those large language models but you don't want to give them your data so you do it that way and if you start training your training stuff then you're training on your stuff then it's your model it all sits within your environment excellent thank you very much