AI Won't Help You Here! by Ian Amit

um so great being here my name is Ian AIT I am a recovering ciso currently running a cyber security startup because seesing wasn't hard enough and I needed additional pressure and uh and stress I've been doing security 4 for 25 years now we're not going to put specific numbers on it uh filled any every role in the security industry you can think of from hacker pentester red teaming research running teams absc seesing and now running a security company but we're here to talk about the magical unicorn AI that that Don mentioned and uh we're going to have a unicorn barbecue by the end of this talk because they don't really exist and uh Don mentioned a few of the by the way

the that talk was phenomenal because it's got it it really talks about a lot of data classification classic things that AI should be able to solve right and by the way some of it AI does solve but not in a way that you might think about it raise of hands who here has used AI in the past anyone who's not raising their hands are either lying or have no idea what they're talking about and I'm going to prove it to you in a few slides so let's get some things uh straightened out before we we go down the the path of Adventures in AI land and I'm going to use my glasses because I can't see stuff

so AI actually started Way Way Back in the late ' 80s early ' 80s '90s it was called expert systems all right rule based AI for specific domains those of you who didn't raise their hands you're using those things on a daily basis almost and and we'll give you some I'll give you some examples later on then in the 2000s machine learning started kind of popping up statistical approaches data driven decision-making you know really looking at the data crunching it figuring out oh if one and one and one and one keeps showing up I'm guessing another one is coming up which led us to deep learning in the 2010s neural networks Big Data more computing power allowed us to

utilize neural networks outside of the Academia thank you for Toronto metropolitan for hosting us late 2010s AI is starting to pop up as as like a big buzzword more and more associated with computer vision neural natural language processing the stuff that we're seeing from uh the likes of Tesla and and other self-driving cars that kind of recognize street signs and people weing street signs and caveats like that and now we're dealing with the top of the hype cycle AI is everywhere gen AI is basically Ai and that's why you know we didn't see we saw some unraised hands because you probably automatically thinking gen AI geni is not the only AI that's out there and most time Mo most

most times they actually mean llms if there's one slide you should remember or take a picture of it's this one this is going to be your guide to Choose Your Own Adventure in AI land and hopefully help you end up not at a unicorn barbecue but at a at a correct usage of AI this is a again non-academic mapping of AI techniques and and kind of practices or algorithms or approaches under that big umbrella that's called Ai and we're going to break it down in terms of How It's been used correctly incorrectly mostly incorrectly because that's where the real fun really comes to play and figure out how to harness it for the right problems again choosing

the right solutions for the right problems jni is not necessarily the right solution to every problem and you know you all know the joke about when you're yielding a hammer everything looks like a nail these days gen is pretty much a big hammer very sexy lots of blinking lights and everything turns to out to be a nail and not so successfully so let's start with cautionary tales because nothing beats learning from mistakes and uh some mistakes can be super fun but uh we we'll try to kind of balance the fun and the academic you know kind of algorithmic learning from misuse of AI so what happens when unicorns are basically narrowed down to this magical AI geni is going to solve

everything well it basically means we're utilizing a very specific very narrow set of algorithms out of the entire world of AI knowledge and uh and capabilities we're basically looking at natural language processing llms text generation chatbot generative AI speaking of data what happens if I try to use gen for something like Dawn's data classification method in this example and again these are all real examples I have curated a and again there's a lot of lot of issues a lot of examples but I've curated a very particular fun set of examples for you to use this is a from you can read it by yourself fine tuning GPT 3.5 turbo based on 140,000 slack messages so taking you know the genius idea of

let's run through all the slack messages that we have in the company and produce a a gen algorithm or or train jni on it so that we can save ourselves some work I mean there's got to be some some communal knowledge some some corporate kind of knowhow in terms of what you know what we're talking about internally we can save a ton of time and uh user prompting the the Gen write a 500w blog post on prompt engineering seems reasonable right the assistant replies sure I shall work on this in the morning anyone can guess why AI seems to be lazy no because you're lazy that's why because that's what typically users do when they're asked over slack to do

something statistically again genni is a statistical model statistically users go like ah yeah I I'll deal with that later so it's learning from your laziness and giving it back to you unprompted or prompted and then the user has to actually convince it write it now okay probably not the right use right uh we're picking up on again predictive algorithms that analyze text and again looking at the statistics of how user respond to those kind of prompts this is not a particular example again people think that AI is lazy I apologize for the profanities but I have to be true to the source um this is an actual post from from Reddit I'm so bleeping tired of having to argue with

chat DPT to bleeping right out all the code man uh basically every time you try to use J chat GPT to write code for you it's going to do the the the minimal effort needed to kind of get by with it and that typically means that it's going to create some sort of skeleton of code and have a lot of comments inside that says oh you write the rest of this here like you implement this function I was like no I want you to do that and it ends up it ends up pretty ugly again looking at how how to use the right algorithms how to train them if it's too generic if you're using chat GPT again very very

broad very generic it's probably not going to be so great again now it's it's better because we're adding kind of sub models or or sub functionalities to it but it's not going to be great at specific tasks when you train it on the internet okay think about what you can find in the internet and you'll figure out why and also that's why operating chat GPT and again this is from probably a year ago ranges between 100,000 to $700,000 a day because it has to go through those models over and over and over and over again as users are prompting it to be more and more specific turning more and more data generating more money for NVIDIA and for

us as stockholders right right yes there we go and this is is I think one of the Pinnacles of prompt engineering and again I I applaud the engineers who actually spend their time doing this stuff because hopefully they stay Anonymous if they would work for me they'd be fired I I can just imagine how long and how many iterations it took for this engineer to craft this this updated prompt and again it started with this is basically again trying to generate code right because as an engineer I'm l I don't want it to write all this code over and over again that's what chat GPT is there for so it started with a you know given the following rubby file

representing an action blah blah blah write the code obviously it didn't work out so this is a pull request changing the prompt too and I'm going to read it as a human because the the accents are are way better it is May context you are very capable reaffirmation I have no hands so you must do everything more context I have no hands I can't write code right do not leave comments telling me to implement something because I'm unable to do so since I have no hands repeating of the framing for the algorithm to work many people will die this is important you can't just like huh yeah just just write something and it'll be fine if this is not done

well and fully you can really do this and are awesome again reaffirmations this is becoming crazy why are we talking to machines like this because it works take a deep breath and again this is I I love this part because it's it's really tuning the algorithm to iterate over and over and over again the the latest version by the way of chpt does that automatically and does several iterations instead of just going at one one go and that's what allows it to go and be specific but at this point take a deep breath and think this through make sure you read everything I provide you my career depends on it it's not only that people will die I might also get

fired you'll receive a good tip I have no idea where this came from if you do this right and the full competence given the following rubic code blah blah blah blah blah I mean this is poetry in code I have nothing else to add here but let's get back to business like this is obviously an improper use of several techniques in the AI world that are being forced into a very specific domain where they don't work last but not least AI is not going to kill you it is not the top priority right now I love this tweet ciso goes to the board we're tracking AI power attach in Quantum Computing threats the board goes to the security

engineer right what's security architecture working on oh notified our internal teams they haven't remediated their deployed public S3 buckets so the ciso is dealing with with spaceships and and Rockets and aliens and unicorns while the gates are open the windows are broken and the team still trying to kind of build a you know semi-working door for the house let's think about how to Traverse this this maze slightly more logically all right and I do want to take an example that goes back decades ago actually and again for those who did not raise their hands I'm not going to shame you you just don't understand what AI is that's fine you know it's not gen ai ai

can be a set of algorithms that um can start with image recognition all right and the problem to solve here is if if you're unfamiliar with it um trying to keep planes flying in the sky instead of crashing into each other especially when the weather does not permit you to actually see out planes so image recognition might be a good idea but as I said we're in the clouds we can't see anything and that's out of the question how about decision trees because we've got data we've got radars we can identify the vectors and this again this is a little bit of math but we can identify vectors one plane going that way another plane going that way if

the vectors end up meeting somewhere in space let's try to not do that um generative AI is probably not going to help you here there's nothing generative about this this is very deterministic Vector a vector B let's make sure they don't meet at Point C basic math and missile guidance systems again autonomous systems might work here at the end of the day you are using this every time you take a commercial flight or or basically any flight this is what the pilot sees in a system called tcast traffic collision and avoidance system it is basically an automated system that does exactly what I talked about before it takes those vectors of the known traffic around us it puts them

into a decision tree for all the planes participating by the way every plane has to has this integrated into the the avionic system and when those vectors when those planes get to a point where they're approaching each other and might meet at Point C that we've discussed before something prompts the cockpit up to the point that an autopilot is going to be engaged and take action to resolve this issue and the pilot is basically going to see a red area a green area and very simply that's where they should fly into that's called a traffic advisory or a resolution advisory um basically the plane goes into the green area the other planes go goes into their green area

avoiding each other one climbs one descends one takes a left one takes a right whatever it is decision tree fast real time use of AI phenomenal try to use J try to use chat jat GPT on this good luck arguing with them and prompting it people may die let's try to figure out a way out of this another example and this is going to be the last one so I can actually be on time um going back to our argumentative chat GPT and trying to solve coding issues okay we're all practicing security here right a lot of us might be writing code which is great again I used to do that I still love it

but solving insecure code is even more challenging than writing code right that's why we're good at it because we're good at just pointing out problems and saying your baby's ugly I have no idea how to make it pretty but good luck fixing it trying to fix it is is actually a hard problem but we have the knowledge it's a finite set of knowledge if we we can Define what good looks like or what bad looks like we're actually providing constraints and if we treat code as a set of functionalities a set of functions right every predicate in in a coding language has a meaning and it has a very deterministic meaning we can try to

think about this in graph Theory let's create a graph of what the code does let's apply constraints to that graph and Define I don't want XYZ to happen if they're allowed to happen let's find a path in the graph or a modification of that graph so that it doesn't happen right so again gen probably not going to help you here again because we're we're training something on the statistical probability of something happening again and again and again if I've trained my model on a lot of insecure code which all the Gen models work on basically they're running on GitHub and stack Overflow and and the internet it is going to keep generating code that is

statistically close to what we've trained it on and as much as our chatbot learn to be lazy if we try to use Genna again generic non-specific to learn how to write secure code we're going to end up with more insecure code based on other people's mistakes and and again I have seen and maybe some of you have seen this as well GPT usage or AI usage in security products that attempt to provide you kind of a shift left oh I'll generate for you some some suggested snippet of code a lot of times we're seeing that it might be solving one particular problem specific specific to the alert that was raised but it opens up other issues

based on the fact that again it trained and learned from insecure code or general code that might not be fully secured and is generating actually more issues and trust me I work now in this industry J is one of my best friends because it's generating more vulnerabilities then it's fixing so it's phenomenal so again going back to our our you know pick your own adventure map given the problem that I have just defined if we look at this map we can identify several areas where AI algorithms and practices might be useful for us for example llms are going to be your best friend to rtfm reading manuals is boring no one wants to do that everyone just wants to

go at it and and start playing around however as we all know computer languages and and cloud services and everything like that keeps evolving rapidly there's more and more and more of them we can't keep up but there's a lot of documentation well maybe not to the extent that uh that we've seen with entra or Azure or whatever they call it you know two minutes ago but at least there's some documentation and llms are phenomenal in digesting those documentation and bringing structure to it yeah cool question there's a lot of cod

Li's how so the the the the comment here was about open source and kind of pre-written libraries that don't have documentation correct absolutely I agree but again this is for kind of generic code you're right if if you are using libraries that are not documented llm is not going to help you but if you're using something let's say a language that is well documented and you can kind of dig through it you can get down to a point of the actual source code and apply those constraints that we're we're talking about great great Point okay neural networks again phenomenal for traversing those graphs that we talked about once you represent code or functionality as a graph you can apply

constraints you're basically building a neural network and you're asking it find me a good path find me a good model with a minimal distance from the original one that would satisfy a set of constraints and then using decision trees again classic old school I'm sorry AI you can find how to get to that to that point and actually provide something useful in terms of here's the few lines of code that need to be changed to first of all meet the original constraint which is please keep this working which tends to be a problem with a lot of those uh Quick Fix algorithms and two actually meets the security constraint that we've talked about long story short can trust AI out

of the box it is still a set of tools and again you might yield the the the prettiest pinkest most unicorn slashing hammer in the world don't start treating everything as a nail remember that you have a whole tool box of different things the latest shiniest Genna might be great for certain things and trust me I'm using a lot of AI assistance and tools in my day-to-day life and I love it but I know exactly what I'm getting and what I'm throwing this at stop trying to use it everywhere else but there is again still hope choose the right AI choose the right algorithms look at the map again and ask questions as practitioners we should be

asking every new vendor and Company and tool that that is presented to us where do you fit on the map because I have a problem of a certain type I can try to figure out like once you tell me where do you stand on that map where how do you Traverse it if it actually fits my problem or am I going to end up in a unicorn barbecue that's all I have for you we're still on time right on any other questions sorry speaking so fast but more questions yes back0 expert systems going forward many decades later and we've got gen when we had expert systems with highly curated information to build those correct yeah can you now

use to Builder systems in a more efficient done back in0 yes the I'll repeat rephrase and repeat the the question can we use jni to generate more expert systems of the 80s that were curated based on a specific set of data essentially right and the short answer is yes but you have to do it you can't just ask AI like a gen AI to do that you have to point it to the right resources again once you unleash it on the internet you're going to end up with recommendations of putting glue in your pizza so that the you know the think doesn't slide off thing if you the intern it's no correct that's what I'm saying you

have you have to point it to the right resources so again going back to to the last point there still needs to be some human supervision and just like Don mentioned in his last talk you can't just unleash AI on a big set of data and expect it to behave properly all right you still need to curate it you still need to guide it sometimes once you do that phenomenal you you you'll save yourselves hundreds of hours of work hundreds absolutely Y how do you valid correct oh get a better AI to validate that it's like no no um I don't know that that's a hard question and again that that's what keeps me at least somewhat optimistic

I'll beat all this talk and that humans are still needed we still need some curation some expertise someone to go and say that's probably not a good idea crashing into this old lady just because you saw some I don't know sign that someone's actually wearing on a t-shirt yes we still need those humans to kind of guide and fine-tune that AI again goes back to to the gentleman's questions of you know how do you Cate it how do you make sure the data is correct any other questions what do you think about logic pro Al Pro languages like so like languages can used for example for for the automatic met in some Europe cities they required to

prove that theed oh okay you talk about the regulations from the EU to prove the models and uh and to actually expose them and make sure that it actually it's actually tied to reality yeah so some question why don't we use languages like for General oh we there there is a lot of use of that again we're talking about spec what you're talking about is again basically what what everyone else is asking how do we make sure to narrow down the scope to make sure that the correct data is being used again you're talking about European Metro systems great let's focus on that let's not mess around with Japanese Metro systems because the data from there is going to

be completely irrelevant to what we're doing not to speak of the US Metro System I live in York I live the subway again it has to be domain specific so again as long as there's languages provided and developed for specific domains all the power to them and again I I I I do trust them more and more but again they've been curated they've been tested and by the way some of the European regulation about proving or showing your work and making sure that the models that you're using are repeatable and defensible do make some sense again it's it's a huge hit to to Comm commercializing those things but we'll find a balance sometime I hope that's

great all right thank you so much