Jason Haddix Interview

I I loved that you broke down using a framework that everyone's familiar with mostly familiar with Metasploit to build primitives for AI assessment. Now if you could talk a little bit about what those those primitives are in terms of you know and how you're using them in a in a like Metasploit like format. >> Yeah, absolutely. So you know one of the things I talked about in the talk was just like how unstructured and how non-existent training was you know even even now in prompt injection which is you know one of the main kind of attack vectors for at least LLMs and systems right now and so you know we started looking at jailbreaks the jailbreaking community

and then you know the white paper community who had been studying prompt engineering tricks and it was getting too much like you're saying it was just like there was no structured mental model to look at it and so I don't know I think I think this is a trope inside of cyber security but I think the idea actually came to me in the shower and I was I was like I was like okay what if we approached it like Metasploit did where Metasploit took exploit generation and broke it down into pieces of your payload you know utilities to help with your exploit and then you know what HD and Metasploit was the ability like once you once you carve up

exploitation work into those primitives then you can swap things like if you need a different shell and Metasploit you just choose a different shell right and if you need a utility to help you do something and generate the exploit that tool's built into the framework and stuff and so >> How how important is it to have a proper system architecture with regards to you know being able to understand the assessment? >> It's so important. It's and so you know like we only have like you know 45 to 50 minutes to do the talk right but if if I had more time I would talk about how important the the initial kickoff call is and getting the architecture and we

have a we open source actually for other consultancies to use a starter questionnaire and it like it goes over like asking them about what models they're using what the architecture is, if we can get a diagram, you know, what guardrails and firewalls they're using, what prompt caching they're using, what routing they're using. All of this stuff just because because of the nature of AI testing, it takes long.