Securing AI Workloads in Kubernetes: Lessons from Scaling Startups

All right. So, I think we can get started. We we're like one minute short so we'll just shut the doors and then we'll get going. So everyone here, thanks again for coming. I've said it what three times now. Um so wanted to say uh one thing before we start. Um I usually like to find out who has Kubernetes experience here. All right. Good. This is awesome. Cool. Because sometimes I feel everyone has Kubernetes experience but have you actually ran production workloads? All right. Two. Three. Yeah. I know you already here. Fine. [laughter] All right. Good. So, what I'm going to be talking about today so that everyone's aware is really how to secure AI workloads in Kubernetes.

Remember, AI is just another service here. We don't really need to focus on the AI aspect of this. This will be part of the talk because that's the hot topic, especially for a lot of attackers. So, when we're thinking about Kubernetes, we always want to say secure is the default. That is the most critical piece. One second. There we go. So, when we're talking about Kubernetes, um, what I want to kind of go over first is this talk is for these people right here. You're either a leader, you're a practitioner, you're a security-minded builder, or team in AI. How do you actually think about this and scale up? So with that, when we're talking about how to do a

startup and you running, you're running Kubernetes. What's the most dreaded word right now? Let's add AI to it. So with that, my name is Chris Mayor. Um, I've been doing security for since the 2006. Originally started as an auditor, pivoted into uh head of cyber security for multiple startups. One in particular right now is focused on how we're actually deploying Kubernetes so that we can scale up developers, but most importantly making sure that they're secure by default. So when we're thinking about product market fit, that is something that really bothers me the most. I have to run systems that have to be able to not only be secure, but keep revenue going because we've had multiple

bankruptcies in startups. So when we're thinking about AI or any kind of new service coming in, how do we make sure that security is available by default and not just bolted on aftermath? So security and Kubernetes is not a process. We want it to become infrastructure first and foremost. And that's what this talk is really going to be about. We're going to be going over not only how to do the security aspects of it, but demoing it as well. So what do we want to think about when we're doing these kind of things, right? A startup by default has four aspects that I've identified in security. If we don't optimize this for we don't optimize our company for speed, what

happens when scale happens? So first problem we have is identity sprawl. And we're talking about identity sprawl. You need to think about is what is the service account being able to access in the cluster? How are your OIDC tenants hooked up? Do you have multiple identities? Because every everyone talks about zero trust. We can do the authentication, but how do we do the authorization once you're in that Kubernetes cluster? Secondly, flat networks. In Kubernetes world, everyone loves flat networks because it's just easy to deploy. You have one CNI and you're able to deploy and scale up. So when you introduce a service mesh, what happens if you do it postfact and you have all these services

running? Things are going to break right off the bat. So another problem that we want to face secrets. So we as security professionals always say where's your password manager? How do you store your tokens? Developers are like tell us we need help. So they're just going to deploy and by sometimes not even on by by themselves they just deploy malicious code or security sensitive information in their npm packages. How do we stop that from happening? And last implicit trust. So in Kubernetes everyone thinks you're in the you're in the cluster it's secure everything's trusted. How do we actually protect against that? So that's what this talk is going to be about. when we're doing a startup. These are

the things that really plagued me through my many years of doing this. So security debt is invisible. It just happens. If you are a successful startup or even a small business or if you're in a corporation and you don't have the security background and you're just deploying a new MVP, how do we protect from invisible security debt? And I always like to say we want to destroy it to deploy mutual or MTLS. We want to be able to deploy network policies. How do we do this? AI is here. And not only not only that, how do we actually protect not be beyond AI but also the microservices in general? All right. So AI really is not the problem. We are

usually the problem. We have a business. We want to run something. How do we take care of this? So there's three core components of what we want to think about with AI when we're deploying stuff inside of our clusters or inside of our business. One is your models. That's your proprietary information. Don't think about it as I'm just deploying something through cloud code or a new codeex or something from hugging face. Sometimes your data scientists are building models that have sensitive information so that it can run inference on your workloads. So that's the first thing that we want to think about when you're talking about models. Secondly is prompt. Everyone thinks of prompt injection but that is a risk but prompts

in general. How do you actually protect information inside your cluster in case you have something scraping or listening? How do you actually protect that information being actually being pulled out from egress perspective? And third, your endpoints. So when we talk about endpoints, we're not talking about your laptops. We're talking about APIs. We're talking MCP servers. We're talking just uh a regular API inside of your Kubernetes cluster. Those endpoints need to be protected first and foremost because if you don't, what's the point? You're you could be going under or most importantly have a security breach, which is what falls in our lap a lot. [snorts] So the AI threat model, four concepts that you need to think

about and we're talking about this, we've been talking about this for a long time, at least I would say the last six months because AI is starting to just change all the time, especially if you've been working with cloud code. So bear with me here. Prompt injection number one, if a malicious intent is performed, what are we trying to get? Are we trying to get proprietary information? Are we wanting it to trigger something to cause another effect downstream on your inference model? Or do you want it to actually hit an API so that you can actually send requests to another downstream service? Because when you do that, you get run into a token abuse. Then token abuse,

we're not talking about API token abuse. We're talking about actually model token abuse. This can cost money. We've had multiple times where our engineers deployed something that was a little too fast. And what I mean by that is they ran a crown job every 5 seconds to run some kind of tax simulation. Well, if because we had specific gates on the tokens, it didn't cost us that much. Think about that. Number one, you're running a business or you're running a small business. $10,000 bill next day is not pleasant. that literally will cut underneath a lot of your your staff and then not only that your your core fundamentals. Thirdly, model scraping. So model scraping is a very interesting concept

here. How do you detect it? Which we'll be going into. But most importantly, when we're doing model scraping, what are we trying to pull out? Because people could be actually training their models off of your models. So if you have a rogue agent or a rogue service inside of your Kubernetes cluster, it could just be gathering information. I've had a very good pentester in my lifetime teach me that Kubernetes is very vulnerable if you don't protect it properly. And they've done these kind of attacks against us and simulated it and it was something that we highlighted. It was like that's something we need to think about. Fourthly, sorry about that guys. I should have put the power in my

laptop. Keeps timing out. So last one is lateral movement. That goes back to your flat network. Like vulnerabilities happen. We know deployments happen. What happens when those services have something that is able to pivot into another model or another API that you're trying to protect? So AI is very good at this and humans have been doing it forever. So this is nothing new especially inside of doing any kind of work. So with that, this is the fun section, tools. We always talk about Kubernetes, all the hot stuff that's coming out here. I'm simplifying it for you guys. So our company um Eboard Technology, we align with the CloudNative Computing Foundation. So I'm going to say a lot of CNCF. That's what

that acronym stands for. The CNCF is an open-source continuum that is allowing individuals to use Kubernetes software that is native and is backed. So Borg which was the original code name of Kubernetes was one of the proc was one of the applications developed by Google and they released it to the CNCF [snorts] because they were not only supporting this for Google cloud but AWS AKS open shift you name it they needed someone to look over this they started to work on this foundation to be able to give you these tools. So, if you check out the the link in the the bottom, the CNCF landscape, it's a little overwhelming, but it's really great if you're thinking

about running a business or even just training yourself. It gives you the tools out there. So, with that, I broke it down into multiple columns, and I really want to highlight this because we'll go into this um a little bit later on like specific tools. So, the first thing on the top left is we have our networking and traffic. So, I've been talking about lateral movement and protection. So we're going to be demoing using a CNI which is a container network interface that is going to be using psyllium but I prefer ISTTO because ISTSTO has a little bit better ecosystem where psyllium is a little bit finicky. The big difference between the two is with psyllium you get ebpf native

observability right off the bat. So you have really fast Linux or kernel level activity being able to observe. Secondly, going down the list is our security policy and secrets. That's the highlight of course everyone wants to talk about this. I'm just just tap into a couple of them that we use in the company isert manager and external secrets. So we already talked about how do we protect our secrets? How do we protect our sensitive information? How do we do mutual TLS? Well, this is how we do it. Cert Manager gets plugged in. It gets plugged into whatever your certificate management supports. It supports an Acme protocol and you'll be able to easily rotate certificates on

the fly. Do it on an hourly policy. You can do a 30-minute policy. It doesn't matter because the service mesh is managing it for you with that service. And secondly is the external secrets. This can plug into your hashb vault secret manager um on Azure or AWS, I forget the the terminology, key vault on on Azure. So always think about how you want to be agnostic when you're building tools especially around Kubernetes because stiff stuff changes someone throws AI into you how do you actually pivot couple other things I like to highlight is the the third one over so cloud infrastructure I use crossplane if you never heard of crossplane it is an infrastructures tool infrastructures

code tool set that will allow you to write Terraform syntax and integrated with crossplane and it has a reconcile loop that sits there keeping stateful management of all your Kubernetes resources. What's nice about this that becomes your secondary security engineer because you instantiate code in infrastructure so that you can say this is what I want deployed and how I want it to be managed and if an admin logs into the console and makes a change crossplane will actually roll it back for you automatically. That's one that I highly use for like running my identity system with key with keycloak. This helps me allow developers to actually do click ops to do their cert certificates and when they're ready they put in code

and crossplane manages for them long term. Going down the line I'll skip over a couple things but one I really want to highlight is your observability tetragonon that is a psyllium based observability tool. So you can be able to look at ebpf traffic at your endpoints. So on your container, if your AI models are acting weird or uh a service is hitting an API that's a little bit too aggressively, you can actually alert and actively block if you choose to through policy management because you have that as well through infrastructures code. Um and the last one I'd like just to highlight is Argo CD. That's how we do our continuous deployment. uh that is allowing

developers to then only focus on using a YAML file instead of figuring out how to deploy all these things. We have a YAML file that is very simple. What do you want your apps to be called? Where do you want your containers to be called? And where what network policies do you want to set? We do this all behind the scenes for them, but they just say what things do we want to communicate with. So here's the security model that I like to use. We just talked about tooling. Well, this is how we actually apply it through a shared model. If you can't really see the slides in person, I apologize the the the lighting is a

little bit different here, but we want to do a shared security model that is very famous through AWS, through Azure. They all have this. So, when we're building companies, we just focus on the really top two layers. If you do your Kubernetes clusters correctly, that allows you to protect your data and your identity. Everything else you can actually pick and choose by using those CNCF tools that will plug into your cloud vendor or keep it open source so that you can pivot your business around. And this will also allows you to not only be successful in security and being running infrastructure, but if you're trying to raise money, you can just take that code and move it around from

company to company without any worrying about licenses. so that your next company that says I want to have X um identity system or X external secrets system you can just plug it in you're ready to go and you don't slow down um the the company and that is something that's really really critical here. All right so the approach how do we actually win when it comes to security? It's because if we do it at infrastructure level, we're unstoppable. And why we want to think about it that way is that when we're deploying our infrastructure, it's not just code here. We're talking about the whole Kubernetes cluster about how you're protecting number one, your identity, your east

west controls, that's the protection of your flat network, AI specific isolation, very similar to actually how you secure your applications. Now, we're going to give you some examples of this and observability. what matters of what we're observing because everyone talks about SRE programs and data dog or whatever you want to use. This is the stuff that we're going to be going over about how to protect this in Kubernetes with AI. So main thing takeaway here, no identity equals no access period. That's how we want to think about this. How do we actually do it in practice? So, I've been talking about mutual TLS. That's the MTLS aspect here. MTLS works in a very nice way here. If

you do this correctly in your service mesh, everything's encrypted. Number one. Number two, you can get prepared for zero trust because you first have your identity system that gives you all of your tokens and then you have your authorization piece. That's the hard part. But if you have MTLS set up, your developers would be like, I want service A to talk to service B to route traffic to service C. They don't have to worry about certificates. They don't have to worry about JWTs or jots. They just focus on deployment of code and we do the rest right here. So remember when mutual authentication means just certificate to certificate. If you don't have one, you're out. You're

automatically denied. And what's cool about this is we were talking about the external secrets and and search manager auto rotating certificates. This will do it automatically for you with ISTTO as well as psyllium. I'm going to talk a little bit more ISTO right now, but we'll get into psyllium in a demo. And then why that's important is transparent to all of the developers. Developers will always circumvent you because they're trying to do their job. And now with AI, they can really do their job. I mean, it's it's unstoppable now. So we want to make it very easy for them to be successful. The more successful they are, the more secure the world becomes. And with that, we're talking about

psyllium and ISTO. The two big things I want to just talk about here is EBPF native issium. ISTO, think about it as a bigger ecosystem, more uh it been baking a little bit longer. Both of them work really well. They actually work in junction with each other. So if you plug in a psyllium at a CNI, you can still install right on top so that you can uh feel free to protect your environment. So when we're talking about east west traffic, if you're not familiar with that term, just think about it as like how network traffic goes on a flat. It's all we're doing, east to west. So when we're thinking about that, how do we actually protect it? So

there's going to be two things that I want to highlight here. When we're talking about network policies, flat network is bad. Explicit trust is good. Default deny period. We do that if you have everything properly set up at the layer 3 level. You put your network policy that is native to Kubernetes. You don't have to add in selium. You don't have to add into. You get that right off the bat. But you ask me, I've been talking about how to protect developers, how to protect AI. Well, we can circumvent that now. Correct. So you want to join forces here. You want to apply that authorization policy so that you're able to protect your service mesh

and your network and your developers and combine the two. So a little bit of layer three and a little bit of layer 7. It's like oil and vinegar but in a very positive way makes balsamic vinegar. Okay. The fun part. How do we isolate things? So we've talked about a lot of different tools. We've talked about how we deploys psyllium and ISTSTEO, how we're doing network policies, how we're doing identity. Well, we need that all combined because now when you have a really easy deployment model for your developers, they're giving you access to something richer than just, you know, doing secure code. They're giving you a business. And with a business, we're all successful.

Three principles we want to think about. Number one, model service isolation. Think about this as we are restricting ingress. And when we're talking about model isolation, how we do it with our company is that we are testing a new alpha release of an agent that basically looks for geo tickets and actually does the work for us. It's been really successful so far. But the good news is my infrastructure team is very very security focused. They're they're the ones that really create all this stuff. They have basically be able to do deploy any kind of model inside of Kubernetes and it has no internet access. It's totally blocked because we block at the service layer the mutual TLS aspect.

Second, the network policies. And third, we only allow it to be able to communicate with its inference APIs, its MCPs, so on and so forth. So that is the only thing it's allowed to communicate with. So if anything deviates from that model, we're able to detect it. And what's nice about this is that we don't have to worry about namespace isolation in Kubernetes. But when we deploy everything inside of that namespace, we're saying from an observability perspective, if anything else goes in that AI model um environment, we automatically detect and destroy because ID potency is very important here. We can be able to just create and destroy at will. Second is your prompt and data

flow separation. Think of as a proxy layer. Proxies are very difficult historically. Not with STTO, you have a service mesh is your proxy. So you're basically saying because you can actually monitor activity through all of the observability to say if I have my jot to allow access to this system allow traffic to flow if not can't say can't allow anything else to do. So it gives you that checking for your your header information your routes so on and so forth. Then last is lease privilege. I think we all know what least privilege means. We've kind of talked about this through this journey about all these tools we're using. If we do it right, you get automatic isolation so that you

can be able to give your development team, your leadership, and most importantly, your customers satisfaction that you are a secure company by default and most importantly, you're able to help them scale and grow their business. Okay. What matters when we observe stuff? We really overthink this sometimes. It's not an easy task. I'm really not the best at observability. I throw it to Data Dog or I throw it to um uh Prometheus. I throw it to Graphfana. I just pick whatever the community does because it's something I don't have patience for anymore. As I've been getting older, it's just some one thing that I just lose patience on observability. So, these are the things that we want to really highlight number

one, especially when we're doing um AI work. So latency anomalies, if we detect any kind of weird anomaly that deviates from what you're basically looking for from uh health, jump on that because that could mean a couple things. Your resources are bad. You could have an outage or you can have an attacker basically um overloading the system because when you have Kubernetes, things autoscale. So as things autoscale over time, that could be a latency check that you can actually detect. Second is token spikes. This is a really fine balance here. Um I usually put caps on API tokens that grant token access to your AI models or your AI services or your data science team so that I know that their budget is

only allowed to spend for this service say $1,000 a month. We'll say that's a good way of blocking it. But what happens if that $1,000 gets ripped up in like 30 minutes? this is what you need to detect on. So you need to think about guards rails. The it's it's really important to have that fine balance. Third is our access patterns. What is happening in your cluster? Very very normal. But with AI, everyone's accessing it. Everyone is doing things. There's no logs coming off of your model, but your service mesh is showing logs. Your API servers are showing logs. This is how you're able to detect those things. And then on top of it, we talked

about Tetragonon as well. You have that underlining activity that's happening in your container. See what's going on there. And then denied request. If you start seeing a ton of denied requests, something's weird right off the bat. Either a misconfiguration or a potential breach. It's usually one of those two things I found. Uh thank god the breaches hasn't been too bad. It's mostly misconfigurations all the time. Um because we get a little overzealous about locking things down. This is how we're able to detect it. and basically move on. [snorts] All right. So demo. All right. So what we want to talk about here is I'm going to show you the GitHub. You can go to it

and and play with it yourself after this after this talk. Um the demo is showing you a kind cluster that's running Psyllium and we're going to be deploying an AI model and showing an attacker as well as a trusted client. when we're talking about this, if you want to add on to everything we kind of talked about, take the code and play with it. I literally prompted it into existence, you can do the same exact thing. I have all of the details there. So, before we go into that, let me uh just bring up GitHub so that you can see it for yourself.

If you go to my repo uh presentation-security or secure- ai workloads, we have the we have the presentation with a little bit more extra text so they can take this back and think about where to go and how to do these things. Secondly, you have the video demos which we're going to be showing you. So you can see when you're if you're having problems yourself running this locally, you can then see what it actually would look like. [snorts] And then third, as you see with with the codebase, um you'll be able to see how I put together the brand, the strategy, everything that goes into building these things. So think about this also when you're putting together

presentations as well. All right, so let me bring up the actual demo.

All right. So the first part, what you're going to be seeing here is the actual installation. This is the kind cluster spinning up. This has really nothing to do with what we've been talking about. Just showing you how when you deploy these things, how you can run these things locally and easily be able to manage your psyllium installation. So in a second, you'll see the status that shows everything is healthy.

There we go. All right. So, after this run, this takes usually like 3 to 5 minutes, maybe 8 minutes depending on your machine. I sped it up so it we don't waste time on watching installation guide. What we want to show you here at the bottom where it says make status the you have multiple clusters. You have your insecure cluster and you have your secure cluster. What's the difference? The secure cluster here has your policies that are automatically set up. Your insecure cluster has nothing. So this is what we're talking about in having your secure code ahead of time and having your infrastructure secured at the same time. And you're going to ask me like, well, as we look at labels

and we look at how these policies are are driven, can't developers just change these things. And one thing I didn't highlight here is there is this methodology called an operator inside of Kubernetes. This is your hidden gem that basically is a it sits as a process in Kubernetes that has this thing called a reconcile loop. I think it was just a big for loop that just takes commands at all times that is doing the enforcement for us. So as the infrastructure team, the security team goes I want to have this completed they enforce it and the developers don't even see it. They just see it at a YAML layer to say I want to deploy my

application. So by default everything is blocked. they only are allowed to expose their their systems to a identity token or a port depending on if they're communicating with like a database or an API service. Okay. All right. So, what you're seeing here is going through setting up your exposure to Hubble. We haven't talked about Hubble, but what you're seeing here is just showing you like the Kind cluster in secure demo mode. On the top left, you'll see your context, which is highlighted right there. Shows all of your trusted and untrusted applications are running. I'm switching to my insecure demo, and you'll see it's exactly the same. No changes. As we move on, what we're going to do

now is port forward so that you can see the Hubble UI. Hubble in a second, I'll I'll go over it with you. I'll pause and we can talk through it. as well as port for so you can actually see your network traffic. And so right here, this is Hubble. We, as you see right there, we didn't have any data just yet. And that's important to understand because there's no workload traffic. So how do we see something? So keep that in mind as we uh run the attack simulation. All right. So as on the top you're going to be able to see the psyllium Hubble activity that is going to occur. So what we're showing you right now next is

taking an insecure command and we're going to run it right now. Do you see no logs and everything is easily be able to communicate with each other. No fuss, no mus. And that's the bad cluster. Well, there's no really good or bad, but the insecure cluster, let's just keep let's keep it real that way. So now what we're doing is turning on Hubble to say go and look for the secure cluster and run the same command. As you see, first something for this trusted client was able to communicate. Then all of a sudden, we see all of our logs. So what you're seeing there, let me pause it so we can talk through it a little bit.

I ran three curl commands. one from a trusted client that said I have a proper label, I have a proper jot and I was able to communicate forwarded traffic no problem and I got a response at the bottom where it's a simple model saying like hello from the trusted environment your your account is 42 so on and so forth but then you see timeouts because those are the insecure clients or the attacker and right off the bat this is what you're going to be able to expose automatically you guys these are flow logs essentially, right? But these are service mesh flow logs. And what's nice about that is that they're structured. They're standard so that you can move

this to any cloud you want and not have to guess or retool any of your tools. You just say forward this traffic from Hubble's container to your SIM or however you collect your data and you now have this be able to be reused multiple times over multiple clouds without having having to guess how um you you know spend money I guess. All right. So next as we go through show what we just talked about here you're going to see Hubble. So if you are not into CLIs, you now have a beautiful UI that comes native out of the box with selium. And what you're seeing there is all those logs that let me just move this a little

bit. Let me pause there. So what you see up here is my trusted client. And what are you seeing? You're seeing on the bottom the forwarded logs. You're seeing what source IP is coming from, what's secure source identity, but most importantly, you see the labels. This is how you're being able to do trusted protection inside your environment. Just set a label. Again, if I'm a developer, can I add my labels? No. This is where you force your operator to do this or some other subservice so that you can protect it. This is how easy it is to be able to allow traffic to flow through. Not only are you enforcing proper zero trust with identity and authorization with your

jot, but you're also enforcing it at your network layer as well, so that you as as a team are knowing how to secure these things, but most importantly, your auditors are going to come in. I just went through a sock 2 type two. We have a very bootstrap business. It's doing well, but most importantly here, we we struggle with being able to do proper documentation and following basic practices. So, our sock 2 type 2 is able to be protected. But what was nice about this, I threw my AI at all my repo. I said, "Go out and tell my auditors, this is how we do things." It mapped a whole network diagram because the AI could

understand my YAML files. This frees you like as I I don't know how many people do like GRC or just protection at a at a senior level or leadership level. If you think about these processes, you you get to be able to get your certifications completed. You have a secure environment and you have happy CEO and CFO especially CFO is extremely happy about this. All right, so let's continue. So next you're seeing a deny untrusted client. You see how the labels are different. That's what I was talking about here. And you get this all by the nature of the CNCF related tools. This is Psyllium and by defaults psyllium comes with AKS. So how many people run

AKS here? One, two. Okay, couple. Good. God bless you. It's it's it's a beast. Um, if you replace your CNI in AWS, it's a little bit more difficult. Um, a team and I over at Defcon, we put together CTFs for blue team village and we tried to do this exact scenario was running selium inside of EKS and it just it it just doesn't work great. It works, but it's very difficult. It's a lot of management. Luckily, with AKS, it comes it's just native. You just be like, change my CNI to Selium, and you get it right off the bat. So you get all of that observability without actually getting charged for all those flow uh uh

the the flow logs that AWS usually charges for. So another bonus there. Again, um another thing to highlight with Psyllium is it was acquired by um Cisco recently, maybe a year ago. It's recently, feels like recently. Um so the the Psyllium is the product. Tetragonon is the product, but the company itself, um they are now being backed and managed by Cisco. So hopefully they don't kill it. uh Cisco has that that problem of doing those things, but I think they're going to really lean into this because they get that rich data that we're talking about. So that's my demo here. Uh you can run this locally yourself, feel free. Um you know, if you want to

use it for your own presentations internally, feel free. This is this is for the people on purpose. I want you guys to understand as security practitioners what these new emerging tools are because honestly, these are new emerging tools. even though they've been around for a couple years, not many people are using them in production and now they are because we have AI here and developers are I mean they're cranking out stuff that they're doing infrastructures code. I don't know if we want that all the time. So think about that. You need to know what these tools are and what they're doing. All right. All right. So with that demo, what to do? Default deny period.

I know it's radical to say this. We love it, but it's very hard in practice to do. But I showed you the way to actually do these things. Start building MVP and showing your infrastructure team because if you are a security practitioner, sometimes in big corporations or is very fragmented. Those infrastructure teams do not want to talk to you because they think extra work. But if you build a demo and you show them what I want to do in code, they're going to be like, "Oh yeah, I see what you're really talking about that we could maybe even use this day one." So use this as a way to not only influence others, but become

friends and allies because as we grow, we have AI potentially replacing a lot of security jobs. This is how you level up. You become the people that are actually building the tools and working with the people that are building tools. And now you're actually in doing proper security. You're and it's easy now. There's no more excuses. You have AI to learn how to develop. It's not it's not a hindrance anymore. And thank God because it's saving me so much time. Um second, always treat AI infrastructure as sensitive infrastructure. Always. It's changing too radically. Even if you're running your own custom models or you're deploying um stuff that's either by Anthropic or OpenAI or or Quen for

example, this is this might change in tomorrow. We might have to change something because there is a new model that the team needs to do their job. And most employing with agents, I mean really tapped into agents really, but when you're talking about agent work, it's really it's really freeing to not have to do vulnerability management anymore. and give it to an agent. It's so nice to start thinking that way. Are we there yet? No. But we're getting close. Make security easy. We believe in two things at at our company. GitOps and no click ops. That's it. That's that's our mantra. I am told they're like do not ever click ops anything into existence. writing code

and then I give it to our infrastructure our plat our platform team and we work at it do the PR and push it this allows you to do auditability and most importantly gives you skills don't be afraid anymore it's so easy to use Terraform with crossplane and Kubernetes because all you do is throw the CRDs I didn't go into that but that's like the custom definitions you put that into your AI and you say here are the APIs documentation build this for me. Set up a plan and you have everything I just talked about. That's it. You just need to know the the terminology and you guys are going to be unstoppable. And last over observe because you will cost it

will cost a lot of money. Get that nice start with use cases. We know by default that it gives you the Kubernetes clusters about health yada yada yada. We care about how we're doing the protection of the AI models. That's what you want to do with security. How do you do proper observability that way for yourself and the team? So, we're all here. We love security. I think starting to I don't know like it as much anymore, but I I know the people need it. So, we're here to help do the job. Nothing's perfect, but the goal is to make everything secure by default. That is the mantra that everyone needs to walk away with. I don't care if it's reused

in papers. I don't care if it's reused anywhere in society. Just always put that in your mind. If you do it right up front, especially as young practitioners, it will go long ways in your career growth. Either if you want to climb the corporate ladder, you want to build your own startup, or you want to be radical and, you know, help out help out the world. There's there's ways to now do this. You can have these small Kubernetes clusters that are just hardened and you can easily just change and deploy things all you want either on cloud systems on prem or even in your Mac mini from your home. It's all possible here. So with that I want to

say thank you again. My name is Chris Mayer. Uh if you have any questions I'm up here for right now. Thank you. [applause]

>> [snorts] >> Go sir.

>> So great talk. Thank you. um this is a good overview of some of the stuff to do natively in Kubernetes, but I'm wondering if you've looked any deeper into um like container runtimes and things to provide additional security um things like maybe kata containers or aera to prevent container breakouts in the AI stuff since that's trained on you know the code of kubernetes and how to break out of things like that. That was a great question and um no I haven't gone heavily into this but I can tell you what we do do. We have a process that runs on a weekly basis every like Monday like patch Tuesday basically and we go out and grab uh the default

container and patch it with all security vulnerabilities and basically have harded images and because we are enforcing mutual TLS and we have everything locked down if there is an outbreak like what you're talking about then we just basically just destroy it because we we think of ID potency number on um we don't invest in that only it's not because we don't want to it's just because we're growing we not really working with vendors just yet until we start bringing in more customers and then we would think about those things because honestly I'm the only security person on the team and to me right now I just want to just nuke and move on more than contain and do forensic because

we're just too small of a company to worry about that any other questions

Oh yeah, sure. Go ahead, sir. You want to come up?

>> Is is there any significant uh barriers or advantages to you have already having or putting your workloads in EKS? Uh >> yeah, that's basically it. >> So the question is is you were just asking about EKS and are we deploying things to it? >> Yeah. Yeah, we we actually do. So we support EKS and AKS currently. Uh we're getting ready to do GKS. Um to us it's the same thing. The big difference um between the cloud vendors is um how we're managing our control plane. I didn't really go into that too much. Um, but the control plane is how we manage things um to then manage other people's companies, right? So, we run our control

plane in AWS or AKS and then we just save whatever the nodes are and deploy it. So, there's no real we have seen any big difference. The exception of the the psyllium CNI thing that I was talking about, I would say that's the only big difference. >> No problem. Oh, thank you. >> Hey, hey, Chris. Good to see you. >> Hey, Don. Nice to see you. >> Uh, maybe I missed this. I walked in late. Do you also support any like K3S or any of the on-rem light light horn, those sorts of, you know, embedded light distributions of >> Yeah. Um, I I the demoed I use kind cluster. >> Okay. >> Yeah. I I personally like using kind.

It's just very easy for us to um just deploy things and lo locally test. We also sometimes use it in CI as well that helps us like build systems more effectively. That oh that's one thing I haven't really got into. So when you're doing your CI with GitHub actions um that kind cluster that demo we were just showing you that's the local cluster. You can actually use that in your CI as well. So you can actually run Kubernetes in your CI process to do a specific test like a dry run or do like regression testing. We've done that as well and that works pretty nice. It's more complicated. Um my team does that more than I do honestly. I'm not going to.

They're They're really fantastic. Um, shout out to like Christian John and Ali and George. They're they're the ones that really are the creators of this. Any other questions? >> All right, guys. Well, I'll be up here. If you have any other thing, then thank you so much.