Container Security, Are You Doing It Wrong?

[Music] well great fantastic let's get some energy in for our next speaker here we have with us today Mr Jeff Weatherford Jeff has been an IT professional for over 25 years he's a United States Air Force veteran he's worked his way from syst system administ trator to Enterprise architect and then moved into Dev SE Ops he spent 18 years in the US DOD intelligence community and he is one of the vetcon admins and staff members he likes all types of rock music nice cars motorcycles and Pirates and so with that being said r hand over to you thank you thank you all right everybody hear me okay all right well first of all thank you for showing up I

know it's lunchtime so you know that means you guys really want to be here which is awesome uh so our agenda real quickly what are containers what are they not some container hardening practices some advanced concepts and then we're going to talk about the software supply chain a bit um obviously that's kind of a hot topic now with the whole you know XE thing I'm sure a lot of you have heard about that um again who am I there's your stuff up there uh my LinkedIn an email if you want to get me uh I don't post on X I do link I do lurk so I didn't really feel it was uh worth my

while to to put it up there um so yeah and right now my current position the big thing to take away from this slide is at the bottom is I am this me the subject matter expert for container hardening on platform one's iron bank uh anybody here know what that is or heard of Platform One iron bank okay all right some some assumptions that we're going to make here that you've heard about containers you have a rough idea what they are uh you have a basic working knowledge of the Linux command line uh you know what uh some knowledge of a cicd pipeline and that you're going to put a container in some kind of orchestration layer uh just to

to make it useful by the way I'm terrible at slides there's going to be a lot of walls of text I apologize in advance there's a lot of slides so buckle up uh what this talk is not I'm sorry I'm not going to teach you how to break into containers or break out of them uh we'll talk a little bit about that but you know you I'm not there's no zero days here you know this is kind of an introduction that's how this is really kind of structured is so that way everybody gets something out of it uh what are containers um and the big thing for this one here is that bottom one why is it

important from a security context how are we going to think about security when it comes to Containers uh containers are not virtual machines I can't tell you how many times I run into this um they don't have their own kernel they don't have their own memory they don't don't have their own network stack right they they you share memory with everything there which is why there are certain attacks that can be so dangerous when it comes to containerization they are not sandboxed by default keep that in mind uh By Nature containers are ephemeral treat them as such right the whole you've heard the they're not pets they're cattle right treat them in the head you have to treat them like

they don't exist essentially you know that they exist in the e um and so what that means is you don't want to have a lot of stuff that is stateful in containers if you can avoid it uh for that very reason one big problem that we have when it comes to container security is how we think about them I I touched on that earlier and that's the one right there the dod stigs have you all heard of the stigs the STI guidelines right well the dod in their Infinite Wisdom decided that you know we had this really Nifty technology and so what we're going to do is we're going to apply our old methodologies to thinking

about it and so we're going to apply stigs to Containers 99% of the stigs out there don't apply in a containerized environment there are some that do but again that goes back to the containers are not virtual machines quit treating them like they are they have more in common with the CH root jail than they do with the virtual machine another way to think about it they have more in common with uh J than they do with a virtual machine now so we've all heard of java jav runtime right you have that executes a jar file usually as your Java program same kind of idea only Java actually has a whole lot more isolation built into it than

containers though by default with the oci and run C containers are not a security boundary so this goes into the whole idea again that that containers are not sandboxed so they can communicate freely within their own name space they can read right to their host system um there's you know an example there anything that the container can see or the spawning user can see the container can see uh because that us that user can alter that container to see anything that that user can see um so the you can do interesting things with containers like you know make uh dos attacks worse you can see there by consuming CPU resources why why I that be well a lot of times it there's

constraints people will sit there and spawn a container and they won't constrain it right they won't tell how much memory that container is allowed to use so by default it's going to use whatever it can so it makes it a good way to get your out of memory killer to start going crazy uh and uh if you misconfigure them you can communicate across Nam spaces your name spaces are really your only isolation when it comes to a container uh your security security boundary here is the host system right that's that's your your nuclear thing because if I get a if I get access to your host that you're running a container on I can do anything I want to

those containers I can modify them I can inject binaries into them I can copy files into them right I can do whatever I want to any one of those containers so you have to protect the host yeah so uh sorry just kind of organizing my thoughts uh what this means is you have the traditional vulnerabilities that the software already comes with right you know all the cves that get applied to whatever software you happen to be running plus potential vulnerabilities in your runtime whether it's Docker or pod man or whatever uh so the there is nothing about running in a container that's going to make you safer people assume that's the case it just simply is not you have you have to

assume everything inside the container is public everything is visible no special sauce no certificates uh no secrets uh those kinds of things uh I did throw this slide in here kind of last minute because I don't know if you guys heard about the sneak uh the they came out with this dcvs that they found um called leaky vessels that have to do with the container run time uh all Container run time times we're affected by this it use runc so that's pod man Docker all that uh I threw it up there take a picture if you want uh I'm not going to get into the Gory details of this container breakout but it's something that you may you know want to

research on your own uh so this is just to to highlight that your containers can actually be a source of vulnerability to your system uh pretty easily actually uh my big side rant here this is a big one again containers are not virtual machines so quit putting big monolithic programs in containers stop it stop it stop it stop it all your atas yiras all your service Nows you know all that stuff quit putting that stuff in containers yes I know they come out with officially supported containers from the vendor this is everybody jumping on the containerization bandwagon saying oh looky shiny container right stop it that's not what containers are designed for that's not how they're used okay

that's not what they are uh let me give you a perfect example here uh this I don't have a slide for this but the largest jira implementation in the world is on a secure DOD Network on a top secret Network it is bigger than it last you thought possible until we showed them that it was the guy who built it a personal friend of mine anybody here uh want to Hazard a guess as to how many nodes that eurac cluster is no six it is six nodes they are enormous nodes but they're six nodes reason why a lot of these applications a lot of these monolithic applications scale vertically not horizontally so keep in mind if

you're containerizing something generally speaking your your uh your model is going to be to scale horizontally I'm going to spin up a bunch of little things to do one thing and then scale it back down again right you're not SC scaling up you're scaling out so you know I put the old quote in there from Jurassic par because people this is a new technology everybody thinks it's shiny so everybody's jumping on to use it use cases people you need to make sure that your use case for your software uh goes with the technology that you're using uh so okay enough side ramp how do we build containers uh this is just kind of some intro container

stuff okay we have Docker file that describes a container there's a reference there for the uh Docker uh all the docker directives and how they work you use some kind of Builder Docker has its own builda is the Builder that breed hat favors uh it's an open source one because podman as a runtime doesn't have its own builder so they have a separate binary for that uh and then what you're going to do is once you build that you're going to put that resulting container into a container runtime or sorry a registry of some kind basically a library a place you can download it from okay what are the security implications of of building containers well we'll start with layers

so you can see right there that there are three Docker direct well I should say container directives from the oci spec that will add a layer to your container run copy and add uh but the big thing here is each one of those layers you can peel that layer back there's a Shaw value associated with each layer and I can go back in that container image and I can pull from any layer I want so let's say for example that I build a container and I say okay install this software configure it this way now here's all my and then I have a separate run statement that okay here's all my security lock Downs cool I

just pull the layer above that and all those security things that you put in went away right so one thing that you want to do when you're building here is you want to consolidate as many of these layers as you can and you certainly don't want to leave out security stuff in it in its own layer don't do that uh you know this like like I said right there it'll under undermine any hardening work that you've done uh now keep in mind containers are just a piece of software they're really kind of very similar to an RPM right it's a piece of software that is a file it's a tarball it's got some extra metadata in it you

know a few things like that but at the end of the day it's a tarball so you can modify it in any way you want end users can and will modify containers at runtime so you have to make that assumption all right so this is why it's like providing an RPM and red hat provides you an RPM or Debian provides you a Deb and it's going to be a series of defaults that are saying hopefully uh some software some documentation maybe and that's about it they assume that you're going to configure it at your end and you're going to be responsible for the end Security State of that piece of software containers are no different you have to

treat them like an RPM that it's something that can be misconfigured will be misconfigured and any Sayan defaults you put in there can and will probably be overridden does that mean we shouldn't give them Sayan defaults no we should but keep that in mind you people think that containers are some special thing and they're really not like there's not much to them when it comes to you know what can we actually do security wise but we will talk about that so if we do want to harden Contin we want to make it you know a little bit harder for somebody to abuse uh well what are we going to do well we need to you know look at our

threat model we want to look at our threat vectors we want to ensure that we see as small footprint as possible again we're talking about attack vectors not attack footprint vectors and footprint two different things footprint is how big something is a vector is an actual weigh in the vectors are what matter I mean you'll hear people scream from the high you know the hills that when it comes to Containers we need to strip out all the excess software you know we need to reduce our footprint and reduce our footprint well reducing your footprint is fine and dandy but if the same vectors are still there the same attack points are still there you've accomplished nothing other than

shrinking size right so keep in mind that you want to focus on the attack vectors the ways in uh configuration oh yeah remove Dev tools and other unnecessary libraries GDB is not your friend in fact I think I say that later in another slide right do not leave compilers in don't leave tools like GDB netcat C [Music] all these things that are unnecessary to the running of your of your binary now it might be you may need those to build but for the actual running of the application you don't need leave them out in fact what you should do is remove even your package managers at the end of the day now at iron bank we don't

usually do that why because we understand that we are not the end state of that container we know that our users are going to go take that container install their own software do their own thing with it and then use it right so we're giving them a base so they need to be able to install things or or make some of those modifications but as an end user if you're putting something in production your final container remove that stuff don't give any tools that you can uh avoid to your adversaries but the most important thing is that last that last one configuration is all important right like I said you can override all the the same defaults

pretty easily you can do it from the command line you can do it in a Docker file you can do it from a a Docker compose yaml you can do it in a Helm chart I mean take your pick right there are ways that you can override all that stuff and speaking from not only experienced but to one of my friends who's a CEO of a security company who does sock monitoring and all that most of your breaches the vast majority of them are not zero days the vast majority of them are misconfiguration someone you know leaving default credentials somebody leaving or you know misconfiguration something leaving a port open that they shouldn't etc etc

but at the end of the day it's misconfiguration it's human error is the vast majority of breaches right why do you think fishing is so successful they focus on that human element well why go through the trouble if it's so easy to to break things apart and misuse them well obvious obvously there's some regulatory stuff right there's Healthcare Financial Federal Agencies anybody with sensitive data is going to want to kind of make things difficult for an attacker I would say and there are some things that we can do and to make these you know a more difficult Target I know I've said that a few times um I'm actually going a little faster than I thought

okay um so what does it look like low hanging for do not run your containers as root don't run your containers as root okay if you can avoid it obviously there's a few things out there that need brute privileges they need that ring zero access to the colonel Seth is one of them it's a file system manager right so I would probably argue anything like that you probably don't want to put in a container but it's not up to me uh if you do need those kinds of privileges there are ways to you know Grant those sscat privileges to The Container without necessarily granting it as root the details of which are going to vary

depending on the container and what you're trying to do with it but just keep that in mind do everything in your power to not run his route by the way Docker runs his route by default keep that in mind Docker requires root that's why podb is why Red Hat came out with podman is because that can run in in user space uh so we talked a little bit about the next one don't you know install unnecessary software packages if you don't need it don't install it uh it's it's really quite that simple uh you have ways of doing that you can tell it you know adding no docs to your RPM command or there's ways to to tell

it you know don't add dependenc you know unrequired dependencies depending on your package manager uh of choice so uh refer to your documentation on how to do that because I don't know which one you're going to use um but they all have something like that uh one thing that we always do when we start if we have a need of a bunch of system stuff we'll start with a base Red Hat Santos Debian Ubuntu whatever uh run an update it's the very first thing you should do make sure you have the latest packages now one thing that we do is we build every 24 hours at a minimum so every container is going to get rebuilt

within 244 hours for this reason that way anytime a cve drops a new patch comes out in less than 24 hours it's patched and published to the registry you should have that mindset because the amount of cves that we have that we have to deal with that every day is it would surprise you probably especially because we're going across systems right we have to worry about supporting Debbie and we have to worry about supporting red hat and all that right so uh it is what it is um and make sure that you're doing your gpg checking right when you're building from appropriate sources this is where we start to talk about software the the software supply chain you need

to ensure your checking these things now was that would that have stopped you from the XZ thing no uh but it's a good way to still avoid man in- the middle of attacks of people trying to inject poison packages into your into to your containers use multi-stage builds if appropriate this is where we talked about uh if you need to compile software right you need to run a compiler whether it's go you know GCC whatever you usually want to do that in some kind of build container right so you'll have a container that you'll call it Builder and there's ways to do that the multi-stage documentation in the docker docs like I'm you I don't want to

go through things that here that you can easily look up right so the idea is uh you do your compilation and then you copy the the artifacts that you need into another container and that and that has just your runtime just your binaries and that's it and that way you don't have to you know worry about uninstalling uh the compiler you don't have to worry about it leaving stuff behind that could be potentially used against you uh now I will say this updating and installing from from packages you're going to hear a lot of people sit there and say oh build from Source build from Source that's the only way to be secure no it isn't and I'll

tell you why it is more secure if you audit every single line of source code who here has enough time to go through the Apache source code and audit it that's what I thought so you're better off in many cases from a reality perspective not a theoretical perspective but a reality perspective perspective of using official sources for your builds uh using official you know Debs using official RPMs because not only are there your eyes on it but you also have the security teams of those sources looking at this stuff as well is that going to save you 100% of the time no nothing's ever going to save you 100% of the time but the idea and I

think that we've all heard this many times before is defense and depth right we take a bunch of little steps so that way somewhere hopefully in the chain something will get caught all right so that's what we're doing here and use other people's security teams to your advantage right I mean I know Red Hat has a whole lot more security people on staff than I do so it is what it is all right more low hanging fruit all right uh you want to scan your registry uh there are various tools to do that uh you want to make sure that you're scanning constantly you want to have a central uh repository a central registry that is your one

source of truth right you don't want to have a bunch of registry scattered all over Helen gon because it's a lot more places to check and it's more places that are vulnerable so you want one robust registry that you trust that you're constantly scanning and has cryptologic verifiable artifacts and we're talking s bombs the software bills and materials things along that the shaw values of your images things that you can verify crypto uh cryptographically that nothing has changed and that everything is still as it should be um you definitely want to use a so this we're talking about building from Upstream uh you want to use some kind of proxy generally to perform validation of your Upstream

sources to check those those Shaw values to make sure that things are coming in so like we build in 100% offline environment when we're building containers we have a special way that we go out and we pull that source code that RPM that Deb that whatever that binary file however you know that container there's a variety of different ways we do it copy it into an offline environment that is not connected to the internet and uses that to build and it cryptog like cryptographically I'm going to get that word right one of these days uh validates every single object it Imports to build that that container right uh few other things uh I know these are

kind of these are in no particular order by the way uh this was kind of a stream of Consciousness it just kind of is what it is uh in your Docker files use copy instead of AD we don't want to use ad ad is bad okay they do the exact same thing with one major difference ad will follow URLs and it will untar things by default for you it tries to be helpful um well that you know becomes a case of stop helping me right that's a good way to get things injected into your container that you didn't intend so always use copy after you've downloaded whatever it is and then cryptologic verified that that is in fact what you

meant to put in okay um we talked about the Run statements a little bit you know we want monolithic run statements you we want to compress as many copy statements into one as we can just because again layers not only for pulling layers but also layers mean more memory more resources uh it's just more efficient to have as few layers as possible and again multi-stage builds there's a reason why I said this again this is important um is the to reduce the amount of stuff in your final container use multi-stage builds all right uh this is kind of what it would look like this is just an example that we pulled from the uh from iron bank is you'll see that

from build that's what you can tag a a you'll have something as something right so you sit there and say from this container as build and then you can use copy with the from flag there to copy stuff into your current container from the one that you had tagged as something else that tag is arbitrary by the way it doesn't have to be built you can name it whatever you want um Su jid yeah don't do it go and look for everything that has those bits in there and unset them why is this an issue you well especially if you have access to something like a debugger now I have a way that I can

inject Shell Code into memory or anything else into something that is running as a as root bad idea make sure those are gone there should be no reason for something to R to need that uh capability within reason uh Apache does some odd things but it has kind of its own Su built into it because it's Apache uh but keep that in mind anytime you see this stuff uh you can anytime you you just get rid of it um let's see clean up after yourself this goes into you know after you downloaded it with and installed it with RPM or dnf or yum or whatever go tell it to remove the stuff that you don't need need clean

up uh again this is I I talked about a little bit earlier but we do this is see we build uh from uh in a sterile environment disconnected from the internet uh scan your stuff for viruses that should go without saying but it's up there just so nobody forgets and scan it before and after all right and when you up upload it yeah before you upload it into the registry you scan it uh you also want to scan your sources as you download them that again that's kind of a no-brainer but you'd be surprised how many people skip that Stu uh and remove the secret so we talked a little bit about this I'll go into a

little bit more detail uh you don't want to store certificates because anything inside your container is visible to the public and modifiable right so don't put your private keys in there uh however if you need to be able to do things like use pipy to install uh python modules you're going to need to leave the Sears you know that are in there for a lot of the Python packages uh to actually function so that's why I say you don't REM remove the stuff that will break the package managers unless you're trying to break the package managers if this is your final step before publication go ahead and break the package managers right but just be in

mind you know be aware if you just sit there and look for anything named. sir. pem you know that kind of stuff and delete it you could cause breakage so just be very careful of that Python's notorious for that of having all kinds of CTS that come from from piie now granted python has a lot of issues with piie in general and malware being in there but that's different discussion um all right you if you do want to delete your certificates there's ways to do it uh you give you uh the two examp two examples with xargs and with exec RM uh I'm not going to fight that holy war for you I'm sure many of you have heard that

uh which one's better but doesn't matter I really don't care as long as you get rid of the stuff that you need to get rid of uh now there are tools out there that will help you uh truffle hog is probably proba one a lot of you have heard of uh we use it extensively it is prone to false positives just being uh be aware of that anchor is something we use a lot and it can detect secrets and you can you can actually configure it in your pipelines to actually stop pushing to the uh registry if it detects those Secrets but again be aware of false positives especially with python it's almost like I've said that

before I wonder why oh yeah that's right because Python's python uh just something to be aware of uh I don't know why python does that side note I I don't I mean G doesn't have that Pro or sorry go doesn't have that issue Ruby doesn't really have that issue it's it's a python thing uh let's see okay minimizing layers we talked a little bit about this uh there are some things here that I do want to talk about specifically though you notice how in this one monolithic run block that we end with the double Amper Sands the reason why we want to do this is because we want to make our run statements one big long command chain

that way if any one step fails the container build fails and we want to know that because I promise you you put semicolons in there it will work not only will it work but if one of those command fails it'll go ahead and Skip on to the next one and we'll keep running and because the only time it will actually fail the build is when it gets a return code that's non zero right but but it doesn't check the return code of every single command that it runs it checks the resultant of the very last one so if those were all semicolons every one of those commands can could fail as long as the RM Tac RF

at the bottom works the build will succeed and you will probably have something in your container or not have things in your container that you and you're GNA spend a lot of time debugging it so save yourself some grief and use the double Amper Sands all right um let's see uh you can see you know removing yum cash that all goes into you know cleaning up after yourself so this just gives you an example of some some things um all right so again wall of text I apologize uh we've had problems XE I know I mentioned that before this is an example right uh you have to validate your Upstream software and watch what your uh watch

your supply chain now let's see you you of course you want to set up all your canaries your AVS all that stuff before building you want to do it after building you want to keep doing it um it's a continuous process I don't want to bore you with that container user again what we said not as a root so what do we do usually we'll either create a user or we will uh you can use just a uid sometimes you need a full-blown uh user with a home directory and all that uh for the software to work it depends on your software Keynes here we try and use Espin no login wherever we can we're not logging into containers

folks generally speaking right you want to avoid people getting shells easy right so don't allow them in uh and a pet peeve of mine in there is not upping the cisu ID max if we are making these or we're telling uh user ad that we're making system accounts uh otherwise it'll throw errors in your logs it'll still do it but you know I like clean logs and I don't like chasing you know stuff in my logs that I know is uh benign so just do that uh that's why login. deaths exists so make those changes as necessary uh you can see here that we're using a command that we're firing off Python 3 so they're going to get a

python shell in this particular case uh but so you want to be very careful as to what your what this container is actually going to do you're usually use an entry point or a command to specify what it's going to run yes you can use scripts we use scripts heavily to alter the um or or to set up environments and do things that we need to do uh for a process to run correctly so uh just keep that in mind um let's see looking at okay we got another 15 minutes or so uh okay we're going to talk a little bit uh more here about some more advanced concepts okay carrier pipeline we talked about

software uh manage or sorry supply chain software your software supply chain how do we do that well it's not just your sources right so we have all these tools and I've listed a bunch here you can use your your tools of choice okay I'm not a shill for any particular company I don't care which ones you use uh but you should use some these tools uh and and I right there in the parentheses again I will let you read as far as what they do um kerno is a big one for doing your your policy engine you must have something in your environment that says what containers can and cannot run if you just let any container run you are

asking for a trouble right you need to have some kind of policy stating which ones are allowed to run and which ones are not or and by default deny and kerno does that uh you want to have your multiple scanners of various kinds that's and and there's several listed there twist lock enor open scap nessus we use actually all the all four of those uh and why is that it's right there in the in the text you don't trust a single Source not every tool catches everything and not all tools will give you the same results right so what you have to do is you need to use multiple tools to get a holistic picture of what is actually in

your containers what issues might there be there what cves might be there what configuration items might be caught by one tool that is not being caught by another right same thing with your virus scanners don't rely on clam a okay don't rely on maffy right smch whatever don't have a single Source because same thing just like your container scanners you will miss things right you need to have multiple sources to get them the most complete picture uh I talked about es bombs briefly uh there's the uh what a software billing materials is there's a link uh I'll let you do the reading on that um and basically it is a list of what is in each container uh in our case

because we're talking about containers what software goes in there to make it up uh hash values for anything so you can actually go and verify individual pieces of a container uh it's a very important thing to have and uh cosign for signing containers when we publish them because again we're cryptologic guaranteeing that the container that we publish to our registry is has not been modified in any way other than you know it's been done by us right it's just like signing an RPM so same deal all right you us container and container builds so if we're going to build containers we probably don't want to do it on bare metal for a variety of reasons um one thing is if our if we

detect something in our pipeline uh that you know some container got corrupted we can kill that container and respawn it right and they're ephemeral so cool problem gone we also want to make sure that we're not storing packages on the host system right so since there are femal containers you have a container spin up do your build process download whatever it needs to inside that container and then when that container is done with its job in the pipeline it goes away it means all those files go away all that memory gets freed up all that stuff is gone all right that's what we want and of course you know you can scale your builds it makes it a lot

easier to scale horizontally because building multiple containers is something that scales horizontally this would be a case where containers ERS work right so I'm trying to make sure hoping that we get to the Q&A section here uh as far as deploying containers don't automatically push stuff into prod all right that should be a no-brainer you'd be surprised how many times I see it you want to have some kind of human gatekeeper that actually Yay verily they have some kind of testing that's been done something along those lines and don't pull from prod you know don't pull something into prod without some kind of human gatekeeper don't do it uh separation this is a big one do

not put your infrastructure stuff in pods containers uh in this case so quick side note when we're talking about kubernetes and other runtimes they'll usually use what's called a pod pod is simply the lowest addressable unit you can have have one or more containers inside a pod but all those containers uh will share the same network address the same ports yada y y uh so for the most part generally it's a one to one unless there's side cars but we won't get into that that's more advanced stuff but uh the idea here is that you do not want your applications or your pods or your containers that are handling data to be on the same physical nodes as your

infrastructure now you're going to have to have cuet if you're running kubernetes on all your compute nodes or otherwise they're not compute nodes but that being said you you know your DNS pods your your ingresses your uh ETD especially your ETD all that kind of stuff you want to be segregated into management nodes and then you want to have the dirty user nodes only running user workloads right now there's actually some regulations about this when it comes to things like Hippa and whatnot and separation to data because they've realized oh yeah that's right containers are are not the security boundary the node is so you will grow you will segregate your data and your workloads via taints on you know

kubernetes 10 saying okay this group of nodes will run my Hippa dat this group of nodes will run my non-hp dat Etc uh and then this is you know our management nodes no touchy right you know that has you know the API server and that kind of infrastructure stuff on it obviously there's exceptions to every rule but we're talking about just kind of General General use cases here and and kind of just kind of best practices uh using Central what do we always want to do we always want to use centralized deployment strategies um I know wall of text I apologize uh uh we want to use our security tools like open policy agent and kerno right

we want to make sure that we're only running workloads that we know of and that we specifically authorize uh you want to so yeah always pull uh on create so the so one of the strategies for containers right we have to know when we're starting up uh a container what we're going to do so you want to use the always pull strategy to say that anytime I start up this container I'm always going to pull it from the registry I'm not going to pull it from cash side note be careful with this because even if you hav't always pull if it if the tag for the container has not changed it's not going to pull it it's

still going to run it from cash right so you actually have to flush it and then tell it to repoll but those are things that you may want to consider to make sure that yes it slows down your startup time um but that you're always is grabbing the latest and greatest container from your registry and that you're not using something that has out-of-date software in it because that cash will bite you if you're not careful uh yep and of course make sure our hashes haven't changed that's kind of you know why hashes exist right uh and you want to do that comparison before you actually deploy things so this sounds like a lot of custom code it does

it is so your pipelines so when we're talking about cicd pipelines are going to be fairly intricate things you're going to be doing lots of things in them it's not just simply pull and the ploy it's not simply going to be one Helm chart that says here's all my stuff not if you want secure containers right you're going to be doing lots of steps everything from from downloading from sources to building containers you're you're checking things at every and scanning things at every step you're going to check formats you're going to check your secrets I mean there's a lot of things to do so when people say oh I'm gonna go to Containers because it's

going to speed up you know my whatever no it's not not if you're doing it right so that's in other words be wary of shortcuts you know we've talk we're talking about a lot of stuff here and all this stuff really needs to be done because if again that defense in depth all right almost done all right here's some fun attack strategies you want to play with someone 's uh kubernetes environment that has done something stupid like let you in here's some fun stuff um these are just strategies to go after like going after the you know CNS or the service mesh that's a fun one use loic once you're inside if you can deploy you know low at

containers inside some of these cries cluster Target C DNS the whole system will go down uh so yeah I mean uh use containers for GS uh use them you know for overflowing memory you use them to exfiltrate data uh all kinds of fun stuff so you know the the these are all different ways that you can mess with containers and you know this is not an a comprehensive list this is just to get your your creative juices stirring uh further reading right again feel free to take pictures I'll leave it up here for a little bit these are just uh very stuff by the way if you do have that first uh slide that I has my email shoot me an

email if you want the slide deck um and uh I'll make sure that it's I can get it to you special thanks guy named Chris Ry guy I used to work with he came up with some of the images and and uh wrote some of this I redid most of it but uh you know I didn't want where I didn't want to reinvent the wheel so thanks to him DC 813 vetcon uh and my current employers Platform One I am on the container hardening team as this me again and lastly uh Tampa B sides thank you for having me [Applause] [Music]

[Music]