Harder, Better, Faster, Stronger: "FROM ubuntu:latest" Is A Supply-Chain Horror Story

Hi everyone, sorry for competition. Uh thanks for being here. Uh I'm today we talk about uh container security. Uh why is the most of dockers image images are fed uh are full of holes and total mess. The goal is simple. uh choose better base image and ship only what your upgrading need. Okay. Uh so uh who am I? I am a product security engineer. I break stuff for a living for I have 20 plus years uh experience with Linux and yeah it is probably right. I use arch by the way. Uh in my free time I pray on the head of the box and uh traffic me uh which is a health way to use criminal

skills and outside work. Uh it is uh 3D printing whole maps do by yourself like this pretty printed color button in our office. Uh does nothing use for it. It just uh make makes our team feel uh feel powerful in the incident time. Uh this tool cames from the real work. I build this images. I scan them and also I have them. So let's start easy. Okay. Uh let's start easy. Um you need Python, right? So what do you do? Uh you start typing from Python 32 for example. And result result is 1.6 6 gigs uh for your uh of image for your five max uh application and lot of learn why because you don't ship uh you don't

pull just the Python uh the rest is apparently system uh tools packages you never want it's like coffee You are ordering coffee uh but you get a coffee with barista prom or kitchen and the go but you didn't ask for the goat right but you have to patch the goat and python is not special there uh default no java go whatever it's same So there is a spectrum of spectrum of uh uh of book containers defaults. U this these are real numbers I scan these images last week by 69.4. I just kidding. But I see uh half of you start getting nervous, right? No. Okay. Load 24 1.6 gigs 33 criticals. Where's one? Python 3.12

9.6 gigs 7 critical. uh PHP with power patch 724 max 1k even the seam is quite lower it is about 179 max and 28 high still relatively high and uh the official Microsoft.net M image fun fact Debian slim inside 350 mgabytes and 16 highs. So when even says but that is the official image official doesn't mean say you haven't written any line of code uh but the mess is already yours random packages hiden bugs uh you didn't choose them but you own them and we de developed Very smart process. Scan panic ticket storm divide about severity. After that markers accepted and repeat water you know that is a compliance. It's not security. It's a audit theater

but we are run four scares. That's bad one. Four scanners is just four option for the same channels. And this one is pretty nice, right? Yeah. This is still pretty cool. uh I send you someone uh 3 weeks after in same person uh who marked this issue as a acceptance. Looks familiar. Yeah, exactly. Yeah. Uh the usual approach is patch everything, track everything and more scanners. What are what is the result? more bugs, more spreadsheets sort of by maybe more dashboards, but still no fix. And uh then someone put into the docker file add install curl v net and some others just in case that's not hardening. that is said gift basket for the

n. Yes. And here's a big idea. Do not patch what shouldn't be there. Just remove it. That's no lie to inside equal less risk. No shell inside, no shell to abuse. Security is not about aiding board. It is about having less fewer packages, small attack surface, fewer 3:00 a.m. uh wake up calls. Uh and this is uh just a list privilege for your file system.

Yeah. And there is a spectrum of uh base images on one end is Ubuntu and Fabian. There is shell package manager tools and a lot of extra risk. Fine for development but really bad for production. And then Debian swim still there is shell is less smaller fewer packages then alpine tin tiny image there is still shell but uh there is it is a big win for uh many apps but there is one thing which is quite different and alpine use muscle instead of GHC which sometimes core uh calls to the weird uh surprises during you try to build up uh and then destroy that's my favorite favorite uh just runtime uh no extra no shell no package manager

No easy abuse. And finally, scratch zero nothing. It's really good for stopping uh go binaries. I heard many times we have always use Debian. Sorry, but that's not a security argument. But we need SDK for the build our app and UF right but only build uh during uh build time. Nobody runs net build in the production. I hope one stage uh stage one uh the builder use the big SDK image build your app. Stage two, the runner uh copy only finished up into the tiny image. Distas slim scratch sur uh your build image can be two gigs. Who cares? Nobody. Yeah. Why? because never leaves uh from leaves the the build server. What matters is what's going to the

production. Think of it like cooking. You need whole kitchen for prep for preparing food. But you don't send kitchen to the customer. Just the plate. And there is example for Python JDK and go you can use scratch. So five easy wings and plus extra one don't run a suit use user 1 on or different number uh if your app need you see that's a really real problem. Second, make file system read only. Use read only flag is harder uh for attacker drop tools and files. Pre drop reuse capabilities. Start with cap drop all and a back only what do you need the most up four enable no new privilege. Yeah. Stops easy privilege aspiration. Sweet

mount only water and never mount broker socket into containers. Small change is big difference

and I will get I want to give a quick shout out Wolfie and Chingard someone new know about yeah okay wolfie is a container first uh Linux APK like uh use APK like fine But there is gypsy instead of muscle. Everything is built from source. Scan it constantly and may stay clean. So go with zero CVE or need near need near need near need near need near need near need near need near need near need near need near to zero CVE and also it is open source. Chingard turns that into the ready already ready made images for Python. M Java and many many others. Uh also minimal sign and verify. So minimal is too hard. Excuse doesn't

really work anymore. Enough. And uh what happens? So let's see what happens when attacker get. Sorry.

>> Yeah.

Okay. Now what? Sorry, sorry.

All right. So we have there two same same applications. One is running Debian. Second one is around the uh this string guard whatever and uh we can try what happened. There are two vulnerabilities. First is command injection and second upload files. So we can try to try try to bypass what happened. So we can start with adding injection right. So basically we start we have our ping somewhere. for something we have there get the ser and some command should be run. So how you can see there is a root okay we are root in the in the in this container. So what we can type on this should be same or different. Okay. Pink is running. Okay.

And who am I? Nothing. Okay. We can try to where I am. It's not working right. >> So we come back to the one mar is which is running from the pool. So for example, this should be working right and we have the risk of shadow. Second second thing but we need more we need uh we need a web shell or some session on the meta right. So we can try to start with polish. Basically there is a with only text PNG and the JPEG. So we can try to upload our our web shell. Okay. But when we try to go to it by passing support and we can also on the one we have the web shell so we

can type who am I oh sorry demo I'm root Okay. And we can try on this tools. There shouldn't be any any privilege of that yet any tools. So we know the bypass is already there. So upload the shell and nothing but that is also it's not enough. We need we have probably better better solution for that. Do something know what should be it. For example, session should be fine, right? So we can try to spray

and start

and try to upload our new uh

We type I just prepared

So we are running interpreter GCP. So set

your mouse.

Set the port. Set the

set.

And we can I would like to show you how this working. So there is a micious one. So we can try to upload our letter practice session and running. Yeah, probably we have to get shell the problem. So we go around this one and there

I move still. I can for ABC shadow as before. Okay, we can use and try on the

on the distance.

So after the session started

around the P and we get better, right? But that is quite an unus because we we are trying to figure out what we are which which user we are there is no and we try to get shell there is error. We are trying to uh get what uh processes are running on the machine and get zero. We can also try something harder. For example, up we can what we can this where we are just we can upload

I go up this egg and execute.

But it also is unless. So in the end I would like to compare how is the difference. So there is a blue one this has 48 mgaby and full arm have 520 mgaby and also we are scanning these two there is a big difference because this one is which have 1,69 9 verbalities, one critical 88 highs and 139 medians. And this class this price is just zero. Okay, I try to jump from presentation.

So, uh after getting in we get uh you really we really need shell writable. uh file system, network access, tools, package manager. Uh the rep up gives you all six. Yeah, it's like Airbnb walk in, use Wi-Fi, open the fridge and glue up zero seats. Yeah, you are standing in the potato uh with the most unless session ever. This is uh this is uh when the attacker or hacker gives up and looks for an easier target that's mean you get it right you have time or you so how to start it so security uh doesn't get budget mostly it's Faster does smaller base floor image image is uh built faster pull faster deploy faster less storage

less cost and what is the most important fewer bugs uh is equal to fewer alerts fewer alerts is equal to fewer 3M work uh calls and fewer premium Most uh is for pure engineers up updating their LinkedIn in the Monday morning, right? Safer is also faster, cheaper and simpler. You don't lose anything. Just remove what you praying. What do you don't praying? So, oh sorry that was that was this one. Sorry I forgot. Okay, never mind. Uh so recapitulation fast uh harder lock down the runtime better fewer bugs smaller target faster faster build faster stronger tough against real attacks not just on the paper make your container hard stronger when your attacker gets session and said.

Yeah, this is now unless it is time where you want. Thank you. I

came to me and talk about containers or share your worst docker files or thank you questions. Yeah. Uh is this new feature of docker hardened image is can >> uh is this feature like new feature from J this year of docker hardened image. Yeah, this is uh basically I think the same. And this is developed by Google but also there are more uh ways how to build it and uh this just

>> you mentioned there there is like a faster build for the still need to build the image proc which are copying the files. >> Yeah. It is mean right because uh from S perspective right uh US engineer make a build s is deployed and time to deploy is much faster right you build all container just build deploy somewhere thank you Let's say hypothetically that you need to use Ubuntu. >> Mhm. >> Uh how would you >> let's say hypothetically that you have no saying and your paycheck is not big enough to take the decision. So how would you sl the image? I mean how would you reduce the the size? >> I think the social is on the on the in

the presentation. So that is uh that is multi-stage built right. >> So multi-build where you kind of like >> yes you you have builder which is built uh application itself and you just copy this application to the runner which is basically runtime nothing else. >> Okay. >> Yeah. Is it okay? >> Kind of. Yeah. >> Kind of. Okay. So, thank you very much.