← All talks

GF - Climbing the Production Mountain: Practical CI/CD Attacks Using CI/CD Goat

BSides Las Vegas47:162.5K viewsPublished 2022-09Watch on YouTube ↗
About this talk
GF - Climbing the Production Mountain: Practical CI/CD Attacks Using CI/CD Goat - Omer Gil, Asaf Greenholts Ground Floor @ 11:30 - 12:25 BSidesLV 2022 - Lucky 13 - 08/09/2022
Show transcript [en]

all right good morning welcome to b-sides las vegas this track climbing production mountain practical ci cd attacks using cicd goat presented by assaf greenhouse and omar gill a few announcements before we begin for sponsors we'd like to thank sponsors especially inner circle sponsors critical stack and valley mail and some stellar sponsors such as amazon paranoids microsoft etc even the national security agency these talks are being live streamed so as a courtesy to both the speakers and our remote audience please silence your cell phones if you if there are questions at the end of the talk there's a microphone in the middle of the room so please ask your question in the microphone so that our streaming folks can can hear

the question

and without further ado

okay yeah i think you can hear me um hi everyone good morning welcome to besides welcome to vegas and welcome to our talk uh climbing the production mountain practical ccd attacks uh using cacd goats um before we dive uh before we deep dive into all these attacks let's uh introduce ourselves quickly so my name is omer gill i have an experience of around 14 years in hacking web applications cloud security enterprise security uh released some research materials in the past like the webcast deception attack back in 2017 and a few months ago the top 10 city security risks initiative which we'll mention in a moment um besides that uh today i lead research at cyber security

we are a young startup that exists for nearly two years focused on optimizing ccd security uh solution provides visibility and detection and prevention capabilities for possible flaws and attack vectors in your ccd ecosystem are similar to what we're going to showcase today hi everyone my name is asi i have about seven years of experience in cyber security i've been doing stock incident response security architecture and for the last two years i've been doing application security and ccd security

okay so daniel which stands by here right here by the way his uh uh cyrus's co-founder and ceo and myself uh we we have been working for several years with many organizations on improving their security posture and uh something that we found was common to all of these organizations uh we um like in most of these organizations we we found many flaws and possible attack vectors in their engineering uh environments and uh something that we found common to all of these organizations was the lack of knowledge and methodologies of how to protect the ccd ecosystem and this huge gap of knowledge for defenders in the areas of ccd led us to uh to releasing the top 10 ccd security

risks project it's a big document a 40-page document based on extensive research that you've done and discussions that we've made with some of the industries top experts are listing and detailing the top 10 security risks in ccd environments and and this uh project after we released it it got some really good traction from security fellas from around the world and really helped our organizations to start protecting their ccd environments but as you all as you all know there's nothing like hacking on your own to actually understand how to defense against uh against uh attack path so this is why we created another project which is the csd go

so as omar said the cicd gold project was created uh as a learning platform basically it's a ctf it's open source you can see the link right here to help people learn about cicd attacks try it themselves we have 10 challenges currently in the ccd go they cover 8 of the top 10 ccd security risks and we got really really good feedback from the community a lot of people have sold it solved it by now and we hope uh you will also try to solve it um as you can see rothenbaum and iron avital also help us in writing the project so let's see how it looks

so this is the environment the whole project is based on docker containers we have these containers first one is a ctfd which is an open source used to manage ctf challenges you'll see it shortly we have gt which is a source code management system also open source looks similar to github it's very lightweight uh the repositories on gt are connected to jenkins pipelines jenkins also has a linux agent and the production environment is simulated by local stack which is an emulator to aws it it allows you to test stuff on aws locally and we have light httpd a small http server and one thing worth mentioning is that the whole project is configured as code

this also helps people to contribute the project to the project easily you can easily make changes and we have tests for everything in the project to make sure everything is works as expected and last thing the project is inspired by alice in wonderland so you'll see some familiar names

and what we are about to do today is to present our three attacks uh that attackers can uh can execute in a cdd environment um all of them are based on uh the research that we've done in the past on a use case that that we've seen in actual environments of customers that we've worked with and on wheel attacks that were published to the world and all of these challenges we're going to present them through the ccd goat project don't be afraid of spoilers it's just 3 out of ten and soon we're gonna be soon we're gonna release additional uh challenges and in addition i'm pretty sure that it's gonna be fun enough for you to solve it even after you uh you'll

see the the solutions today um and this is our this is our starting point uh we are attackers that obtained access to a user account in the organization that granted us a user account to the sem to the source code management system which is gt in this case we have access now to some of the repositories some with read permissions some we do i permissions and you also have basic read permissions to gen case now we do have some three challenging demos today so i need you all um to sorry it doesn't work i need you all to hold hands and pray with us for the democrats uh we even created an illustration of three goats on bentleys

with folded hands praying to the demogods facing a big bronze left of statue in a mysterious temple thanks to the recent delhi invitation that i just received and let's start with our first attack for today okay guys can you hear me yeah okay so the first attack let's log into ctfd as you can see this is the the platform we have three categories of challenges according to difficulty and today we'll start with the caterpillar so as you can see you just have read permissions is that enough use your access to the wonderland caterpillar repository to steal the flag 2 secret which is stored in the jenkins credential store let's make it bigger a bit okay

so basically what we see here that we only have read permissions to this caterpillar repository and we need to steal the flag 2 secret that is stored on jenkins so let's go to the one dollar and organization we see that we have a public caterpillar project here we can see that the other ones are private this one is public and we can see that we have uh some code in the project but let's let's look at the jenkins file which is interesting for us so we can see here again you can see from the back so we can see here a very simple pipeline with some linting installing requirements and a very interesting stage the deploy stage which as you can as you

can see can only run on the main branch so we have this stage that runs on the main branch and it uses credentials the flag 2 credentials that we want to steal and it uses them to authenticate to some remote server so just to remind you this is a public project we don't have any right permissions on it only read permissions and we want to influence it somehow to execute code of our choosing so the only possible thing for us to do is to try to fork the project and uh try to try to create a pull request hoping that as similar to other open source projects some kind of pipeline will will run on our new pull

request and maybe we'll try to execute it execute code by doing that somehow so we created the pull request we can see it's on our user and let's try to modify the jenkins file with a malicious payload

so what i'm doing here basically i'll uh i'll remove some unnecessary stages just to make the pipeline itself quicker and we'll run some initial reconnaissance basically we'll try to print the environment variables inside the job itself we'll also try to access the flag 2 secret and print it as a base64 maybe we'll have access to it maybe no but maybe maybe not um and we'll encode the flag itself sorry postcode everything is okay had it a month ago um i'll try to print it as the base 64. uh to circumvent jenkins has a default mechanism that masks credentials that are accidentally printed in the console output log so we will just print it as basics to 64 so we can get the value

itself

so i'm committing the changes and i'm creating a new pull request to the original repository

and if we go back to jenkins

we can see here that we have many pipelines on jenkins two seem relevant to caterpillar we have the test and the prod pipelines we can't see much on those pipelines we have because we have really basic read permissions but we can try to scan for changes and we see that once again

we can see that it did detect our pull request and it ran a pipeline we can see that the pipeline failed which doesn't look good but if we look at the pipeline itself we can see that it did print all the environment variables that we've wanted and we can see that the pipeline itself failed because it didn't have access to the flag to secret that we tried to access but the step before that did run we have the environment variables and if we look at the environment variables we have one interesting variable make it bigger for you uh we have the gt token so we can assume this token is an access token to the kitty server to the source code

which jackies uses in order to clone the repository to the jenkins agent and run the old pipeline using that repository so we'll try to take this token and see what permissions any special permissions that it has so we'll clone the repository

ok so uh so we have the repository we cloned it we cd into it and what we'll try to do now is we'll try to commit something to the main branch maybe this token has permissions on the main branch um so what we'll try to do same thing as before we'll just delete some stages to make things uh run quicker so we'll delete all the unnecessary stages we leave only the deploy stage and same as before we'll try to run this in order to access the the flag itself

and let's commit the changes get all is an alias that

okay guitar is an alias that we made just to do a git add git commit and git push and we see that we successfully pushed our new code as you can see here to the main branch so that means that the token that we've used did have write permissions on the main branch on the repository and we successfully pushed our changes if you go back to jenkins i'll make it shorter for you the caterpillar test project won't run on it because the the other one the pod is the one that's running on the main branch and not on pull requests and we'll scan for changes and we see that the main branch here trigger the

pipeline and if we look at the output we can see here the flag that we try to print and let's put it inside ctfd oh sorry yeah it's a basic c4 let's decode it

yeah so this is the flag

and we're correct we solved the first challenge yeah let's go first challenge so let's talk a bit let's talk a bit about what we've seen here so in this challenge in this challenge we performed an attack that's called ppe poison pipeline execution this attack has three variations direct indirect and public today we perform the public one and we'll only talk about it so to explain 3pe attack we'll see an environment similar to the one that we've seen on the our project itself and you can see that we have a public repository that that is connected to a jenkins pipeline which deploys some artifacts to aws which of course holds our organization's crown jewels so because we only have read permissions

the only thing we can do to influence the project and to poison it is to form the repository and try to modify the jenkins file hoping that the project is configured in a way that runs our malicious jenkins file from the fork and not the jenkins file on the repository itself a good configuration would have been to run the jackets file for the let's say the main branch of the original repository and not run our jenkins file our malicious jenkins file that comes from the pull request so what we try to do is to create a pull request and let's see an example for a different junkies file so we can see here we have

two stages and we can see that the build stage uses aws credentials to run make build and make clean and to perform an attack we'll try to get the environment variables from the pipeline itself and in this attack we'll send them to a server controlled by us let's say if we don't have access to the jenkins console or this is some kind of a private project we'll try to send it over the internet so so what what what what we've done here basically we poisoned the junkies you can see it by the evil jenkins that you see on the right side and jenkins executed the code that we gave it and it sent us it sent us our credential

that we wanted to steal so what we what can we learn from this scenario the the first issue was that secrets were exposed to prs from forks this is generally a better idea and it's recommended to never do it in most modern ci systems it's disabled by default but it's some not maybe someone changed the configuration so the solution here is to prevent access to uh secrets from forks next one is that the access token the gtx token that we've seen in the environment variables uh was configured on the agent itself i can tell you this because i know the configuration behind the scene but this is a common misconfiguration that we've seen people configure a secret on the agent

itself as environment variable and not for the job for the specific job to make life easier but this allows any jobs any job that runs on that agent to access that secret environment variable and the solution here is to limit access to secrets as much as much as possible there are many ways to limit access for secrets and the third issue was that the access token had the right permissions the project itself didn't need right permissions on the repository and it was unnecessary so solution here is to limit the permissions uh to only the permissions that are needed and if in if in any scenario you do need right permissions maybe think you through and

think if you really need those permissions try to avoid it last issue was that no reviews were required before pushing to the main branch basically the main branch wasn't protected by any branch protection rules and we can push to it directly maybe the maintainers of the project thought that only are they are the only ones that have right permissions on the project so they don't need any protections but as you as you see we did get access right access to that project and we could push any code we wanted without any pull request reviews so the solution for this one is configure branch protection rules on all imported branches they require at least one reviewer two is the best

and that's it for the first scenario okay uh moving on to the second attack let's see let's see what it is it's called the mock title challenge uh in this one we need to our objective is to push code to the main branch of the mock of the mocktail teleport and we need to do what's needed uh to steal deflect and credentials uh uh which is stored in the jenkins credential store uh so let's let's have a look at this smok telltale repository it's right here you can see that it's a private repository uh where is attackers that uh reach out uh to you know to an scm organization for the first time uh or to a specific repository we'll try

you probably want to check out uh what our what permissions we have in our hands uh i can save you some times letting you know that this time we have right permission on this private repository um and our objective again is to push code to the main branch i can save you some time and letting you know that there are branch protection rules in place on the main branch otherwise this challenge would be uh too obvious and boring uh so we can't directly push code to the main branch i don't even want to try it uh what we can try and do is uh something similar to what asi just uh presented and this is

um to uh create a pull request maybe uh to that with our own malicious jenkins file try to uh execute malicious code in the pipeline uh we can see that there is a jenkins file here indicating that it's probably linked with jenkins uh we can see that there's one stage here called pr checks uh only one stage in the entire pipeline uh there's lots of logic here um we can see for example that there's a viable called prid being populated uh with the pull request that the trigger the pipeline um what i can see here on jenkins is indeed a pipeline for mock telltale i can see that it runs on the main branch and on pull request so it is

possible theoretically that i can create a pull request modify the jenkins file and execute malicious code in the ci let's check if it works or not so we'll head back to the repository and we modify it quickly the jenkins file i prepared i prepared it here it's a simple command that loads flag and tries to print it maybe this will solve the challenge let's have a look okay so i'll quickly quickly push this code to a new branch and create a pull request out of it this should trigger a pipeline because we saw that the pipeline trigger is triggered by pull requests i'll scan it and let's see we can see that it indeed it was indeed triggered

let's see what happens inside okay so there it is uh there's the console output of the pipeline that uh that was that just uh finished one i can't find my flag here and also i can find lots of logic here but my jenkins file was pretty simple white it just loaded the credentials and printed it to the console output but what i see here is i if you remember we saw that there's a variable called pr id being populated that seems like the logic of the jenkins file that we just modified right not our own malicious jenkins file meaning that this pipeline when running on pull requests it doesn't take the modified jenkins file out of

the pull quest but it always uses the uh jenkins file from the main branch which is protected meaning i can't execute a pp attack as as he just just explained and i need to look for another path uh to push my code to the main branch and um and to access deflect and secret uh so let's have a look again at the jenkins file and find this another path that we need um okay uh so this is the jenkins file again i'll make it a bit bigger uh we can see that it loads our credentials named mock turtle token sounds interesting but we don't really know what this is yet and inside it's a bit messy now it's better

three small but only only this way you'll see it uh properly like the flow we can see lots of logic here but what we mainly see is uh four if conditions uh the first three of them check for some conditions and then they populate a variables named check one uh check two and check three with boolean of uh boolean values uh if they are if these conditions are met then there is the last condition checking if all these three first conditions are met uh this command would run let's check it let's check what it what it does so it's basically a kerala require a cal command sending a post request to the gt api back to the source control and it

reaches the pull request merge endpoints uh it sends also the pull request id it's the pull request that triggered this this pipeline that sends this request and this action is what sent uh do merge now i'm not an expert in the gidi api i'm sure that you're not as well but i'm pretty sure that if it reaches the pull quest merge api endpoint with a post request taxing um asking to merge to merge it it's probably uh an automotive feature white asking to uh automatically merge the pull request that triggered the pipeline um so to summarize it all we have three conditions in the pipeline triggered by pull request and if they are all they are all met um these pull

quests uh merge this uh pipeline merges this bullet quest automatically into the main branch now this sounds exactly like what we need uh but we do need to find a way to meet all these three conditions so our pull request would uh would be merged automatically so let's have a look at its contents i will start from the from this third uh if condition because it's the easiest one to understand uh it's pretty straightforward this is it uh it runs the git div command which uh checks the differences between the pull request and the target branch uh in this case the main branch and it only uh presents the the names uh of the files that were changed

and it looks for a file named version uh so if i'll head back uh to the repository files uh i will see i will see a file i will see a file named version which is right here it's pretty simple pretty straightforward a simple version all i need to do is to modify this file in order to meet the third conditions out of the three conditions that i need to meet okay pretty straightforward let's head back and see the second condition so the second condition this is it sorry for zooming out uh only this way it looks uh properly okay so this second condition is actually built out of two conditions uh two conditions first one uh counts the

number of new line characters in the version file and checks that there are zero characters meaning there's only one line in the version file as we've just seen it make sure that we didn't add any new lines so we moved the existing light out of the version and the second part uh make sure it verifies that the structure of the version stays the same three octets with one or two numbers each we can easily meet this condition right all we need to do is to modify the version file and to keep its structure pretty straightforward okay moving on to the first condition uh this one checks that uh these two variables qtp and git m are equal let's see how they

are evaluated make it uh bigger yeah okay i'll try even though it will uh look pretty crappy like that but uh it's better than nothing okay okay so what you can see here is that uh we have a variable called kit p uh which runs git diff again checking the differences between the pulley quest and the main branch um and it takes all of the rows that were added as part of the pull request from that it it takes only the all the lines that were editory from there it takes only the words that were added and removing uh uh redundant spaces uh the next variable git m which would be uh uh equal to git p uh does the same

but for removed words so basically what this condition does is to counts or to count all of the words that were added the ones that were removed and to make sure that uh we didn't add uh or removed any new words as part of our pull request now as the second third conditions are really easy to meet this one is a bit more challenging right uh if we add our payload uh it will add some new code into the pull request and it won't automatically merge our pull request so we need some way to bypass this condition but if you think of it this condition is pretty naive right it just checks the words that were added and the ones that

were removed and make sure that they are balanced but what we can do if we add around i don't know 14 words in our payload maybe we can simply remove 14 other words from and other files you know in a pull quest right maybe this will work so let's try it out uh by the way i know it sounds naive but we didn't invent this logic we found it on stack overflow and it's a pretty pretty popular answer so uh okay so let's see how we can take advantage of it so we have a mock detail repo here what we can do again we need to modify the version file and keep the structure as is that's

pretty straightforward and and we need to add our payload to the jenkins file uh that will uh load flag 10. this is our objective again i remind you uh and print it to the console output so that's that's here right now right it's loads plugged in uh encode it and print it but now we edit new words as part of our pull request around 14 words again so we can just head to another file like readme which we don't care about uh and just remove some words from here like that i can tell you that uh tell your secret that i counted the number of votes before i reached out to this talk today but

don't tell anyone okay so i have everything saved i believe so let's commit it yeah saved uh guitar again is just uh a shortcut to push everything uh to the remote branch um let's head to the branch that we just created called hack uh and create a pull request out of it

okay there it is you can see uh pull request exists what we can do now is to head to the mock hotel uh jenkins pipeline scan it uh the pull request should run again there it is uh we still expect it to run the the protected jenkins file from the main bench right not our own melissa jenkins file because it's still a pull request but what we do expect to see is that it automatically merges this pull request let's see if it works so we can see that all conditions here resulted as true and the post request using curl was sent using the mock turtle token uh credential to the giddy api so we'll head back again to the mock turtle

repository to the main branch and we can see in the jenkins file that we successfully bypassed the branch protection rules and pushed code through the pipeline with our own malicious payload that takes the flag down and prints it now we expect the pipeline to run again on the main branch there it is because we push new code to the main branch and now it should run our own malicious jenkins file uh with flex and printed and there it is we've got it let's copy it quickly decode it and we've got our secret which we have to validate because we have formal and that's it i see yeah thanks uh okay cool so what we've done in attack number two

is to abuse automotives okay auto modules uh abusing auto modules is where uh there is automation to automatically merge our code our pull request according to pull requests uh that we push and trigger pipeline in order to uh uh use abuse the pipeline to bypass the branch protection rules on the repository side and from there maybe we can introduce our malicious code to these protected branches uh a little bit about automotive rules in general without the offensive side engineers use auto modules sometimes in order to save some time sometimes uh they they create pull requests very simple puddle quests with minor changes and they want to us keep the need of getting another engineers uh approving

it uh and just automatically merge it using the ci pipeline our examples are simple ui changes for example or version bump as we've just seen in the example uh and there are simple ways to implement auto modules uh one of them is to simply uh like manually implementation implement uh in the implemented in the ci uh and approach the scm api like we've seen in the challenge another way is to use a github action uh github actions out of the marketplace there are many of them to approve and merge pull requests uh some of them use the github action spot user in order to uh which is just a user in the in your github organization uh to

approve and merge the pull quest you can some of them request for your personal access token to approve and launch the pull request and similar methods exist in other vendors like circle ci as you can see the screenshots and and in other vendors as well these are examples of uh real uh open source projects that automatically uh merge uh pull requests uh using ci cd pipelines upper one is in a jenkins file in a real open source project are reaching out to the github api uh pull request merge endpoint second one is a github actions workflow and the problem starts when there is a logic uh that checks for in which conditions you should uh

you should merge the pull request in some cases there's no logic at all that you even need to bypass and and again this example of the naive check of words that were added and removed this is a real example from stack overflow we didn't invent it and there are many examples like that you've seen many examples like that in organizations that we worked with and there's a really cool use case uh released by uh in the blog post of rio decay which is uh um one of the top uh hackers today uh for ccd and in general they released a blog post around how they can bypass abuse the automotive rules used in the homebrew project that

you probably all know maybe even use to push code to the brew executable that the entire world uses right are pretty pretty uh harmful so our solution or recommendation is pretty straightforward and just don't auto merge i mean you can auto merge if you can make sure that your entire logic is safe um but it's really hard to know it sometimes sometimes you use third-party tools that you don't know if they have any vulnerabilities uh inside that's what happened in homebrew for example uh so if you can avoid it um please do and if not uh cross your fingers that no one will abuse it uh that's it for the second attack moving on to the third and

last one okay guys i hope you're excited this is the last attack so same as before let's go to the domos challenge let's make it a bit bigger basically in the end here it says hack the door house pipeline stick still flag 9. good luck so we don't have much information about it you can go again to the wonderland project and we see that we have here at almost repository which is also public let's say it's pipeline so we can see here very simple pipeline as before and we can see here one interesting interesting stage called unit tests basically it runs some tests and then uses the flag 9 credential that we want to steal inside

this code block somewhere i guess it's somewhere here inside the shell script and before that we can see that it fetches this specific shell script from a remote location somewhere and it executes it so we want to somehow influence this pipeline we want to steal this secret let me tell you a secret we don't have any right permissions on it we can't influence the repository itself in any way so what we'll try to do is we'll try to uh maybe see if we can influence its supply chain the report called shell maybe we can somehow attack attack it so let's go to let's go look for the project and we can see here we do have a project

in a different organization also a public open source called report code and it has many files here we can also see that it has the report called data sage file so let's look at its jenkins file we can see here two stages pull request and uh release let's start from explaining the release stage so we can see here that it runs only on pushes to the repository itself and it runs some sorry it runs some installation stages and then it uses this environment variables which we don't see it's declared anywhere in the jenkins file so we can assume this var this environment variable is accessible from anywhere inside the job itself uh it's not the same as last uh

is the first challenge it's sort of the agent itself it's accessible by the job that specific job um so what the pipeline here does it writes this environment variable to a file and uses this key to authenticate um to the http server we've seen earlier in order to upload a new version of the report code dot sage file so it's really interesting jenkins file for us if you want to attack it maybe we can influence it somehow and if we go back to the pull request stage you can see that this runs on opened pull requests on events that the gt sends to jenkins on any open pull request and it has two stages it runs echo which prints the title of

the uh pull request and it sends the mail to the maintainers about the new uh pull request so we can see here something very interesting we see here that it uses the title from a pull request the title and the pull request can be controlled by us it's a user input and it uses it inside the bash command so maybe we can try to do a command ejection and run our code inside the pipeline so what we'll do is same as the first one we'll focus the repository

we'll make some change anywhere it doesn't really matter where

and we'll create a pull request to the original repository but it will contain an interesting title

so we're using this title let's see so you can see it clear what we're doing here you can see the two the tick that starts the line it's a command substitution syntax in bash basically it allows you to evaluate a command inside the bash command and we can see here that we're trying to write the same key into a file and to send it to a server in our control so let's run the server quickly

i have a simple http server that can receive uploads um so we run the server let's create a pull request and now we're doing the most important thing in hacking is waiting three two one one and one okay yeah i got it right right in time we can see here that the upload was successful and we have a new file that's called key it wasn't here before i promise you um we can see it's an open ssh private key so what we can try to do is take that key which mod is because the sh like likes to mod and we can try to use that key to upload our malicious reporter file which i prepared

here earlier um to the http server that we've seen before so this is a very simple line as we've seen before we're trying to echo the flag that's what that's what we're putting inside a report called.sage and to encode it as base64 so we can see the upload was successful and the next thing we have to do is let's go to the doormouse pipeline so now we're looking at the pipeline of the original repository the the open source that uses report code we'll scan will uh we see the main branch here let's run the main branch again so i'm just running the same thing again it's like a rerun i don't have any real permissions i didn't change anything on

the project itself and while while it runs it will take some time oh no never mind didn't take time so we have the challenge here the base64 here

and we have the key i hope everyone's ends already

and we've got it thank you last challenge let's challenge okay so let's talk about what we've seen here so so what we've seen here is we compromise the third party basically you compromise part of the supply chain of the dormant repository in order to influence that repository and steal its secrets by attacking a different target so what we've seen in this scenario is similar to what happened not too long ago to a company called codcov codcov is a company that allows you to manage code coverage reports and do analysis on them and attackers attacked it not too long ago using the similar scenario so let's imagine you're a company that uses uh cod cove so you run their

shell script inside your pipeline what you do is you fetch the the shell script that they give you from their google bucket and you run it to upload to upload your report coverage uh your coverage your testing reports to a quartco so what happened in the attack an attacker got access specifically to the google bucket and added a malicious line i can tell you what this line does basically takes the environment environment variables and sends them to a server that's controlled by by the attacker so what happened uh customers that uh run those pipelines that always every in each rerun of the pipeline it fetches the same version of the report called file the same file not the same version

so what happened they fetched the malicious new file that was uploaded by the attacker that got access to the bucket and it sent those customers credentials to the attacker so what we can learn from this scenario uh first thing first issue was that the pipeline was vulnerable to command ejection uh the suggestion is to abstain from using any user control input in your pipelines uh in shell commands and generally in any in any other thing because it can be really vulnerable and can allow people to influence your pipelines second issue was that the pipeline could initiate album connection to any address basically when we send the key from the jenkins itself to our server it did an outboard

connection so if it's possible block outbound connections using an allow list if it's possible last issue was no integrity check or validity was performed on the executable that was fetched during the pipeline so the recommendation is to check the integrity and validity using a digital digital signature and the hash every time you fetch remote resources and run them inside your pipelines there are also some other issues in the scenario but those are the main ones

okay there are a few messages that we can pick out from uh for our summary but we picked a pretty similar one which is that it only takes one flow in your cacd ecosystem uh to compromise your production now the ccd ecosystem and environments are great attack surface and a great attack and a great opportunity for attackers then we already see them using it and taking advantage of it in the recent uh in the in the recent years so if you're defenders cacd uh needs your attention and please don't lie i know that uh i know that this is what you think right now thank you for attending uh we do have some stickers here if you want as you

can reach out later and we are here for questions uh that you can ask now or later on thank you and enjoy the conference [Applause]