CG - Raiders of the Lost Artifacts: Racing for Hidden Treasures in Public GitHub Repositories

thank you so thank you all for coming to this talk I'm very excited to be here before we begin I would like to thank our sponsors um just kidding basically I managed to hack into these organizations and some of them did offer us generous back bounties and let's see how we did it oh sorry

it's on not yet cool so these is the organization I managed to hack it's more impressive right now I guess uh so let's see how it did it so first of all my name Isa vital I'm a security researcher at paloalto networks I have about 20 years of experience in the cyber security space started off as a developer then moved to be a security researcher where I have done vulnerability research mobile security bug bounties and many other cool stuff in the past few years focus on cicd Security First cider the creator of the OAS top 10 cicd security uh which was acquired by AP Alto networks I've been working there ever since uh last year was here exactly one

year ago I gave a talk called actions have consequences The Overlook security risk and third party G actions and one of the concept I raised over there is the permiss I nature of the gab actions Pipeline and how very few organization do manage their pipeline permissions and today we are going to see the consequences of not managing your pipeline permissions so the agenda for today is a new novel attack path I found against public repositories on gab basically allowed to run my my malicious code into your devices even the ones in this very room we are going to see some demo and of course we end our talk with mitigations and takeaways so gab action in a nutshell is

a a widely Ed C ICD platform by GitHub basically code execution as a service enable you to build test and deploy your code and because it's doing a lot of integration with your cloud provider or slack or J it needs secrets so a lot of secret is going on over there and gab allows you to upload art facts as part of your pipeline now these artifacts are not packages they're not like the formal binaries they're mainly designed to share data between jobs or for developers to being able to procas data in in order to debug the pipeline afterwards and stuff like that now this example here is um an artifact from Firebase uh from by Google and uh if the

if the repository is public then the artifact is public and well as well and you can download it so I had this hunch of uh doing Secret scanning on these artifacts because they're being compiled at a very sensitive environment uh and I haven't had any chatter about it anyone that does that so I decided to take the most popular star repositories in gam and try to scan them for secrets and it worked and I even have a name for it I called it artifact which is artifact packed with secets I found various secets for various Cloud providers like CL FM digital ocean COV GitHub and I did found some personal access token which uh the users use to

manage their repository push code Etc H but that is not the focus I did found additional tokens these are belongs to GitHub GitHub token and actions R some token and this token used by the pipeline it's not like the uh previous token we saw it's explicitly used by the pipeline to interact with the repository and we will cover them in a minute so the way things work in the kab action every time a workflow starts an Emeral short live token called gab token created and this is how the pipeline can interact with the the repository clone the code push code and do stuff like that now the permissions of the giab token is basically up to you you can

configure it inside your your yaml pipeline like this and if you don't the default configuration from the repository will kick in as for the actions rtime token it's a different type of token you can use it to access the cash system to upload a new uh artifact or get new crash entry and stuff like that now because secet scanning involves with a lot of false positives I wanted to be sure these tokens are the real deal they are legit so I decided to dig in and try to find out how they ended up in the artifacts from the begin with so I quick quickly find out the the immensely popular Exon checkout I think it's the number one action on

GitHub it used to store the the credentials persist the credentials the giab token inside the local git folder uh by default this is the Bas 64 representation of the G token and on its own it's not a problem but combined with the fact that a lot of users simply check out their code and upload the entire directory including the hidden git folder that's a problem because your token is leaked sorry moving on another problematic pattern I've noticed is that user that use the super popular the super L Which is popular secret um sorry code lint which is supported by many languages it used to print the entire environment variables to logs and when it comes to context of cicd where SEC

secrets are being loaded as environment variables probably not the best idea and these log were uploaded as artifacts as well and uh I reported this Behavior to the maintainance of the super L and this Behavior was fixed so now it's the favorite part of my work hacking I got a lot of tokens different kind of tokens let's try to use them but quickly I found out there is a problem because during the time of my research artifacts were only available after the workflow has ended which means the G up token already expired so by the time I try to use it and push code I always got the same Arrow over and over gun 401 uh because again the token is

invalidated and there's no way I could win win this race condition it was rigged basically so from the majority of tokens I got I lost I lost the majority of tokens from the tokens I got I still got the actions runtime token and moving back this this is the the decoded part of the jwd token I have noticed the expiration time of this token was 6 hours which is plenty of time to commence an attack so this is what I did a deep dive into the upload artifact action I wrote code that DRS the logic of the upload artifact I traced a victim's workflow I remember I have six hours to do so and SW artifact with a malicious

one and let's see a cool demo of doing exactly

that so on the right you can see I'm tracking the uh the workflow I've downloaded an artifact this is by the a real Attack this is a project called schemer a real project on GitHub with many stars and I've noticed the action R token was leaked over there inside the

artifact so I wrote a nice PC that downloads the artifact extract a token and use it and I have swapped the artifact you can see remember I don't have any permissions to this Repository

and just like that I replace replace the artifact with a malicious one [Music] now just a second Now what's the meaning of this attack basically I could achieve an RC in this way remote code execution because if you remember I told you that artifact is used to share data between jobs so a job that tries to download this artifact which can be binary of course and execute it will get compromised the same way developer trying to uh consume the artifact and uh trying to see what's going on in the in their pipeline will get hacked as well now this time I was pretty happy uh from achieving this rce I managed to use the the actions runtime token but then

something truly H magical happened um as I was going through the GitHub change log this is something I do very often I have noticed they have done a complete overhaul to the artifact V4 and now the artifact is available as soon as it is uploaded by the job and I didn't have to uh basically wait for the workflow to end that means I have an opening for race condition right so I need to do the following to trace uh work for triggering identify the exact moment the artifact is available download the artifact extract token and create a branch now why create a branch because I wanted to prove I have right permissions and create branch and

obviously I didn't want to push malicious code and create Branch does require write permissions so this is exactly what I needed to do and I needed to do all of the above under 2 and a half seconds you see the little animation says exactly 2 and a half seconds um because that's the time between the artifact is available and then the job dies which means the token is invalidated but sadly I was nowhere near to win any race condition I simply wasn't fast enough because the downloading the artifact took a lot of time but then I had this really cool idea of why not using gab actions as an attack infrastructure because I was able to be

much closer to the Target it can be triggered remotely which is something I needed and it offers much lower latency and faster downloads so I did exactly that and Route an offensive G workflow I present the repo Reaper and basically all I needed to do is to point this guy towards a potential repo and wait for results and this is the attack flow I needed to monitor pipeline triggering so rot software that tracks when the pipeline is triggered pipeline can be triggered by a a contribut to contribute code or U or a nightly build or something like that the pipeline uploads the artifact I need to download the artifact use the gab token in my case

create a branch and six by the time the uh job ended and the token invalided I don't care because I managed to use it and as a bonus because this attack was fully automated I sent a telegram message to myself so I know something has happened and it worked soon enough I got a lot of these messages meaning basically that the repo reer has managed to compromise the repo this is my favorite one by the way with me doing the dishes and this is the first race I won against uh p Clare by Red Hat I've managed to create a branch called Impala it's a very popular container scanning it has 10K stars on GitHub just as easily I could have

pushed malicious code and let's see a cool demo of the real attack against clear so on the bottom screen I have the software that tracks the pipeline triggering in this case it was a nightly CI on the left side we can see project clear I did wake up at 4: a.m. to record this I do appreciate the effort and on the right side we can see the repo repair workflow waiting for the artifact to be

available in few seconds oh okay so the reaper download the artifact created a branch use the token and you can see by now the job has ended that's means the token is invalidated but I manag to use it nonetheless and let's see the branch we manage to

create cool so we see the new branch inala which of course wasn't part of the Repository

so that was cool but I wanted to win more and more races and sometimes I lost depending on the how big was the artifact so I did some optimization some tweaks like extracting only the files I needed because artifact were compressed and sent a lot of requests per seconds um to identify exactly when the artifact is available it was crucial and I did some communication tweaks like disabl and certificate and remove unnecessary headers and it got me to win this race against Ubuntu uh it's a component that ships out with every obuntu called Adis it's used to integrate with active directory and I've managed to create a new version of it called drunk duck trust me you don't want to download this

version and in the aftermath of this attack I managed to basically avert a massive supply chain attack I managed to compromise Firebase by Google which has 1.6 million projects references on GitHub um as you can imagine if I take a look at your phone I probably see some apps written with the help of Firebase we already discussed about Ubuntu this is a major um distribution of Linux project lay Cyclone DX is a f famous famous U scanner by OAS respectable security vendor I managed to compromise several repositories by Microsoft used internally by the their developers and AWS open search which a popular service by AWS now to the to the good news all you have to do to mitigate this attack is

basically change one line of code instead of using the original upload artifact you can use a tool I wrote I get a action basically that's called upload secure artifact and what I did basically is integrate a layer of secret scanning inside the uh upload process of the artifact so if it found any sence secet being uploaded it will block the upload and fail the pipeline you can try to use it it's it's free this is how it looks like found gith up token fails the pipeline and some takeaways before we end this talk I can't stress enough how important it is to reduce your workflow permissions it can help you mitigate against this attack and many others like

command injection PPE whatever so do it uh integrate artifact scanning as part of cicd uh you can basically use my solution or any other there are plenty good open source out there and uh as as security Defenders we have to see the big pictures from code to cloud in this case because often the vulnerabilities will reside in benign features like like this one thank you very much if you have any [Applause] questions any questions yeah was there a response on how ared or redesigned yes so ga basically said it's up to the users not to upload secets within their artifacts I think this response is somewhat limited because they can offer secret scanning as they do in other

components of their ecosystem but uh they don't they also can uh change the behavior to action check out not to prist credentials by default which is something that is lesser known by the users I wasn't aware until I started this research myself um the actions runtime token variable how did you find it um so going through uh logs I notice the environment variables and it is less or know by the way it's not formally documented this actions rtime token but once I've noticed it I start to uh dig around and see how it is being used so I reversed a little bit the upload action the original action and I know L what is the usage of

it thank you very much thank you [Applause]