Are your GitHub Actions secure?

foreign [Music]

hello uh so my name is Luis fonse I work as a product security engineer at a bank and I've been working with GitHub action for a while and trying to secure some some GitHub actions for some companies and I've seen some common patterns let's say and bad designs that I want to bring some awareness of and that's basically what I'm going to do in this meeting in this presentation so first of all um I'm not going to be talking about things like cicd top 10 or something like that just common designs that I've seen everybody implementing that had some security implementations so I hope it can be helpful for you guys so first of all what is GitHub actions

so it's basically a CI CD tool from GitHub where you can basically build deploy test your your code and it works basically like Jenkins or or a bit uh bit bucket and so on and um it has some nice features you basically create uh definitions on on the codes and it will run the CI for you but this brings also some some challenges mainly uh securing these these pipelines um so most of the things that I see are basically on the on the cd part so on the deployment and this is because you either need to to deploy for example to AWS or to Google cloud or something like that and you need to have some kind of

permissions uh to to push your code either credentials um Runners with with whitelists and so on and there are a lot of things that can go wrong here so first of all you can basically uh somehow leverage the secrets on on GitHub um you can compromise the machines that do the deployment for you and from there you can do a lot of privilege escalation or lateral movements and I've seen for example a case where a user have a regular right access to a GitHub repo and it got from there to an AWS admin uh so that's that's how bad that can go and let's hope that we don't see a lot of those but it can go that way so um

first of all the most important part is to understand how GitHub actions work right so as long as you have right permissions to a repository you can basically run the the pipelines the the actions itself or the workflows and you can basically change them as you want and do whatever you want with them so um let me let me show you how you can break with with all of those things so first of all um GitHub has a thing called secrets and it's basically supposed for you to to store your your passwords your credentials there so that you don't hard code them in the code but the name can be misleading right so when you actually

have a secret you can extract that secret you can dump it and although you cannot do it from the UI it gives you a false sense of security because if you can actually control the action or the workflow and for that you only need right permissions you can basically extract any secret configured in a repo or in an organization so that's that's mainly the the first thing that that you see failing so a lot of companies create secrets and store them most of them even in organization level and you can just create a random repository with no permissions you create an action and you have access to all the organization sequence and it's just dumping all of

them quite easily so in this image you have a definition of a workflow at the left and at the right you have the Run of that workflow and basically as you can see if you try to do an echo of a secret on GitHub uh GitHub will automatically hide the secret from you in the output the issue with this is that GitHub is basically doing a fine search and replace so if you have a secret with a value test and if you have a test in the output GitHub will filter that word test so if you start doing things like basically for encoding it or print a character in the space a character a space or just a character per line you

can easily dump a secret obviously you can do things much more complex like doing requests to external web servers to send the secrets but basically as as long as you don't print the secret as is you can basically uh print it in the output um and this is where it all starts right so most of things are secured with Secrets still and you can start dumping them quite easily again you just need the right access to the repo uh Branch protections don't work uh because as soon as you create a new repo you can control the workflow and you can change the entire workflow and put that whatever you want and your version of the workflow will run even before it's

merged to the default Branch to master your Dev or wherever is the default Branch so this is one of the most common things usually one of the most dangerous as well so still on secret uh when you run the workflow GitHub generates automatically for you uh a secret called GitHub token and this is a femoral so it only lasts as long as your your action is running so um the thing about this token is that it has basically by default uh write permissions to the repository that you're working on so you can basically call the API and do operations on on the repo level so for example here uh you have Oh I thought I thought then I

have an example but here you can see uh the default permissions on the workflow which has read and write permissions and you can also uh uh have the token to you the token can also be used to to approve PR so uh there was a security issue reported uh while ago where um Branch protections that had like a minimal one approval for merging PRS could basically bypass that by using the GitHub token to to approve the pr and merge it so obviously you can also leverage this token to do malicious things it's it shouldn't be that necessary because if you have right permissions to the repo by default you don't need to use this token but this gets more

interesting when when you start using third-party actions right so basically an action it's it's can be compared like a library in a piece of code so it's something that you import to do a generic task or a common task and you can send that GitHub token to to the action to to do something for you for example in this case we have a a creation of a release in GitHub and you need to to send the GitHub token for it to be able to to call the API and do the release process so by default when you're actually sending this GitHub token to the to the action it will inherit all the the right permissions that the token has right

um what can go wrong with with this is that you don't know what the action is doing and for example in this case you you just have the the action tag to version three but you can have multiple version threes and the one lay the latest one can do something that you're not expecting for example and leveraging that token for for something that it's not supposed to so it's really hard to control um what what the what the action is actually doing um unless you you for example pin it to the commit to the committee and you review that that got me to make sure it's safe um so one thing that you should do with

with with this so whenever you're you're sending GitHub tokens to to third-party actions um you should always make sure that you specify which permissions the token can can have so there's a there's an option on the workflows where you can say this token will only have read permissions to to the repo for example and uh you should trim it down to to the basically Essentials of what the the action needs to to use the token for uh I've seen I've also seen some some common issues with the guitar tokens on on malicious third-party actions that um Leverage The the token permissions for things that shouldn't be uh using it okay um in in this case uh we have an example

of running the our workflow in your own Branch as I said before so let's say that you have branching policies uh branching rules in place so that you need let's say two two reviews to merge to master for example and you have an entire workflow to push to to production let's say and you're quite confident that you have a secret there and can't be dumped because uh you need you need to two approvals to actually merge the code so that's that's not what happens and this is a a big misconception and there there are a lot of security issues also around this because people don't understand that you can create your own branch and just

change the workflow and run your version of the workflow even before being approved so approvals mean nothing code owners don't do anything either so the the workflows can always be controlled by anybody with with right access to the to the repo so here the same thing so you have on the on the left side an example of a change on on my own branch and on the right you can see that the changes that I did run and took effect so another cool thing is the workflow dispatch tag so this is something that you specify on on the workflows and basically mean that as soon as you add this you can run the workflow from the

UI so by default you run workflows on trigger events like when I do a push when I do a PR on this case you can basically manually invoke a workflow and you can even specify user input if you want to earn it and usually there's also a common injection here when you when you ask for inputs because they almost always end up in in a shell script but yeah so a thing about the workflow dispatch is that you can run the workflow dispatch also from any branch that you want and uh a lot of times when developers add this this workflow uh Dispatch they don't have that in into account so you can basically go to the

UI run a workflow on your own branch and for example deploy a malicious version of your code another another issue with that is that um I don't know if you can see it here uh nope but basically uh let me uh yeah so I think I have a line there so when when you create this you need to actually push this master to be um you need to push a version a code or a version of the code with a workflow dispatch to master to actually show up in the UI so if you don't have this in the master Branch the first time you don't have this option but as soon as you put this option on on the main

branch it will show up in UI and you can basically call the the workflow from from any from any branch so all all of these issues are can can happen because when you work on an organization You're Expecting to have some segregation all right and some some restrictions about permission so if if I have a team working with repo a b and c I I don't want them to have access to 2D and E for example right because they shouldn't have access to things they don't need to do their work and when when you start um thinking about all of these things and and The Insider thread risk that you have here most of the times companies a

member inside the company can leverage all of these permissions and basically take down a lot of things that they they should not have access okay another cool thing and that GitHub implemented and it's somehow trying to to help with the secrets it's the the concept of environment so environments allow you to Define like a set of rules like you can say that a workflow will only run um on this specific Branch or for this workflow to run I need two approvals from x y and z so that's interesting but the thing is that you define which environments are associated to the workflows in the workflow file so if you own the workflow file right you can just remove the the

environment tag and it's game over so although environments work and you have you have to be careful with that because a cool thing also about environments is that you can store secrets in environment so instead of having Secrets associated with the repo you can associate them with the environment and they are only injected in your workflow if all the conditions in the environment pass so let's say that I have a password to deploy to AWS obviously you shouldn't do this way but let's say you have um on on an environment unless the environment passes like with two approvals and it's running from the main branch the that secret will never be injected into the into the workflow so

for environments to actually work you need to have secrets and those Secrets need to do something uh interesting otherwise you can just remove the tag and you bypass all the all the features of environments and it's also a common misconception that people add environments with approvals and you just go to the workflow and just remove it so another nice thing and because I think I think I made my point that secrets are are hard to to manage on on GitHub another thing is to use cell full set Runner so by default when you run a workflow it runs on on a container on controlled by by GitHub right so they have a this container is called Runner

by the way so you have a concept there that is cell phones Runners and basically you can configure your own machine anywhere to run the the workflow for you so for example if you configure an an ec2 or some container on AWS it's much cheaper than being running a a runner on on GitHub so that's mainly the main motivation to use cell phones Runners most of the times but they have some some cool advantages as well so when you're using a self-hosted runner you don't need to to have secrets on GitHub because if you have a cell phone server for example in AWS you can leverage that and just use the permissions that your Runner has

inside the AWS to do wherever you need to but this also comes with some challenges so if you do this you basically have a back door to your systems right so uh if I control the workflow and I can run whatever I want in my workflow I can basically control the the cell phone server Runner which can be a huge risk as well so there are some pros and cons on on using cell phone surrounders you need to make sure if you use one you need to make sure that you lock them down pretty pretty well otherwise you'll have some issues the cool thing is that when you when you configure yourself with a runner it's

the cell phone surrender that connects to GitHub and not the other way around so that's why you don't need any kind of permissions on GitHub so even admin users don't control the runner by itself just from the workflows so having seen all of this uh let's see some ways of of actually exploiting GitHub actions and try to be silent um so that you you don't get caught and nobody sees what you're doing so obviously there are tons of ways and the imagination is the limit right but there are some some interesting things of what you can do to hide your tracks so in this case um you can see that I just run a workflow and for each step in a workflow

GitHub prints what it's going to do so in this case I'm pushing a version of an extension that I have and you can see basically all the code that it's being run on the on the workflow so if you change a workflow and add some malicious code there it will be printed as as the output of the step so what what you can do is that you can find for example a workflow that is calling an external script and you can change the external script so you create your own branch you change that script and you don't see anything in the logs because if you if you're just calling a script from the workflow what you see in the output is that you

call the script you don't see the content of the script right so and you can start from there and you can create even more complex things so for example in this case uh this is yeah a little bit complex uh for for trying to to be quiet but the idea is that you have an external server and you find a workflow that is calling a batch script or something and you create your own Branch you change the the best script you add your logic for example to to retrieve some malicious code from a server and after retrieving it you shut down the server so nobody else can do the same request and get the same code

and uh you basically run your your script and you shut down the server and you delete the branch so from there it will be really hard to try to understand what you did uh because you basically deleted the branch with with the changes on your script so it's it's not easy to find these kinds of this kind of actions and uh that's why you should always try to limit the GitHub actions as best as possible and I have some tips for that as well so first of all as I said before it's securing the GitHub token so you can basically specify the permissions that GitHub token will have so when you're sending it to a third-party action uh

they they cannot abuse that much of your token so another cool one um is to is to specify which actions you you allow to be run on your organization on or you're on your own ripples uh if you're doing this on your own ripples it's not that that important because uh probably nobody else will control them right uh but when you're when we're talking about organizations anybody can just use any any third-party action and there can be a lot of issues with that so you should try to restrict which actions can be run and you can even pin them to specific commit ashes which is the The best scenario here because you can review those specific comments and

make sure there are no issues if you pin them for a version uh I don't know if you guys try this but you can just delete the version that you created on GitHub and create the exactly same version with different codes so it's not that worth it okay another one and I I cover that one as well it's disabling the the pr approvals permissions from the GitHub a lot of companies have like one approval two uh if it's just one it's critical because you can basically bypass and merge to master wherever you want and that's basically it because I don't have more time as well so questions

thank you Luis any any question for Luis we have before our break anyone okay thank you that was easy thank you thank you all

[Music]