BSides Berlin 2023: Mackenzie Jackson - The attackers guide to exploiting secrets in the wild

I think we can all agree that no one really wants their secrets to be out there in the internet but I think only a very small amount of people uh is interested in your kindergarten secrets and our next speaker really focuses on these kinds of secrets that can be turned into money I would say and also how to find them so please welcome McKenzie

all right uh thank you so much it's a real privilege to be here uh I've been to a lot of conferences but I have to say the quality of all the talks leading up to this has been extraordinar high which makes me much more nervous than normal but uh we'll give it our best shot uh so just a little bit about me I'm from AO or New Zealand uh I live in the Netherlands and I work for a French company called get Guardian I just like to keep everyone guessing uh you can find me anywhere on social media at the handle Advocate Ma and I'm also the host of the security repo podcast it's my mom's favorite podcast

she highly recommends it to all of you so make sure you check it make her very happy to check it out uh okay so let's start off with what are secrets I know everyone in the room probably knows this but just in case to get everyone to the same page what I'm talking about when I'm referring to secrets are digital authentication credentials and these are typically your API Keys security certificates or anything that allows you to access different services or ingest decrypt data they're basically the keys to the kingdom and every attacker is after these because they provide a way of legitimately authorizing themselves into different Services which means that they can often go undetected they're a great

way to not only get initial access but also to move laterally through systems and prolong their access and infect their way into different areas so it's really important that we keep these protected but as you'll see you know these Secrets actually end out sprawling uh everywhere um and you can find them in the weirdest of places uh and attackers really rely on this so what I want to do in this talk is go through a couple of different techniques that attackers use to discover these secrets and I have some demos of exactly how they do that so if you could all pray to the demo gods that everything works well I'll appreciate that but we'll talk a

couple of ways the first one we're going to talk about is finding secrets and source code we'll talk about public source code and also uh accessing private source code I want to talk about discovering secrets in containers uh in this case I'll use Docker images uh finding secrets in mobile applications and then uh at the end we'll talk about how to prevent our secrets from leaking which can basically just be summarized by don't hardcode them but I have a couple more slides uh all right let's go into the first part abusing the GitHub API so you're probably all familiar with this site this is GitHub the largest uh store of publicly available source code uh on

the planet what's interesting about GitHub is that for developers is not just a place to interact with open source code it's a place for them to engage with the community it's a place for them to show their portfolio uh and it's a a way for them to kind of work on their own code as well and often you can have a a a kind of a private GitHub account that's assigned to your work and then you have a public one but you use the same account for both in most cases why this is interesting is that even if your organization has no exposure to GitHub it will through its developers which makes it really interesting place

to Target particularly because of the sheer amount of information that is there so some stats um uh that we kind of uncovered at good Guardian last year there was over a billion commits made publicly to public repositories there was 85 million new repositories made um in the public spaces and about 94 million developers according to GitHub were using it uh one of the things that we've been doing for about six years at gardian is looking through all of this public information and trying to identify sensitive information there so we actually scanned all of those billion commits last year to try and identify Secrets uh and we found a couple which probably won't surprise anyone uh but in

total we found 10 million Secrets last year in public repositories on GitHub um and some people might question this and say well how do we know that they're real Secrets how do we know these aren't just highry strings or test keys and the answer is that no we have individual detectors for each secret we have about 400 individual detectors that we use and in about 70% of the cases we can validate the secrets with the provider so not only find that is this secret fit the format of an API key but can we check with it's provider and see that it's valid so just a little bit of kind of what we found 20% uh of the keys so

if you do some quick maths 10 million 20% 2 million Keys um were for cloud providers and all of these we can validate so 2 million valid cloud provider keys were pushed publicly in GitHub last year which is a pretty extraordinary number and we also find lots of interesting things like the very last one 3.8% is your Version Control platform keys so basically you've leaked your keys to your private code and your public code uh which kind of makes a private code not that private um so how do we do this as an attacker how do we find keys on there the first one I'm going to talk about very quickly um but it's not my favorite

uh and I'll explain why but that's just using the GitHub search feature it's called GitHub doing um in this case we're looking for a file name called credentials good file name um AWS access key you're going to get a lot of results from this there's a bunch of Dos that you can do but it's not that interesting you will find valid keys this method but you're going to have to sift through a lot of information and the key thing about git is that in all Version Control Systems it keeps it persists to history so if you do something on a development branch and you later remove it it's still in the history what the search feature does is

it looks at the latest version and only the latest version but where you'll find most of the secrets are buried in history or pre versions so this doesn't get a whole lot of interesting information but luckily there's a much more interesting way uh to be able to discover secrets and that's using the GitHub public API so anyone can go to this address api. github.com events you don't need any authentication although you will be rate Limited at some point um and that has a complete Ledger of everything that's happening in public GitHub and there's two events that we're specifically looking for the public event this is the most interesting because this is when a private repository turns but with it comes all

the history and the other one is the push event uh which is when you push code to a public repository so we can quickly look at that this here is the API um and as I said anyone can go to this but what's interesting is that when we're dealing with GitHub and the scale at GitHub you're kind of protected by how much there is and as as I said there's 10 million API keys and if you're looking for a specific API key for an organization even if you just had all the valid Keys it would still take a lot but what we can do is we can uh sort through the information that we have for

example we can look at that email address of the authors of this which again is all public so here we have uh one interesting one I just saw looking down was for Yandex which is they call it Russian Google um and so if you were just looking to Target this company then you could just look filter out the domains and then once someone commits with that domain you can look at their GitHub ID and then be interested in everything they do so it's very easy easy to start filtering these results to actually get to something more interesting so do adversaries actually use this as a method to gain access to Keys uh and the answer is yes and I have

a quick experiment that I'd like to do uh very quickly what I'm going to do is I'm going to create uh a honey token so this here you'll see is an AWS key it's actually a real AWS key but before any of you get too excited and try and do any crypto mining with it it won't provide any access instead what this is is a honey pot it's a honey token which if someone tries to use it I'll be alerted so I'm now going to commit a cardinal sin I'm going to paste this into a file a Dov file which should never be in a git repository and I'm going to make this public and we're

going to Commit This publicly so why would I do such a a silly thing I don't know if that worked actually oh yeah so why would I do such a silly thing I basically want to prove the point that a attack is are monitoring this by continuing all my talk and as we get near the end we'll come back and we'll see if anyone has tried to access this so in here KY that was quick it's usually not that quick I'll come back it'll be more impressive later um so is someone going to find that in that time the answer is yes someone already has uh but some more people will too um and it's not H these aren't

people refreshing the GitHub API you know these are Bots that are scanning for that this is an AWS key it's a prefix key which means it's easy to find uh so it's the perfect key to try and run this experiment on um okay but there has been some breaches of this this one's interesting not because of how severe it was um but because of how who was involved so Toyota car manufacturer they were creating a mobile application called tconnect they built this application with a third party contractor that contractor an employee for them accidentally leaked some of this code in a personal public GitHub repository inside that code were hardcoded credentials to the database with a whole bunch of personal

information of anyone that was using that which is interesting for an attacker because I have personal information I know that they have they have a Toyota and I could craft some very specific campaigns to Target them so this is an interesting example it sat there for five years before a security researcher found it and notified Toyota but in that time as we just saw attackers are very quick so there's no saying that an attacker didn't find this beforehand um and what's so interesting about this is that Toyota almost had nothing to do with this leak apart from the you know the fact that they had a contractor that had an employee that did it and it just goes to show the kind of

scale of which we can find uh Keys like this out okay uh so I have another little demo here which is what if you want to do the opposite what have you wanted to see if your keys have actually ever leaked publicly in GitHub um and you can actually do that now pretty easily uh thanks to a new little service and so what I have is I have a file on here called secrets. txt I wouldn't recommend that you create a file called secrets. txt but for the demo purposes this is it and I have some keys in here I have a gitlab key an open API key and a stripe key and what I

wanted to do is I want to see if these have ever made their way into public GitHub now we have a database that getg guardian of over six years of all the secrets that have leaked on GitHub so if you were to query that database you could find out if your keys aak and a lot of people might be squirming thinking that I may suggest sending your plain text credentials to us to query because if they weren't leaked before they would be after that um but we're not going to do that and what we're going to do is we're going to create a hash of our keys which if we now uh look we have a a payload txt file and

this just creates a hash of each line so our keys are in here and what we're going to do is we can send this uh to the G Guardian uh database to query which will send us back uh an encrypted file and if we decrypt that uh then we can find out if our secrets are leak so in this case it's found four leak secrets from that and it tells us exactly where those Secrets have leaked so if you want to find out if your secrets have leaked you can can now do that uh with this service and you do it by hashing it so that the secrets are actually never exposed but it's a cool

little thing that if you wanted to find out uh if you are because in some cases people like to may not even realize that their their secrets have have actually leaked uh so the service is called has my secret leaked we have a very fancy UI that's completely useless but it gives you instructions of how to use the CLI which is much more useful um I probably shouldn't have said that but never mind um all right now let's talk about accessing private source code so public source code is really interesting because it's public there's lots of information and I can show big numbers on the screen like 10 million and everyone's impressed but private source

code is really what you want to get access to when you're trying to exploit secrets so private source code is really bad at staying private there's been a huge amount of leaks from very big companies over the years so last year Microsoft Nvidia Samsung all had large portions of their source code leaked twitch the year before had all of their source code leak and these are companies that I wouldn't consider to be amateurs in security particularly not places like Microsoft so why why and how did these companies get their source code leaked and the problem is that source code is distributed in so many different places it's accessed by so many different people that the attack surface for it is

really wide you can Target individuals but you can also Target misconfigurations so this makes it really interesting now a lot of people will say yes but source code shouldn't inherently be private or shouldn't inherently be sensitive and whilst I would agree with that the problem is is what's inside that source code generally is when we're talking about Secrets uh and we find a huge amount of secrets in private source code that really surprises anyone when they actually have a look in there particularly when we start looking in the history so we have an example here a typical company of about 400 developers that we look at at GG guardian we we'll typically find on our initial scans 13,000 secrets in

their private repositories so that's just kind of based off some average on there a thousand of those Secrets will be unique so it's not 13,000 unique Secrets it's 1,000 unique secrets that have been duplicated because that's what happens when they spread into systems they sprawl everywhere but unless you have a tool that can tell you that they're duplicates you have to look at each one so the same company may have four appsec engineers and then anyone here working appsec will know that I'm probably being pretty generous with the 1 to 100 ratio but imagine trying to deal with 13,000 incidents of Secrets as a team of four it's completely impossible so you do the next best thing

which is just ignore the problem and hope it goes away um but you know this is this is where we are in it and attackers also know this and if they can get access to the source code get into the history then they can uncover huge amounts of sensitive information so we can look at one kind of case of this where source code has publicly leaked and that's with twitch so they had their entire repositories all of them are leaked from their core products to their secret tools that no one was meant to know about there's about 3 million documents and 6,000 repositories so this was uh uploaded made publicly available or as I like to say involuntarily open sourced

um and we scanned this for secrets and we found 6,600 exposed secrets that sounds like a large number but actually that's way better than the standard of what we' see so Props to Twitch for only having 6,000 Secrets leaked out but they still had 194 AWS Keys 14 GitHub OS tokens so that would kind of persist access and four stripe Keys uh so Payment Systems now in this case the attacker did them a huge favor by uh making it public because had they wanted to be more malicious they could have not let notified anyone about this and just sold those Secrets or run more attacks so that probably is the best case scenario if your source code gets leaked

like this um but still probably not a great week for the security team at twitch so how does Private source code get accessed uh with these so the twitch case was a misconfiguration we can talk about that one that's really popular at the moment is fishing um and you'll know that developers that work with developers know that they're really getting targeted more and more through fishing campaigns to try and access the their repositories um and a friend of mine who works for a penetration test company called net regard knows this all too well and pretty much in every penetration test he does he tries to get access to GitHub repositories of developers so how he did this was he

bought the domain git lhub so it kind of looked close enough uh that people wouldn't be suspicious and he created a complete mirror of GitHub so anything that was happening on GitHub was also happening on git lhub so he then was targeting some people he sent them out this email with get Al Hub and the next part of the story is you'd probably expect me to say that they clicked on it and then he had access to the source code that didn't happen they actually didn't click on it they they figured out that it was fishing they reported it and then the story kind of ends but it doesn't what ended up happening was uh Philip kind of went on

tried to find other methods of attack for this for this uh appointment um and he forgot all about it but he obviously kept get lhub running because he might need that in the future then something weird happened someone on Twitter was sharing a repository but accidentally made a spelling mistake and shared G lhub instead of GitHub now that wasn't be too bad it's not like this tweet went viral but it did create something weird which was that search engines now discovered get lhub and then you started seeing results for open source reposit that were from G alub instead of GitHub that were appearing before the original ones um and then Philipe who was trying to Target one developer actually

realized that he was targeting uh thousands of developers he found out about this because he got a notification that his logs were full uh and they were full of uh passwords for other people now there's moments in every penetration test his life where I'm sure they wonder which path they should go down I'm sure this was one of them and uh he decided to go down the the good path here um but there's lots of other ways that source code because this is just a kind of a fun example of you know your source code can be accessed by lots of lots of different people because it it has so much access but misconfigurations are another big one

when you go and initiate a new get repository you run the command get in nit and then this folder here that circled popped up which is your dogit directory this is where all your metadata and history is stored so when I talk about when you work on Old branches and you're you're you're going back in the history it's in this folder it's kind of buried in here but what often happens is this folder which contains all your source code Forever becomes public so cyber news did a study they did some wides scale scanning and they actually discovered that about two they found two million dotg uh directories that were exposed to the public internet so that's basically 2 million times

where companies think that their source code is private but is actually accessible if you just bother to take another couple of steps uh to look and yes this happens frequently as a attack pass uh there's better examples but this one it's just with an organization everyone knows uh the United Nations uh there was a hacking group called secure Samurai they did some domain numeration just to find out lots of subdomains and check if there's any got dog directories they found one with hardcoded uh credentials for a database um and even more kind of scary they found another subdomain that had a file called dogit credentials in it uh which I can understand if you accidentally expose a

dogit directory not thinking it's sensitive but doget credentials kind of has it in the name a little bit which had access to their their their private uh their private source code uh all right so we can go do uh another little demo here of looking at uh how this how an attacker would go about this so the first step that we need to do is to enumerate through our domain to try and find something interesting so I'm just using Alia as an example uh they have a a program that allows me to query this and I know there's no vulnerabilities so I don't want to to uh expose any but if we just rub run the tool SubFinder with algolia

doom this will enumerate through uh all of its uh no and domains you can combine this with another Co tool called chaos which will find kind of additional domains for it and what this will do is it will spit out a huge number of subdomains that belong to ala.com and why this is interesting is it's not that common that you will actually find a exposed dogit directory on algolia docomo docomo algolia

docomo and the internet's a bit slow I was really trying to talk as much as I can so that hopefully uh it will there we are now we have it uh so you know at list of domains and then the next step that we can do is just use another tool called git

scanner and we just input our domain here and it's going to go through and check for us anything so you'll see two results here not vulnerable that means there's no dog directory there maybe vulnerable means that there is a dotg directory there but we don't have the permissions to access it but often if you take another step and try and access individual files and you you can often find uh you can find that you can sort of access things and get into different areas so worth further investigating on some of those particularly if the domain is interesting so this is just a quick example really easy of how you can find uh some source code that may not be uh

may not be private and then recreate those repositories and then from there you obviously can scan for Secrets uh and gain your access so this is the process that I just use it chaos SubFinder get scanner to to find them and build them and then I use a tool called GG Shield uh to to scan to scan them so why is source code such a problem well it's a problem for a couple of reasons but the the number one is I've already talked about it is about the history and a lot of developers will say to you come on McKenzie I'm not going to leak my secrets I'm not an idiot um and but there's scenarios where

that often happens so here is one where a developer this is an example of a g tree so if you're not familiar with software engineering this may be a little bit foreign but basically is how we build software into the latest version what we have there in the main branch is what the latest version of the software is this is kind of when you download something when you visit it visit it this is the production version let's say but off that if we need to create little features we'll create what we call branches so a developer in a team might work on a little branch and I say hey I need you to connect to this

database and the first thing that they want to do is just check all their connection work so they hard Cod a key not because they're going to leave it in there they're not idiots they're going to remove it but they just want to check that it's all working the next commit maybe they remove it add it as an environment variable or however else um and then they continue on by the time code review comes down that that secret is 200 commits deep the code reviewer just looks at the latest version with the latest version on the main branch and that all gets merged and no one actually knows that there's a secret that's persisted in the history so this

is why it's so interesting to look in the history of these things because you can really see how the secret got ignored and how the developer that did it didn't actually know that they were making a mistake autogenerated files so if debug logs get in your code repository it often happens things like that can have environment variables which are secrets uh no dot get ignore file so that's a simple way to make sure that none of these sensitive files get accidentally into your git repository which combined with the secret. txt example I had before uh you know if you do get add all that gets captured uh and you can accidentally expose that you can find

keys and template files like if you're creating ajango project it will have the keys embedded in it now you have to remove those it's not insecure how it does it you just have to know that you need to remove them um and then also we just see a lot of people sharing uh secrets on git because they think that it's far more private than what it actually is all right let's move away from source code and get into some other interesting areas so let's look at uh Docker images finding Secrets inside Docker images uh I wasn't sure if I was going to do packages like on pie or Docker but I decided Docker because there's some cool research uh that made

it a little bit more interesting uh but this is Docker Hub so Docker Hub is the largest store of publicly available Docker images in the world if you don't know what a Docker image is uh then it's basically a mini virtual machine think of it as a way to to package your application in with an environment with an operating system that it can run within so you know it means that you can not only send out your application but send it out with an environment that it will work on so that's what a Docker image is really really cool technology and a lot of people choose to ship their applications in Docker images and they

store them on Docker Hub so people can pull them and use them there's about 10 million public Docker images on Docker Hub now there are some really cool research done recently by a German University so Props Germany um from uh rwth University which looked at how many what percentage of Docker images contained plain tech secrets and the answer was 88.5% of Docker images contained plain text Secrets this is absolutely wild um and not they they went a little bit further and they told us how many Secrets they actually found as well so they scan a total of 337,000 Docker images we've also done research at this with get guardan but this this is far more extensive than what we have

done so it's it's a better uh case to point to and they also found 52,000 private Keys one of the coolest Parts about the research was that they actually created public keys from these private keys and matched them with no one public keys so they actually were not because they were able to prove that these private keys were actually sensitive uh which I thought was really cool uh so this is goes to show massive problem with trying to with identifying Secrets inside docket images so the name of the study was secrets revealed in container images an internet wide study on occurrence and impact uh these were the authors just to give them uh credit on that

study so have secrets and Docker images be used by attackers yes they have uh regularly the the worst one was one called uh was one from the code COV attack this was a supply chain attack uh so code COV was a code coverage tool and it sat in your cicd pipeline that may have sounded like mumble jumble to some people but basically when you test your application test it as working you do this in a cicd pipeline generally uh and what you want to do is you want to be able to test all of your application code C lets you know how much of your application is being tested how much coverage you have in your tests the

point I make on that is it doesn't do something wildly sophisticated it doesn't have access to huge amounts of things it's just a good tool that did a simple job well how you deployed this application was via the docker image but inside the docker image the publicly available Docker image the main image was a hardcoded credential this credential gave access to some source code which was a bash uploader script so when the docker image was running when code COV was running it actually communicated externally with this script and the keys that were leaked were read right uh to this script so they were able to modify the script and therefore modify the code COV application and they

turned it malicious so what did this malicious code did it was actually only one line it was a very good line though um and it said every time code COV runs I want you to dump all the environment variables and I want you to send them to me the attacker so when you're testing your application it does it in an environment and that environment needs to have access to your database to your third party tools to your infrastructure because if it doesn't have access to that then it can't you know test the build the application and test it so all of those secrets are actually in your environment now some people say yes but we use test credentials for testing not

production and that's great and I hope that you do but some secrets you can't get to avoid like having uh secrets for your private code repositories and that's what the attackers were after so they were able to move from the public Docker image into the private code repositories of the people that were using Code COV so it was a I was going to say it was a cool attack it was an interesting attack it sounds better um so one of the companies there was a few companies rapid s monday.com twio none of these kind of did anything wrong they were just caught up in a supply chain attack so I'm not going to pick on them

apart from one of them one of them I'll pick on and that's hashy Corp so Hashi Corp is a pretty good company and uh they create interesting products one of them is a Secrets manager they invented the term secret sprawl and they said that with our great product developers will no longer need to have access to Secrets they'll never be in your repositories you won't need to hardcode them we're going to solve your problem and fairness to Haik is a great product but because the attack has got access to their private source code Hy Corp had to announce that they had secrets in their private repositories the point is if Hy Corp has secrets in their repositories

there's no hope for anyone else all right so how do we discover Secrets inside uh Docker images I have another very simple uh demo here first I'll just show you this one thing that you need to understand this is what a Docker image looks like on the inside it's built up in layers this is a tool called Dive I won't talk about it too much other than say what you see on the right is python files that's source code there's source code in a Docker image a lot of people think that this is black magic and that you can't extract anything but there's source code in your files and you can pull that out but you don't even

actually need to because uh you can scan a docket image directly I have one here I'm using a tool GG Shield again which is going to pull this Docker image and then it's just going to scan it for any secrets and in it and I can pipe this uh it'll take a few seconds hopefully um and in this case it's already found a bunch of Secrets these are all fake um but you'll get the get the idea from from that so very easy to uncover uh Secrets inside Docker images I'm running a little bit low on time so I'll go quickly here the last part that I want to talk about is finding Secrets inside

mobile applications so what are mobile applications when we look uh on Play Store our app store this is what we see but what are these icons so there's two types but they're all nonhuman readable and everyone knows that non-human readable means it's secure right that's how that works it doesn't because we can always get back to a human readable version from it so if you're dealing with apple applications you have a IPA file and Android has a APK file and these are basically zip folders apple is slightly harder to get back to the original source but Android is very easy and apple by no means uh you can't that you can't do that so very quickly I have another

little demo that I want to do here and the first thing is that I have uh this file here my APK my Android app.apk the first step I need to do is to get back to some source code that I can read so using a tool called JX I'll decompile that and I'll get back to the source code so that I can then look through that and try and find any secrets Now the secrets happen in mobile applications for different reasons um but it's quite scary how often they are inside there and we're now done so now in my Android app folder I can go back to my sources and I can see all the source

code which will include interesting things like the Android manifest which is often we will find a lot of Secrets uh from them so this is a very interesting file if I then want to scan this I can just use the same tool here GG Shield run this uh and then that will uh go through numerate through those files and let me know if there's any secrets so very very easy about the level of a script Kitty or below because that's what I am um uh and this will take a little bit of time but I'll go back to the slides to in the interest of time so the workflow for this if you're interested uh I download apks via a tool

called gplay which imitates the phone so you can download it I use JX to decompile it and then I scan for secrets with GG Shield so very easy to to to do this uh for anything an example of this this one's from a friend of mine Jason hadex he explained this on on my podcast uh uh and this one involves an application for a bank so Jason was doing a penetration test on this uh and he looked in this bank which was a tier 2 Bank in the United States so a very very big organization this bank had some features in their mobile application one of them was that you could take a picture of a check and then that check

could be cleared those pictures after he decompiled it and looked through the source code he realized those pictures were being stored on the the phone itself but not encrypted he then realized that they'll be sent to an Amazon simple storage bucket or S3 bucket and right next to the line of code that said send to this bucket it was use this key to send to that bucket so he decided to do that and access 10,000 plain images of checks that other people had deposited uh from their bank so this is an example of uh it's not just small companies that leak Secrets this is an example of a very big Bank uh and it's a big problem and one of the

reasons for this is because uh people often think that uh uh you your phone's obviously a computer if you have to do everything through your backend it can feel inefficient so if we just connect our app directly to the source it's much better but there's no way of doing that uh well there's no it's not a good idea it's not a good idea uh to do that because you have to hardcode the secrets so Cyber News actually did some research into how many mobile applications have secrets and they discovered that 55% nearly 56% contained hardcoded Secrets plain tech secrets in mobile applications now not all secrets are created equally right so you could have different things in here like slack

like slack keys or things that may not seem that sensitive um but they can still be used by attackers in different ways the most common secret they found was uh Google storage bucket URLs so around about uh 177,000 of these which is in line with the example of connecting to some kind of storage system directly from your from your app uh okay so just quickly we'll go right back uh so this is the mobile application I scanned and we have some secrets that have been exposed and what I want to do is just show you a couple of these secrets now this is a real application I downloaded from the Play Store I've masked all the secrets and I've hidden

the name so I'm not exposing anything but this is a real example and we'll have a look at what we found uh in here there's quite a bit so first there's some high ENT strings which uh could be interesting it's got API key here but I'm lazy so I'm going to go straight to the keys that I know are valid so we have some Google API keys that are valid here couple to choose from we have a valid slack web hook now uh slack web Hooks and things like that are often used for catching crash logs or sending information somewhere about the app uh so that you can look through it but there can be great places to try and

send fishing uh campaigns because they they appear to come from your internal system so they can be interesting for attackers uh with some more Google API keys and this one's really interesting some Google or Keys which can be very sensitive and some Facebook Keys which are also valid uh and a few bit more so that's a real example from the Play Store um of what you can look uh if you want to ask me about how to do this for iOS or for Apple app apps I can it's a little bit more involved but uh certainly possible so how do we prevent hardcoded secrets well number one is uh don't hardcode them even to test and we

could pretty much finish on this but uh apparently it's very hard not to do this and to be honest I've also done it so I can't really complain too much uh but this is a number one if if we stopped hard coding Secrets this is how Secrets end up basically in all of all of our end files um if we didn't do it from the source code they won't be in the in data effects uh but using Secrets managers using the the the best ones or heaviest ones might not be good for you for example hyot Vault very great product but quite heavy and if you're a team of four then three of you are probably

going to be trying to manage that on their own um so I wouldn't recommend that but there are kind of SAS versions of that that can happen then the clouds also have secret managers which uh can be can be great at at least taking to the next step and the step that I wouldn't really recommend to anyone is that you can encrypt them and store them and get I only mention these uh because look if this is what it takes to get you from hardcoding them then let's start there but I really wouldn't recommend it uh because it gives you a pretty sensitive uh file uh using automation so I work for a company called GE Guardian I'm wildly

biased without anything that we say but uh you should definitely use git Guardian um but there's also lots of Open Source tools so GG Shield is is one from G Guardian that I have but you also have truffled hog G leaks Yelp detect Secrets some other ones as well basically there's no excuse not to have some kind of secret detection in you it's very easy to set it up you can put them in your pipelines you can create gith hooks there's lots of ways to do Do It um so obviously there's commercial tools but there's lots of Open Source tools as well uh and then finally what what can you do Beyond just not hardcoding them uh rotate them

regularly uh limit the privileges on them and whiteless your services so rotating regularly is really important because it means that you actually know how to rotate them and then when you have a leak you will know what to do uh because most of the time when leaks happen people have no idea what the keys are for are you going to break production if you shut turn it off and how it got there but if you have a process to rotate it then you can do that limit the Privileges obviously don't create admin keys for everything if they only need to read data then make it read data and then whiteless services so that only trusted Services talk to

each other uh so that's it thanks very much for for for listening I'll obviously be hanging around here's some more reading material if you want uh we have a a model on how to maturely manage Secrets or if you're interested in some of the research I presented the one on the the uh right uh is going to be the state of Secrets rule uh so thanks everyone for listening I hope you enjoyed

it thank you so much we will have room for questions after the event for now we will have a very short 10-minute bio break and then we will conclude the day with the last two talks so see you in 10

minutes