← All talks

Oops!!... Did I Reveal Something? Detecting Leaked Secrets in Azure Deployment Templates

BSides Cape Town · 202315:02150 viewsPublished 2023-12Watch on YouTube ↗
Speakers
Tags
About this talk
Azure deployment templates stored in resource groups are accessible to any user with reader role and often contain plaintext credentials if not protected by secure strings. This talk explores how attackers extract secrets from deployment history and outputs, presents the Deployment Grazor tool for detecting misconfigurations at scale, and discusses defensive remediation strategies for Azure infrastructure.
Show original YouTube description
Protect your Azure infrastructure from insecure secrets in deployment templates with deployment grazor - an Azure PowerShell script that detects potential misconfigurations and leaked secrets. Infrastructure as Code (IaC) has been a valuable tool in the arsenal of DevOps teams globally. IaC tools such as Bicep and Terraform promote speed and consistency of deployments. These tools and associated deployment templates are typically leveraged in environments requiring large scale infrastructure. In Microsoft Azure, deployment templates are static files written in JSON, and used to define the configuration of a deployed resource. Whether a resource is deployed in Azure using the portal, Azure API, or IaC, these deployment templates will be stored in an Azure tenant by default. When a development pipeline is executed, a new instance of a deployment template is created. For teams adopting agile methodologies, this may result in numerous deployment templates created for a single cloud instance. Consequently, secrets insecurely created by IaC without using secure strings will result in multiple instances of secrets stored in plain text, in such deployment templates. Deployment templates stored in Azure tenants make for juicy targets for attackers, as they can be a great source of exploitable information. Additionally, the minimum permission required to view deployment templates in Azure is the reader role. Consequently, details such as passwords, access keys, and other secrets are potentially accessible if not secured correctly. From a detection perspective identifying such potential misconfigurations of deployment templates at scale can be a daunting task. This will require discovering the templates containing leaked secrets and the associated cloud resource. Furthermore, there are limited tooling available to automate the process on detecting leaked secrets post deployment of cloud resources; present tooling mainly focuses on credential scanning within a pipeline. The deployment grazor aims to address this. Deployment grazor identifies potential misconfigurations and discovers leaked secrets within deployment templates. From an offensive perspective, deployment grazor assists cloud penetration testers to discover sensitive data that could be used for lateral movement. This session presents deployment grazor, an Azure PowerShell script that should be a companion to defensive and offensive security team operations. Filmed at BSides Cape Town 2023 AV Sponsored by BITM Cyber Security
Show transcript [en]

uh good afternoon everyone uh my name is Javin and today I'm going to be speaking about oops did I reveal something um a little bit about this this is an Azure talk um Cloud um I've been started pesting about five six years ago um started with web application network security and then I ended up within the cloud space and I enjoyed doing things for Azure so this is a lightning talk so yeah hopefully we get through everything so to set the scene for oops D reveal anything um specifically for Azure they have got something always called an Azure threat metric this is kind of um like a MIT attack framework um which describes different ttps uh for that and

the aspect that we're going to talk about today is within the credential access column and this is for the resource secret reveal aspect so it is specifically for the technique a t605 and for for number three there's one as aspect for Resource Group um deployment history secret dump so when I was checking this out I was like o this is something interesting um how can we dump Secrets through this and the reason being why for me it was interesting if you look at the specific action um all you need is read permissions so what are deployment templates so these are static files um stored within Azure so when you're deploying an Azure resource let's say if

you're clicking in the portal or you're using asure CLI or aure Powershell or any infrastructures code in tool then there will be a static file that will be created by default don't confuse this with terraform th State files that's something totally different U but even if you're using terraform you'll still have a deployment template stored within Azure this literally just defines what you are deploying so for example if I'm deploying a VM then it'll say okay this was a configuration of what you had for your VM your skew your network um addresses Etc and this is written with in Json um and purely for this one here what we're going to be looking at for this

talk is from an infrastructure as code perspective so one key aspect about performing this attack is about secure strings I didn't want to leave no one left behind but if you don't know what secure strings are in very lame and ter this is just a technique to avoid storing sensitive information in plain text so you can just declare a secure string let's say if it's a password then if you want to um call that um variable out then it won't output that in plain text just very simplified in layment terms so how is this done in infrastructure as code so if you're doing as I mentioned in um the portal and you're just clicking through

Microsoft will create your credentials using secure strings for you they'll handle it but let's say you're using infrastructures Code as me your terraform or asure bicep then you would in terraform let's say when you're now creating in this specific example a database password then you would want to have the aspect sensitive true that will then convert the database credentials as a secure string and then in bicep is you have a parameter and you put it as password um you uh be a password string then you'll add the secure flag for that now let's look at the offensive technique so let's say you on appentice you're performing a red team engagement I won't get into let's say you have

compromised a specific account or anything like that is if you go to the portal um you've probably seen this before when I was starting out with Cloud this was something that I I skipped um CU I think okay there's not much here just literally defines what the deployments are but if you click on that there are specific aspects that are there you will have one aspect which say inputs and this is inputs where these are now your defined variables whether it be usernames or if let's say you've created a subnet it to be your subnet range um for this one here if you see at the specific at the bottom that is the admin password or key and here is a

administrator as an example the other place where um your credentials or information can come up from this is within the output section so let's say you're using infrastructure as code and you deploy resource um some scenarios you would want to then pull the specific variables out from your output let's say if it's an IP address sometimes developers they may be output now um credentials um or they say maybe output do um admin um password or credential so that will then be within your outputs section and then the template that's literally um the template for it so to come back here that's an example of how a template will look like cool so now let's look at

what are the other techniques so that was the azal portal perspective now there are different types of ways that you can also pull templates out as mentioned you can use azro Powershell to use a Powershell you need the a Powershell module um and then you can run this command get a resource Group deployment Resource Group and you specify the resource Group name and as your CLI is just a deployment group list and that will just give you a full list of what the deployments are and just to recap on the minimum permissions that you need is the reader roll so if you have the rback reader roll you can then read what the templates are so let's

look at a demo of how this looks like so we looked at initially at how in the portal this is now if you're using Azure Powers showell if I can find the play button where's the mouse here just give me one second Dem go not with me even with the recording let me just unplug this seems like this SMC froze okay so why is this okay there we go so I'm running this command I'm just showing you here how this will look like a lot if you if you run this come

on okay this is how it to look like if you run this specific command um asual partial as mentioned I'm not going to go through the whole um compromise um approach so this is a whole deployment configuration for for for an estate so you see for example you've got a network security group I'm just going to pause it here for example um here you can see you've got your parameters and you also have your outputs section as well um and this will literally Define everything you can see this if you go to a resource Group either way if you've got reader WR um you'll see that network configuration if you're clicking through or if you run

specific Azure power pow shell commands but the interesting aspect is for this specific instance I'll just for towards the end is there's a credential here from running um the specific command and that is the password there for the SQL server admin password cool that is that is great so we we're able to pull um crest for that so I was curious okay this is quite trivial like it's quite easy what are the defense measures that Microsoft provides so I went back to the threat Matrix for ttps and this is what Microsoft says in terms for detection details it says that if you go to the input page and you see that um the credentials and clear text and you're

not using a secure string then of course there's an issue here that's the detection that is there that didn't make sense to me as well in the logs there's nothing available for um for resource or any action so I also did my researches are there any specific actions that will trigger this um so I looked at specifically from log analytics and Sentinel and the only ones that had were right actions so if let's say you're creating a resource and you perform a right action that wres to it and that will be within the log configuration to pick it up but if you are performing read actions of pulling this from your deployment templates Microsoft see it as

a legitimate use case for your environment mainly because you have the reader permissions so inherently you should be able to read anything so that's why I came out with this tool for deployment Grazer how can we actually DET take detect this in a master state so for one example which I showed you now is you saw there was a lot of Dum from the deployment history you saw the Azure web web app the network security the database but in large organizations if you are constantly deploying within an estate how will you be able to pick up easily that within this specific deployment file or within the specific Resource Group or this subscription where are we actually leaking Secrets

also keep in mind this is a post deployment issue so let's say within your development devops pipelines whether it be GitHub or devops you might have S scanning tools or git leaks um try to scan credentials but assuming that you know developers are using let's say random functions to create Secrets these secrets are created after the pipeline has been executed so it won't be picked up within your secret scanning as well as if if you're outputting secrets that won't be picked up because this is a post deployment um technique so this is a demo of a tool which I created hopefully it doesn't crash again so it's quite simple um I wrote it in powersall so you just import the

deployment um Grazer module and what this will do is just um it pulls all the different functions that are built up so if you don't have the azal powers shell module installed that will install it for you um as as well as in your context um this one here had already logged in in the environment so you won't see the login prompt coming um from here I'm just going to pause it there because it goes quite quick is that you can see the beauty of this is that it tells you what subscriptions are available to the user so if you're on a pentest engagement and you can see okay what is available within my context as

well or if you're performing trying to detect where are leak Secrets within my estate you can pick specific subscriptions or specific resour groups um so here I just chose yes I want to use the specific Azure context that was just one subscription and from there to start scanning um it's pretty quick I made it threaded and it just says okay there's one subscription that is there and it is already scanned it and you'll see now when I just cat that it'll tell me okay that's where the secret is is that specific deployment and as well in that Resource Group where's pen testing sweet so remediation first of all you want to transform your sensitive data

through secure strings um this is quite basic but I've seen um it's something where we might skip because you might think okay my developers actually put in secrets and credentials but as previously mentioned this is a post post deployment technique um be careful as well as outputting because that can it's not just for your inputs but as well as for your outputs um if this does occur within your estat you want to rotate your secrets and as well as Purge as well um the next step for what I want to expound on this tool is to provide support for Azure CLI um as well as the Azure API the reason being for Azure CLI is let's say um you know Unix box and

you want to use Azure CLI that's more for bash so that's why for Azure API um that's literally if you just want to maybe use Python directly interact with um the Azure environment and the other useful thing is which uh also want to create is also devops extensions so maybe get actions so if maybe your developer is doing this then is to block these specific um actions from actually occurring with within the pipeline as well as devops tasks and these are useful links for anyone who's interested um if you want to check out the Azure threat metrics the different type of ttps and also that's the link to my tool any questions and thank you

what principle are you relying on to detect the secrets are you looking for like password and variable wa okay first thing is um is so I'm assuming for those deployment scripts you don't necessarily have to have sec right no so it's it's kind of like if they require for the set process add people that are creating so so not entirely so let's say if you are creating a database you will need a connection string um or you'll need basically a username and password Whatever It Is So within my script I've I've used common types of secret terms that Microsoft has so I've got uh connection strings um client secret that's for if using service principles

for let's say app registrations you're passing that in your app functions um tokens that is now if you're using a storage account and you've got SAS tokens that's a different discussion I would say that's ideal for you to be using SAS tokens um um and also password as well cuz that is mainly for VMS databases and things where if you're creating local accounts or things like that so that's kind of like the RX that I have for the common types of secrets that Microsoft uses and it will scan that within your deployment so there default for the different parts ofy yes and then I'm guessing there's no equivalent for output so output is more or less the same yeah so if you're

outputting your connection string um both both for terraform or bicep or also um it is outputs. let's say connection string or password because you want to Output that specific variable for what has been created there was another question somewhere

there no worries um it's East African Tech Guy fun fact yeah I'm from East Africa so that's why East Africa take care um any other question yes sir your power module is it on the PS Gallery um no it's not no it's not but you can contribute as well any other questions cool thanks everyone