Sign Here: Abusing AWS Presigned URLs for fun and profit

[Music] Hi everyone. Hope everyone's having a great bid so far this morning. So I'm here to talk about abusing pre-signed URLs in AWS for fun and profit. So just a little bit of an introduction before we kick off. So my name is Jacob. I'm a cloud security consultant at MWR and we're going to be breaking down the talk of today into a couple of key aspects. So, the first of these is just going to be covering a little bit of a basic overview of what pre-signed URLs are for the uninitiated. And then we're going to cover some security problems with pre-signed URL implementations and how we can avoid them. And then finally at the end of the talk we're going to

touch on a little bit of a sanity check on what we've discussed and also then dip a little bit into incident handling on S3 briefly. So S3 buckets are something that I like to call the well that never runs dry. And there's many reasons for this but one of the main ones is that S3 has become very widely used now across the internet for sharing and storing both data and web content. And this includes both sensitive data and then public web content which is intended to be accessible on websites. So there's a number of advantages to S3 in this regard, but I think the big one for businesses is the fact that it's fast and infinitely scalable. If I need

more storage for my business, I can just knock on Jeff Bezos's door and ask him for another hard drive in one of his data centers. And this has led to some interesting use cases with with S3 over time. And because of the fact that S3 is a blob storage, it tends to avoid a lot of the traditional file upload vulnerabilities that you might encounter in web applications. I mean, in terms of interesting use cases, I've even seen S3 being used for session handling, which is not something I'd recommend, but it was nonetheless very interesting to see. So, one of the sort of cruxes of S3, I think, is that it's easy to use, but it's kind of hard to master. And we can

see this time and time again because insecure bucket configurations have lay behind some of the biggest cloud security incidents in history. And this is actually because there's a number of reasons why this is the case. Bucket ACLs, for example, access control lists for controlling access to individual objects in S3 buckets are an absolute headache. And AWS has added a number of features over the years to sort of keep up with managing access in S3. The other issue is public S3 buckets because ultimately these are just too permissive and if you mess up the configuration on public S3 buckets, you can have a bit of a security disaster on your hands. So this brings us to some S3 flaming

disasters or a couple of little things on what not to do with S3. And for those of you who are looking for it, I see there's a QR code that somehow made its way into my slides over there. Not sure how that happened, but there it is. But essentially, sensitive data in public S3 buckets, absolute no. Misconfigured IM credentials with overly permissive S3 access that can access a variety of buckets across an environment is another no. and also allowing modification to S3 buckets serving web content because then you end up with stuff like cross-sight scripting or just the defacement of websites. So this is all sounding pretty bad for managing access to S3. And that

brings us to pre-signed URLs. So what is a pre-signed URL? Well, a pre-signed URL is a temporary narrowly scoped link to access a specific object within an S3 bucket. So AWS essentially handles the authentication with pre-signed URLs. You generate a pre-signed URL and then you can just use that pre-signed URL to access whatever object it is scoped for in the bucket. Now, I'm not going to go into all the crypto magic behind how this works, but AWS has got some pretty good documentation. if you want to pour over it on a weekend or something, probably wouldn't recommend that. But yeah, in any case over here, one of the key aspects with pre-signed URLs is that

they cannot be changed once they've been issued and they cannot be used once they have expired. So they are temporary and they have an expiration date. So if we look at a pre-signed URL example over here, essentially the highlighted portion in red would just be the bucket that you're accessing with the object in the bucket that you're accessing and then a little bit of information about the region over there. And then all of the stuff coming after that is the pre-signed URL magic that includes all of the parameters that are bound to that pre-signed URL. And finally, the most important part at the end there, the X amz signature, which of course is the signature that determines

what that pre-signed URL is and if it's been changed. So pre-signed URLs offer a number of advantages for ma managing access in S3 over the traditional methods that we talked about like bucket ACL or provisioning credentials because traditionally if you wanted to provision credentials for example you might do it through something like AWS Cognito where you give a user temporary access to a set of AWS credentials and then they have access to the resources in your environment. But pre-signed URLs win in this regard because it's very narrow controlled access which is temporary. And that sounds great from a security perspective cuz it's hitting a number of good nails over there. But on another hand, there's also no need to make

public S3 buckets or then share or distribute AWS credentials which is a very big win for security in this regard. And finally, you also don't have to worry as much about AWS permissions and managing the headache that is IM because you can just make a set of IM credentials that scope down to a bucket or two and then use those IM credentials in your application for distribute or generating and distributing pre-signed URLs. So on this note, what on earth could possibly go wrong with this? Well, incidentally, there's actually quite a lot that can go wrong. Now, before we continue, I do need to clarify. Pre-signed URLs themselves, at least according to AWS's implementation, are secure. Unless any of you in the room

has a zero day that you're hiding, they're pretty secure. But implementations of pre-signed URLs are not or not. it ends up in a situation where when you start integrating these things with web applications, there's a number of mistakes that can be made that can have pretty bad consequences. Now, I've gone and classified this into what I'm going to call the four horsemen of pre-signed URLs. Now, those are authorization, parameter handling, content poisoning, and finally caching and persistence. And what we're going to do now is we're just going to break down a little bit of these in more detail and depth, go into some practical examples, and then we're going to see how we can avoid these

issues to begin with and hopefully end up with a more sane security posture. So the first one I'm going to talk about here is authorization. So let's take the following picture. Imagine you have a user who wants to upload a file to an application that is making use of pre-signed URLs. How this would work typically is the user would upload the file. Then the application would conduct some authentication with IM to generate a pre-signed URL. Then the user would receive the pre-signed URL and use that pre-signed URL to upload or interact with an S3 bucket that is storing the data. A key point to note here though is that pre-signed URLs while they do provide authentication to

S3, they do not provide authorization. Authorization needs to be handled on both the AWS side and also on the application side. In terms of the application side, there still needs to be sane authorization checks to determine whether the user who is requesting a file or interacting with a particular file has the rights and permissions necessary to do so. On the AWS side, the set of IM access keys that is used to generate the pre-signed URL and that the application has access to needs to have the requisite permissions to access the bucket and should have its permissions scoped down narrowly in that regard. If we look at a potential attack over here, we can see a vulnerable

application where authorization is not necessarily implemented correctly. In this case, the attacker can just request another user's file and the application will go ahead and happily generate a pre-signed URL for that attacker to then go and access another user's file. One of the key points here is because the identity that is actually used to generate that pre-signed URL needs to be able to access all the users files in the bucket, it has to have broad permissions over that S3 bucket. That essentially leaves authorization in a bit of a nasty spot here where it has to be enforced on the application side. You cannot rely on AWS to enforce authorization over here because that's just not how things work in this regard.

So if we look at a vulnerable implementation for authorization over here, you can see this is a function that is used to generate pre-signed URLs. And this is something that we're going to be looking at a lot over the course of this talk. But essentially this function in this case is just going to take in a file ID from the client side and then it's not going to do any particular authorization checks to make sure that that is associated with a particular user before it goes and generates the pre-signed URL and issues it to the user. In this case, we could look at a fixed implementation over here where a sufficient authorization check is conducted there on line 15 where it

actually checks whether the file that is being requested is associated to that particular user before the pre-signed URL is generated and issued. Now the next thing that I'd like to talk about over here is path and parameter handling. Now there's a number of different parameters which can be used to generate pre-signed URLs. I'm going to talk about three of them here and those are the object key, the operation and the bucket name because I think those are probably the most security critical parameters that if we're going to break it down and boil it down to that regard. So you can have an issue here where parameters are passed in from the front end. So an attacker may be able to

influence the parameters that are used to generate a pre-signed URL and that can result in a number of security problems which we're going to be going through according to the different parameters which are actually sent in. So in terms of object key now the object key is of course the particular object in the S3 bucket that you want to access. Now if this is being passed in strictly from the front end without additional checks being conducted. Now, before I do go into this, I actually want to say this does overlap a little bit with authorization. Of course, some of you might have already picked that up because you still need to have the rights and permissions to access that.

You need to govern those rights and permissions on the web application side. But when we're talking about object keys over here, if we consider a user who's interacting with say a profile picture, they want to pull a profile picture from an S3 bucket. So they'd make a request, the application would generate the pre-signed URL over here and then your user would essentially have the permissions to access that profile picture. However, if the object key was being passed in from the front end and no additional validation or checks were actually being conducted here, an attacker might try something like this where they try navigate to an admin folder or prefix on the bucket by using something like directory traversal.

While S3 isn't a traditional file structure, depending on how the application is actually handling this, it can have disastrous consequences and your your attacker may actually be able to navigate through the S3 bucket and access files that they otherwise wouldn't have access to. If we consider a vulnerable implementation over here, you can see that in our generate pre-signed URL function, a file path parameter is getting passed in from the front end. then there's a little bit of path normalization that is happening and that parameter is getting directly used in the code that is used to generate the pre-signed URL. So in this case if an attacker simply just passes in a dot dot slash or a different file and no

authorization checks are being conducted we have a situation where they can essentially navigate throughout the bucket and access different types of files that they otherwise wouldn't be able to. So the next one that I'd like to talk about here is path and parameter handling. And this is specifically related to operations. So when you're generating a pre-signed URL, you can specify an operation that S3 uses or IM uses when it generates that pre-signed URL to then determine what operation you want to perform on that S3 bucket. So in this case, you could have a user just requesting an object. So the get object operation would be used. Now, there's also a put object, list buckets, which

lists the bucket, and a bunch of other ones, but I'm not going to go into those right now. So, in a normal situation, a user might request an object from a bucket. But if an attacker can actually inject their own operation into the uh parameter in this case, and this is getting used to generate that pre-signed URL, then they may be able to change the operation that is being used on the bucket. In this case, this bucket just uses a get operation by default, but if an attacker can change that, they may be able to list the bucket or even upload files to a bucket. If we're dealing with something like web content, then I'm

sure you can see why this is a problem and where this could be going. So, if we look at a vulnerable implementation in this regard, then we can clearly see over here that the operation is being passed into our function to generate the pre-signed URL. And then that operation is essentially being directly used in the code that is used to generate the pre-signed URL. So there's no other validation that's not being enforced directly in the code itself. It's just user controlled parameters making their way into code that is used to generate pre-signed URLs. The final one that I'd like to talk about here is the bucket name. So there's a couple of different scenarios where this may be applicable. And one of

them is if you have a web application that has a bucket for user content and another bucket for admin content. Now this may be a good idea because you're segregating the content into two different buckets and you're essentially splitting that data and creating a good data separation between the two of them. That also means though for generating pre-signed URLs that your IM access keys that are used to generate the URL need to have the requisite privileges to access both buckets, the user bucket and the admin bucket. In this case, if the admin or if the bucket name can be passed in from the front end, an attacker might be able to change that to reflect the admin bucket and in that

case potentially gain access to a bucket that they should not necessarily have access to. Of course, this would also depend on actually being able to enumerate that admin bucket, but the point of the matter still stands. You shouldn't be passing in these kind of parameters from the front end whatsoever because ultimately it's a guaranteed security problem. If we're talking about vulnerable implementations over here, we have a very similar scenario as the previous two where the bucket name is getting passed in from the front end and then that's getting directly used in the code that is used to generate the pre-signed URL with no other enforcement. So if we look at a fixed implementation here, there's a whole variety of

different fixes. So I'm not going to provide a code example here because I think it would be several slides long. But I think the key thing to consider here is that this needs to come after authorization. So there still needs to be those valid authorization checks that we previously talked about in the first uh example that we gave over there. Then the object key that is assigned needs to be assigned on the server side and it needs to be uniquely generated because you don't want to give the client side any control over that whatsoever because ultimately that's just going to result in serious issues. And then thirdly the bucket and operation absolutely should not be coming from the client side at

all. If you have a specific S3 bucket that's associated with the web application then you should know what S3 bucket that is. there's no need to allow the client to specify that parameter in the first place. The application can just have that S3 bucket hardcoded in. Similarly, with the operation, you should know what operations your application is sanctioned to perform, whether it's uploading files or downloading files. And I I don't know if you want to give someone access to list your buckets, but uh yeah, uploading and downloading files is a good one to start. And then that again should be hardcoded within the function used to generate pre-signed URLs. And there should be no ability for that to be

influenced from the client side whatsoever. Now the next one that I'd like to talk about is content poisoning. So content poisoning is essentially a case where we're looking at uploads of files. So take a scenario where a user wants to upload a file. So they'd get their pre-signed URL from the application which would then have the put bucket operation in the pre-signed URL and that would essentially be used by the user to interact with the bucket and upload whatever file they needed to. So the key thing to consider here is in the default generate pre-signed URL function, it's only strictly requires the bucket name and the object key, but there's actually a bunch of other different parameters

that AWS allows you to specify which can have various implications for security. It's important to consider the content type over here and I'm going to tell you why. So when you upload a file to a bucket with a pre-signed URL, AWS knows what type of file that is based on the content type that you specify when you upload. When another user comes and retrieves that file, that content type that you specified is advertised to them. That means that if you upload something like JavaScript to a bucket and you set the content type to JavaScript, if another user comes and accesses that, then their browser will be informed that JavaScript or that the bucket contains Java or the object

contains JavaScript. So if the content type and the actual payload are not signed, you end up in a bit of a situation here where you can actually just arbitrarily modify those to whatever the heck you want. Now, this is a serious problem in terms of file upload because if you're conducting valid file upload checks for security on the application side, it doesn't matter if you don't sign the contents of the actual payload. Those file upload checks can just be bypassed then because once a person has access to the pre-signed URL, they can just change the body to whatever they want because that's not signed by default. So, this can lead to a variety of issues. As I mentioned, when you start

injecting JavaScript and arbitrary HTML content into buckets, you can get some rather interesting problems. Like for example, I could host my own website on your S3 bucket. Alternatively, if that site is or if that S3 bucket is being used on other like content on the site, this opens you up to downstream cross- sight scripting attacks or other types of issues like that. So in other words, you definitely don't want to give someone the permission to be able to arbitrarily upload whatever they want without any checks whatsoever. You could even do something like a DOSS attack by just flooding the person's bucket with absolute garbage if you wanted to. Now this is kind of difficult to

prevent, but if we look at a vulnerable implementation over here, we're just going to hone in specifically on the code used to generate the pre-signed URL. So in this case we have the bucket name and the object key as I mentioned which are really the only default parameters which are actually necessary and then we don't specify any other parameters over here that could be used. So if we look at a fixed implementation here we'd look at something like the generate pre-signed post function which is different to the generate pre-signed URL function. In this case it allows you to specify a variety of different additional parameters for signing. In this case, I've specified the content type. So that has to be enforced. So

only image J or image PNG in this case can be uploaded to the bucket and AWS will then check that. But more importantly, the content MD5 parameter needs to be considered. So that would be the signed body of whatever you're uploading. In this case, if you try and modify this once the pre-signed URL has been issued, you're going to end up with a rejection because S3 or AWS is going to see that and they're going to say no. In this case, that's going to fix a lot of these problems because I won't necessarily be able to arbitrarily upload content fully to the bucket. Now, the next issue that I want to talk about is caching and persistence. Now,

this is a bit of an interesting one. So it comes down to misconfigured proxy caches at the end of the day. So in AWS you can do a bunch of different configurations with proxying. Um you can use cloudfronts to act as a cache for buckets and other things. So in this case if we have a misconfigured proxy where the time to live is set too high on caches. you have a really niche scenario where an attacker who has access to a pre-signed URL may be able to download objects after that pre-signed URL expires. Now, the impact here is not really that significant, but it's a nice niche example. I think things get a lot more interesting over

here where you start talking about different cache keys because if the cache key doesn't contain all of the parameters that are also supplied with the pre-signed URL, then all of a sudden you have a problem. So, if our user over here has a pre-signed URL that they use to get file.pdf, for example, and obviously that URL has all of the other AWS stuff stuck on after it because it's a genuine pre-signed URL. And then our cache is misconfigured. So, it only has a look at the file.pdf part. In that case, an attacker may be able to just request file.pdf. And we have a serious security problem where an attacker would be able to get access to other files.

I'm going to give credits here to Tomas Salai on his blogs for this insight because it was really interesting. Recommend you guys go check him out after this talk if you find this particularly interesting. So in terms of caching and persistence, how do we go ahead and actually fix this? Well, the first of these is to ensure that the time to live in caches is set to sane values. If you're using pre-signed URLs, you need to consider that the time to live plus the original pre-signed URLs time to live may be what would be the total time to live in this case. So if you're using an application that deals with pre-signed URLs, you may

want to configure that to be lower even if it comes at the cost of proper caching. More importantly over here, you need to make sure that all parameters are included for the cache key. We can't take shortcuts over here because that can literally invalidate all of the security that the pre-signed URL provides in the first place. Next, we're going to get into business impact over here. So, we're going to talk a little bit about the fun things that this could have on businesses. So, in terms of S3, as I mentioned earlier, it's been used so widely across business nowadays that you have a lot of situations where businesses are storing sensitive data in S3 buckets. And we've

seen this time and time again now with a lot of the cloud incidents that have come through in the past few years where you've had S3 buckets holding really sensitive data that have been compromised and the attackers been able to gain access to a lot of stuff that they shouldn't have. In this case, all of the vulnerabilities that I previously discussed because we're dealing with S3, they involve the disclosure or potential modification of PII and data. In this case, there's wide reaching implications for businesses. So in terms of pre-signed URLs, they're not necessarily the silver bullet that everyone thinks they are. The implementation still needs to be sound and secure in order for these to be used properly. Then we have

downstream attacks. A lot of data in S3 isn't the ending point for that data. It ends up in an S3 bucket and then it gets used in additional business processes for processing data or conducting operations using business logic. In this case, if an attacker can poison that business logic, then we have serious problems. This is also becoming more and more relevant in the age of generative AI where we have um data stores being used to train or inform models on particular stuff. And finally, we have reputational damage and all that nastiness. I'm not going to go too much into that, but all of the previous things that we have discussed over here in business impact have the

potential to incur reputational damage for businesses. So, how do we regain sanity over here? Because like this is sounding fairly bad. I mean, we initially discussed S3 access control and how bad that was. And now we've discussing pre-signed URLs and we finding similar problems with implementations over here. Well, the truth is all of the problems that I've discussed over here are not novel. They're not new. We've actually seen these problems before and we know how to fix them. They're just problems that have been long associated with web applications. So if we can bring ourselves back to a grounded fundamental logic in security, especially when it comes to implementing stuff like pre-signed URLs, we should be able to

avoid the majority of these security problems. So if we're talking about a layered sanity check over here, okay, I think the first thing to talk about is probably authorization. So we can have a sanity check where we can consider is authorization or the principle of lease privilege being enforced on both the application side and also on the IM identity that is used to generate the pre-signed URLs. The next point over here to consider is never ever ever trust user input. Users are always going to be able to send in malicious or nonsense things and your application should never be trusting that when it's generating pre-signed URLs. The third one to consider is the one that's kind of difficult because it

involves a lot of deep diving documentation and reading what is or isn't possible. And that is are we using all of the security headers that AWS actually provides us with with pre-signed URLs to enforce it? Are we making use of the the tools that we have available to further enhance and harden the security in this regard? And the final one is is the network infrastructure that's actually supporting this entire setup sane. Do we have issues where we have time like super high time to live on caches and are those issues getting to the point where they're invalidating the security that the pre-signed URL was providing? These are all worthy questions to consider in this regard and I think

going through this kind of sanity check when you're implementing applications with pre-signed URLs is a good way to avoid a lot of these problems. So the last thing I want to talk about here is cleaning up the mess with S3 incidents. Now, incident handling in the cloud is something that could probably be made into its own talk in entirety because there's so much to cover here, but there's a couple of key actions that we can take if our S3 bucket gets compromised. The first of these would probably be to identify what IM access keys were actually used to compromise the bucket and then rotate those access keys immediately. This can also come with secondary actions like rotating

bucket encryption keys. In terms of actual IR though, we probably want to make sure no further access is taking place on this bucket or the data is being modified because we need to worry about forensics over here. So the first thing to do here would probably be to back up the bucket policies and then immediately enforce a deny all on that bucket to ensure that absolutely no one else besides your IR team is getting access to the bucket. Then we can go ahead and do stuff like enabling bucket versioning for all the objects in the bucket and MFA delete which is another rather niche setting in S3 that essentially prevents objects being deleted without explicit

multiffactor authentication confirmations. This would allow us to create snapshots of bucket data and prevent further data loss. And finally, we could do something like enabling bucket replication and pulling the data in that bucket to a separate security account for analysis. AWS recommends in architecture that you do have a separate security account presiding over your organization. So in that regard, this can be extremely helpful. So that's essentially what we've covered here with pre-signed URLs and the security surrounding them today. I hope all of you have learned something about pre-signed URLs and the implementations and we come out of this a little bit more secure. Now before I end off and go on to a couple of questions, I just want

to do a little bit of shameless self-promotion over here. Uh I do have a YouTube channel. You guys can go check me out if you enjoyed this talk um with the QR code over there. But going forward, are there any questions? >> Got one over there.

a pre-signed post for a user to upload a file that's probably not a file I have an MD5 hash of. So yeah, can you explain that bit again? >> Yeah, of course. So the thing is with uploads with pre-signed URLs, you want to consider that the upload file upload security both on the application side and also on the pre-signed URL side, right? So when you're uploading a file, you'd upload it to your application initially. The file wouldn't be stored by the application, but it would still be processed and you do some processing to determine, you know, your usual security checks on that file for application security and file uploads. And then when you're at that stage, you

can also then compute the hash of the file, right? Then you can issue that to your function that computes your pre-signed URL. And once you've got that, then you can issue the user a pre-signed URL with the the hash encoded into it. And then you avoid the issues that I was essentially talking about previously. >> Okay. Thank you. >> Anyone else? Cool. Thank you everybody. Really appreciate it.