Payload Delivery Networks - Abusing CDNs to bypass WAF and DDoS protections

Taylor's uh got a great presentation for us today uh it's payload delivery networks abusing cdns to bypass W and dos protect protectants so big round of applause for Taylor thank you first of all thank you for coming to my talk payload delivery networks abusing content delivery networks to bypass web application firewalls and dos protections so in this talk we're going to cover how to bypass W and dos protections from CDM providers such as Cloud flare AWS cloudfront um and Azure front door by finding and attacking the origin web server directly uh we're going to look at the different ways origin servers are configured um to prevent this sort of direct access and some of the surprising ways that they

can be bypassed as well uh including by using the CDN to bypass the CDN uh we'll also discover uh or discuss some more secure configurations and some various fixes that Defenders can Implement to prevent becoming victim to any of these attacks uh the talk will mainly be from a red team perspective as that is my day job um but we want to make sure Defenders get something out of this too so there'll be a lot of advice about how to fix these issues um and why they occur but before we do that uh we might start off with a little bit of terminology because there is a bit of domain specific um terminology and acronyms and things so uh the origin

server is the first one uh this is basically the target web server so this is the Webb server that we're trying to attack or that the tager has interest for some reason um and is protected by the CDN the CDN of course is the content delivery Network so whether that's Cloud Azure you might have aam or and perver or whichever one you use um which is basically a bunch of casing servers or it typically was that sit between uh the web application and the person uh the uh website viewer um and cases content between it the next one is the wa the web application file wall um so this is an application aware firewall that inspects

your HTP requests uh and looks for potentially malicious traffic and blocks it um at that application layer and the last one is Dos so distributed denial of service which many of you have probably heard before is about affecting the availability of An Origin server by just overloading it um yeah so a little bit on the outline of what we're actually going to cover today and how it's going to be structured uh we'll start off with uh the situation how these um cdns are set up and kind of what it looks like to make request between them uh and be a refresher for most but if you're completely new to this hopefully it bring you up to speed

so you get something out of the rest of the to uh then we'll look at finding the origin server IP because if we're going to attack it directly we need to know the IP we'll then look at the origin server uh protections you should put in place to prevent this direct access and some of the bypasses for the different setups depending on what you choose to go with um and then lastly we'll do a bit of a summary and mitigations um for more secure configur ation and how you can avoid these attacks um just as a note we'll focus mainly on WS um being from the red team background that's more sort of um my

focus um but it does affect any of the security protections that the CDM provides which is Dos protection it's bot management to stop scraping and um a whole bunch of other things that you can figure right and as a second note um the concepts will apply to kind of all content delivery networks not just uh Cloud flare Azure in AWS which will be the focus of this talk but if you're running acomi or fastly or Pera you know the list goes on um the concepts will be largely the same so you should still get a takeway cool so let's get started with the situation so we have an attack and we have an origin server um and they sit

on the internet so they have kind of just their regular public IPS right and without CDM protection uh this this will be the first case so we make a web request in this our case we have a website called legit target.com uh and this resolves the DNS to the origin server IP and then we access it directly right so the 1 192021 that's just the public IP of the web application whereafter uh and we get a response right there's nothing unusual here it's pretty basic um in our case we're going to use this a little bit but the web application um that we're testing against is just going to show back the IP that the origin server

observed uh observed requesting it that's just going to be for debugging and working out what's actually happening um and we get the headers that go with it as well when you make the requests so if you have a malicious payload so an example here we just have a simple cross-site scripting maybe a particularly trivial example um but it could be anything could be the the latest cve or some sort of template injection or or whatever it is um but in this case it's just some JavaScript that we have here um you can see the script tags and the alert function uh and we send that to the origin server it's a common vulnerability pattern um but the

origin Server doesn't know anything about it so it'll just send back the request and we can see the little popup that happens there right this is without the CDM protection the W protection the difference with the CDM protection is you put the CDN in the middle so this is CL a AWS whatever um and the origin is going to be protected by this right the W and the DS protections are all going to live here so we make the request um to our website and instead of the DNS resolving straight to the origin server it's going to resolve to a bunch of shared IPS uh that are on the CDN provider right so um yeah it's not going directly it's

going to the CDM the CDN then has to work out how to route this to the origin server so it looks at the host headers um so it sees that the host is legit Target um someone set that up on one of the providers and it knows to proxy that um to the origin server so this particular part is not visible to the attacker right it proxies the traffic in this case we get a response and we can see that the web application shows us that the in this case Cloud flare IP reached out to it and we send that back to the attacko right uh and if we put our vulnerability through here uh the W

should trigger and it'll inspect the request uh in this case it says oh the request looks like uh crossy scripting because maybe it has a JavaScript in the tags or something like that uh and it can choose to block that request and give you this page instead of the actual one so it doesn't reach out to the origin server and you kind of prevent that so that's basically how the W Works in that case but as an attacker we still want to get our vulnerability across right so we have to think about how we can do that most research around it and when you look this up and dig into it will be about trying to trick the W somehow um

you might be able to increase the pay Lo size and if it's sufficiently large the W might not inspect it for performance reasons um that's a pretty common one or maybe you need to change your payload a little bit so it looks a little less like JavaScript and doesn't hit one of the rules or you know anything like that um you can try and do but that's not going to be the focus of our talk today instead we're going to consider when we go completely around bypass it by accessing the origin server directly so before we do that we need to work out what the origin server IP is is uh and there's luckily for the attacker

anyway a large number of techniques to find this so we'll look first at a bunch of passive Recon and basically how much you can Google the first uh technique is with passive DNS so this is current and historical DNS records that helped us find the origin server um there's many providers for this like virus total and domain tools um the list really goes on um but we'll work through an example with virus total and see um current historical records that help us find the origin server so an example here is a readout of uh virus total and it'll show all the DNS that it's observed currently and in the past so currently at the top uh the

most recent resolve will show the CDN IPS because that's what the domain is going to resolve to um but you may also be able to find the origin server before you implemented the CDN that's the prec CDN origin IP and if you haven't rotated your IP since you've onboarded the CDN which you definitely should and in a lot of the documentation it does recommend you to do that um but surprisingly uh it's kind of common not to do that um but if we look at that IP the 1 19253 1681 72 in this case um we see this is still a node server on the internet and this is indeed our origin server second part of the passive

records is the subdomains so sometimes the CD end won't proxy all the traffic that goes to An Origin server um for example if it's over a protocol that's not supported um we're mainly looking at HTTP and standard web applications if you're running something like SSH or FTP or you have those other services they may or may not be able to go through the CDN um and you may find that there's subdomains that go straight to the origin server to facilitate this access and you may be able to work that out by taking a look next is the SPF records um so if you've got your email um validation set up uh and it's coming from the web server

um this is basically the IP that you're allowed to send emails from um and you can look at that and you may be able to find the origin server particularly useful if you're maybe looking at exchange server or something like that speaking of email um any emails originating from the application server or the origin server may contain the IP and the email headers so it's worth taking a look if um if you can so this could be sign up emails or password resets or basically anytime the application needs to send you message as part of your application you may find like receive from um headers and it may have the IP in there or you may get an X

orating IP header uh which is commonly used for anti spam and and may be in there um it could be a different um IP but it may be the origin as well so it's worth checking the next is certificates um so we can search for certificates issue to The Domain and look at historical records as well again like the passive DNS there's many providers for this census Showdown virus total again like so many and we're just looking for certificates where the subject is the domain that we're interested in so there's an example here we can do it in census um you just look for the certificate names in your query and then we can see here that it shows us how

lude um origin IP again we can do the same thing in Showdown SSL um legit Target and we can see that showing up as well right so it's the same for all of them just just go through your favorite provider or whatever is available to you the next one's um similar lines but a little bit different uh is about favicons so this is common for trying to find fishing um domains that are emulating you uh but in the same way you can find previous versions or the origin server as well so we look up the fabcon that you have that's hopefully unique uh and we find any IPS that currently have or have had um that favon and then TR

out the origin server from there so for anyone who doesn't know the faon is the little icon that goes next to your browser window um and yeah we'll need to search on a hash of that and the hash is going to be a little bit different depending on which data source you have available to you also wanted to note on this one um when I tried it out in practice um you may have different versions of the favon as well um so you need to look at the icon links in the page source to find the different versions of favon so in this example we have favon 32x32 2.png uh because it's a slightly higher

resolution and and we have this one but you may have favon Ico or uh which is more common but kind of lower resolution so just know that you may need to check a few of them if you're trying this technique um there's a few tools that help generate the hashes as I said they vary across providers um this is favon tash. cam. UK um pretty handy you just put the fabon link in there and it generates different hashes and gives you links to Showdown virus total and census and you can just click through so in Shan we can see the search and we get the same result uh as the certificate did you can also see the

hash there is a murma hash 3 um without going into too much detail it's a obscure hash that Shan use um for performance reasons but it also varies across implementations um a bit so it's kind of weird if you're trying to generate this from scratch you probably need to use the python Library as some other languages um produce different results strangely enough but in virus total it uses the md5 um although you'll need a higher tier subscription so really whatever is available to you um you should be using um and you can do the same techniques for any other unique identifiers um similar to how you find fishing sites if you're familiar with that so repeat with the process with any

that identifies the web page so it might be Google analytics token or a unique HTML title or basically whatever you can think of so that's the passive side um and now we'll talk a little bit about the active Recon side so if you're actually probing the web app things that you may look for that may help you to determine the origin server the first one is scanning the ASN or your IP block so you just want to if you have the IP block say you're doing a pen test and you know um your range or you're a large enough organization that has their own ASN um you can just do a search and see if you can find the

origin server or do a scan uh you may need to include the host uh in case you get a load balance or a virtual host or something like that that's just a classic Co command to do it um but really depending on however you want to scan it's up to you um the technique also assumes you won't run into any origin protections um which will be the focus of the next part of this talk the next thing to look at might be headers and cookies um surprisingly um inspecting the headers and cookies and other information in the web app can reveal IP leaks um so for example a load balancer May issue you a cookie directly um to

make sure that you go to the same server for your session um but sometimes the names may include kind of public IPS or or more disclosure than you might expect um so example of this is uh F5 big IPS give you a server IP sometimes in the pool name um if the admin set it up that way you may get u a public IP or maybe encoded values you may be able to take a look at and see if you can find something depending on the product the next is server side request forgery um for those who aren't familiar this is where you try to get the web app to make a request um on your behalf and

if you can do this and send it back to a domain that you control you can see where that request is coming from and possibly ascertain the IP that way an example of this might be if you have an upload feature and say you can give it an image URL um and you want to upload it to that web application the web application may take the image URL and then reach out to grab it so that it can upload it right but you're kind of coercing it to make that request so instead of doing a legitimate one or the uh image that you're after you can just give it an attacker controlled listener so we'll use this out of bound testing

Tool uh out of band testing tool a couple times oh it's not fun but you give it that um unique uh domain and then you can just observe the requests that come back so this is an example of it and we can see um the from IP address popping up in that which might be our origin server there may also be some more legitimate functionality um that might call back um possibly a call back for an API um so we have an example let's say you have an API that you need to submit a job might be training a large language model or submitting a scan or or doing something that might take a couple hours

or something that's um past the timeout of a standard HTP request so you submit the job and then the API will call back to you later when it's done um so you get the notification and you can continue on doing what you need to do um but if you have control over that callback URL um then you can put it to something that you control uh and observe where that request comes from another example that's a little bit more specialized is the WordPress pingback and I include this in case anyone comes across a WordPress site because it is in by default and it's um a little bit strange but also an example of how like common functionality that's

included uh can let you direct requests so a pingback is a special type of comment for WordPress that links back to any mentions of a Blog so the idea if you make a Blog and it gets popular and other people start mentioning you you'll keep a record of that on your blog the way it works is you have two blogs blog a Blog B blog a makes a blog post wres a blog blog B mentions blog a in a new post and as part of that it'll reach back out to blog a and say hey my post mentioned your post blog a says okay that's cool and then it reaches back out to blog B to verify that it was

mentioned right so part of that blog B actually coerced blog a into reaching out to it and if we observe that request we may be able to get the IP out so we have an example um just a out of thee box straight default setting um Target WordPress site um with surely interesting blog content um and we can craft a pingback message um which is basically just an XML U message we put uh my blog post in here which is actually an attack control listener uh and we can say that it mentioned your blog post which which is whatever the target is we then send uh the XML ping back so it's basically an XML IPC page

that's included in WordPress by default uh ideally you'd probably Harden it so this isn't available but um in this case it's available by default um and then we just observe the request that comes back and get the IP that way so if you are running WordPress um it is on by default and this is a setting here allow link notifications from blogs uh and you probably want to turn that off and do some hardening but if running WordPress you probably want to do a fair bit of a hardening anyway and this will probably be a part of it once you have the origin IP uh just browse to it and see if it works right

just make the request and maybe you'll get lucky and get back however if this is a residential IP or some sort of VPS like a Tacker VPS um it's probably not expected to talk to the origin server right if you have set up these CDN you're expecting that the CDN is the only thing that should be interacting with your origin server right so most organizations will have some level of origin protection set up um that prevent this direct access and that's what we're going to cover now and the first one we'll look at is possibly the most simple but prone to um these attacks is allow listing IPS so this is the layout of what it

looks like um as we mentioned the legitimate path is going through the CDN and then only the expected IPS go to the origin server and the IL legitimate pass is going straight to it and we should be blocking that so just using your normal firewall um there for cloud flare specifically they have documentation on this as do the other providers uh and they say they should explicitly block all traffic that doesn't come from cloud IPS um makes sense and they rate this is sort of moderately secure if we look into what the IPS actually are um there's a list at cl.com and they give you an official list to work from right but they also mention in

this little blue note down there um that Clare also uses other IP ranges for various products but they shouldn't be making connections to your origin server so say that you really do need to use the IP address range that they have provided and there's a good reason for this so uh this is the IP range here um it's just a bunch of ips they don't split it by services so it is kind of All or Nothing U plus the one they they didn't mention but they shouldn't be talking to your origin over anyway so this is the official list so as an attacker the question becomes how do we get a cloud flare IP right there are plenty of cloud flare

products out there um and which ones can we use to make requests on our behalf um there's warp there's workers there's a classic DNS and interestingly enough even iCloud private relay often uses um Cloud IPS so we have a bunch of products available to us so we'll start off with perhaps the most simple uh which is warp right it's effectively a free VPN client from cloudflare um you don't need an account you just download the client and then you get Cloud flare IPS right so you turn it on uh and you get a cloud flare IP so in this case 10428 196 whatever um however this would probably be a little bit too easy right this

doesn't work in every case so because it's not in the official Cloud flare IP range right so if we look for that IP and that official IP list it's actually not there right if you remember before we had that little blue box saying that there are other various products that cloud FL users this is included in that right so um that's not to say it doesn't work though um it might work if you're allowing all traffic from the cloud for ASN or you try to generate your own IP list or something like that and I have seen cases where this does works so it is definitely worth trying and it is fairly um simple to implement or test as

as a red teamer having said that the documentation says to use the official IP list so our question now becomes uh what if we use that list are there other products that we can use that gives us a cloud flare IP so the next one uh which was actually the first one that I looked at when I did this research uh was the cloud flare workers this is basically a way of running serverless code on the cloud flare Edge um basically you give it JavaScript and it'll run on a cloud flare node so you can make request to your worker it runs the cloud flare um workers JavaScript code that you give it uh and you can also make fetch requests

um using JavaScript right so if I can fetch to websites from a cloud flare node I'm expecting Cloud flare IP here so we can test that out they have the work as playground uh that you can play around with uh and apologies it might be a little bit too small but but um we just uh got the simplest thing where we're using a fetch to another attacker controlled listener um and just uh sending that off you can see here uh it reaches out and we can get our IP in the same way if we look at that IP uh it is actually in the official IP range right so we can cross reference that list and

this is looking pretty promising right the next thing I was thinking about when I was doing this was well can we just Pro like all of our traffic through workers some sort of like worker proxy um which is indeed what I ended up developing so this uh is the flow of how it worked so an attacker makes a request like with a browser or with a HTTP proxy aware application uh and use mum proxy sitting in the middle to redirect all the requests that are coming in to a cloud flare worker the cloud flare worker then uses fetch to reach out to the origin server from its Cloud flare IP so if we look into the proxy here we can

see it takes all the traffic and redirects it to that cloudfl worker that's the workers dodev and the subdomains like whatever worker you deploy um redirects it to the worker which has that official Cloud Player IP and then does the fetch to the origin server so the next thing to try is just to browse to the origin server IP directly um and unfortunately it's not so successful the first time so there's a restriction in workers where you can't directly access IPS so fetching 19253 whatever uh doesn't actually work in this case but we're not going to give up this easily we still got something we can work with we can request domains so the question becomes how do we turn an IP

back into a domain right are there existing domains that resolve to our origin server IP or can we create a new domain record that points to it that that we have right these are our options so we look for existing domains that resolve um to it uh I came across a service called nip doio you basically give the IP a subdomain um and then go niip and it the feature of it is it just resolves back to whatever you gave it right so this seems kind of good for our use case so we'll give that a go going into that website and it does work um the host header in this case is going to be the nip doio uh host header

so that is a consideration um but for all intents and purposes we're getting the cloud FL IP and making that connection also of note if you're OBC um Savvy uh the cloud FL worker will get mentioned in here um and included as headers so just something to consider but we can also make our own domains as well so the next Technique we look at is the Imposter domain so this is setting up your own domain in CL FL with the DNS pointing to the origin server we're just going to let the CDM proxy the traffic in the same way that the legitimate flow works right so instead of getting legit Target or whatever it is we're going to get

attacker controlled that's going to go to the CDN and that's going to route to the origin IP right because the CDN doesn't necessarily know who owns the IP and you can have multiple domains pointing at that IP um so it'll let you set this up um and this is how we do it we just go in the DNS records in our CDM provider of choice in this case Cloud flare um and it will just proxy the traffic through then we just browse to our domain and it goes through the legitimate um CL I piece we're in control of this zone so uh we can control all the protections on it while the legitimate zone is going to

have the waft turned on and everything else like that uh our attacker control Zone we can just turn the WAFF off um we can turn off the cing if we wanted to do the denial of service attacks we can do whatever we want with this and and be as permissive as we want thing to know as we sort of briefly mentioned before the host header is going to be a tack controlled doxyz here right or the worker uh in that example it's the nip IO um host header so um we do that but is that is that really an issue um it is if the origin server validates the host so if they've got a

reverse proxy or something like this and it's actually looking at the host header uh it might not work so example of this is uh in Caddy like this is just setting up caddy file um if you set it up this way with legit Target being the uh the domain that you're trying to serve and you have your reverse proxy in there um if you hit it with a different host header it's actually not going to get proxied back to your target application right so in some cases the host header may actually matter um and you should be um doing this host header validation as well if you want to save yourself from these attacks as an attacker though we want to

think well can we spoof these host headers um is there a way to do that in CL you can do a transformation rule so this applies to any incoming requests it'll add a header on um so in this case we try to add a header name host um and make it legit Target but when we do that we get this little red bar um which I blend up here and it says that set is not a valid value for the operation because it can't be used on the head of host right so we're actually being restricted um on being able to change the host head up so if we look at other Cloud FL products um CL is restricting

the host headed modification pretty heavily as I expect um for security reasons so they basically require you to have a subdomain of a Zone that's already associated with your account or something like this which um we don't because we don't have control of the legitimate domain right so uh can't see any way that we can actually spook the the host header in this case so the summary for cloud here is if you allow listing IPS uh if it's restricted just to the cloud ASN uh or that larger space outside of the official list you may be able to use warp uh to do the work around and the mitigation for this is to restrict to

the official IP list if you're using the official IP list though um creating an imposter domain with your own zone or using cloudflare workers it's a possibility to to get a bypass and you should be validating the host header to mitigate against this against the legitimate domain so such as using the reverse proxy um Azure front door um the Azure CDM uh is somewhat similar uh the documentation suggests IP filtering and header matching um as an extra feature U but we'll look at IP filtering first um and a little bit differently to Cloud flare is that the Azure IPS are categorized by service tags so there's a set of specific IPS um that are only for

front door for the back end of it that are talking to it um and won't be shared across other services in Azure so a couple examples of some of the services we might wanted to have tried so Azure functions kind of similar to workers um or we have Azure VMS as well um but you can see they have different service tags um compared to the front door back end so you really should be restricting on the IP list that only is associated with front door and has that service tag um in addition the documentation mentions to validate a header um this is the xtac a attack FD ID or front door ID header um which is specific to your

Zone uh or your profile I believe they call it here um and this is to avoid having imposter front door profiles so in the same way we could set up an imposter domain we could set up an imposter domain with front door um but you need to have this ID so that it ties back to your tency that's what they recommend so the summary there is uh if you're allow listing um and it's not limited to that service tag for front door you might be able to use the other services to get an Azure IP um but really you should be limiting to just that IP range and if you do uh an imposter front door profile may be an

option unless you're validating that unique header um which you should be doing front door works basically the same way as Azure does uh it says to use IP filtering and header matching and the IPS are also uh tagged by service tags so we have a cloudfront tag we have an API Gateway tag we have ec2 tags um Lambda and ec2 share ec2 tags probably because Lambda is kind of bills on ec2 um so yeah a few products but ideally um you should be restricting just to the IPS that are Cloud front so summary is if you're not limited Cloud front um you may be able to use the other services that we mentioned um another note is that you

can easily check the API Gateway with fire approx which is a um open source project that will just fil contr all your traffic through an API Gateway um fairly easy to set up um but if you are limiting to the cloudfront service tags uh as a documentation recommends um you could do an imposter Cloud front distribution um but you should try and add the secret headed to medicate this similar to the front door uh option although it's not included by default but it is easy enough to set up anyway um speaking of that will be um the next bit about secret headers uh this is where you add the secret header uh in your incoming

requests and you drop any requests that don't have it on the header right on your reverse proxy so when the CDN reaches out to your origin server you're expecting that to uh secret head to be added you can do this in cloud with the transform rules that we tried before with the host header but this case we just put like an X Tax secret header or whatever you want to call it uh and give it a secret um to put in cloudfront gives you the option to do it as well when you set up your distribution uh it asks you if you want to add custom headers so i' recommend doing that um in that case but you can

also do it after the fact as well and as we mentioned before Azure front front door has the fdid header uh included by default so you can just go ahead and use that but in any case whichever one you use um your request should be reaching the origin server with a secret header of some description with some random unique key or secret um that you can validate against you then want to validate this header on your origin server so probably using your reverse proxy um to validate the host header whether it's Apache engine X or caddy or whatever it is and you want something probably like this where if that secret head is not equal to your

secret um just return a forbidden or drop the traffic or basically do anything that's not pass it on to the web server up to you uh we should just be aware that it is uh potentially possible um that that static header might leak out um if you get something like a local file inclusion vulnerability and you can read the reverse proxy configuration you may see the secret um you can add it on yourself or maybe you have something like uh debug messages containing the request so if you have some sort of API testing or like console logs or something um you may actually be able to see this although it is perhaps a little bit less likely um but if you do find

that value You' be able to add it to your request yourself right so it's best to to sort of like uh group these um protections and Implement a few of them the next one is about certificates um this is particular uh particularly authentication between the CDN and the orig the first one we'll discuss uh is one that is known a little bit um and it's a cloud flare specific feature called authenticated origin pools uh this enables Mutual TLS so authenticated um network connections between in your origin server this means that the origin server can reject any requests that don't present a certificate and that aren't signed by a specified certificate Authority so if you try to connect

without a certificate um you get an error something like this so you get an SSL client orer it's not valid um you don't have sufficient um authentication um to make that request so the question becomes as Ino which certificate do we actually need well when you set up the certificate by default Cloud flare will offer to generate you a certificate from the cloud flare certificate Authority for your convenience but that certificate Authority is a public certificate Authority that applies to the whole of cloud flare this means that you can use any Cloud FL certificate to make this authentication by default so we can set up our imposter domain as before um using our attacker controlled domain and we can turn on

authenticated origin pools and then when Cloud flare reach outs out on our behalf it'll use a certificate that is signed by the cloud flare CA which means that we may be able to access the origin server and authenticate that origin pool they do mention in the documentation um that authenticated origin Pools by default only confirms that the traffic came from cloud flare not that it came from your specific zone so if we creating our own imposter Zone and using that um we can still say that it came from cloud flare but it didn't come from the cloud flare Zone that you expected that has your wife protections that has your dos protections and everything else if you want to tie it back to your

specific Zone um you can upload your own certificate instead right and then on your origin server you're just uh validating on the certificate Authority um that you own and that you're in control of and that hopefully no one else can actually a certificate off right um now for Aon AWS there's no native um supporter equivalent um for authenticated origin pools or that mtls um equivalent the similar solution um that's thrown around a little bit would likely require AWS Lambda or Azure functions um to do so you would have to send it to the CDN which would forward it on to basically the equivalent of a worker uh in clef terminology you need to make that request and do the

authentication there you need to funnel all of your traffic through lamb dra functions which you may or may not want to do uh and it's pretty custom and not a very common um strategy uh for these providers um instead uh luckily we have other options that are um good as well and that will be sufficient and this includes tunnels and vpns which we're getting to the um Parts where these are um pretty good protections to have so the first one is cloud FL tunnels um so this connects the origin server to the cloud network via a wire guard is sort of tunnel um rather than communicating via public internet IPS it basically means you can start

dropping all inbound IP traffic um and the IP uh will effectively not be routable uh on the public internet right so even if you know the origin IP it's not usable right because the legitimate flow is we get our legit Target goes to Cloud flare and then it talks under that tunnel rather than by making a request to an IP then on the origin server side there's an agent that sits there that just uh reaches out to the uh listening web server locally uh and you don't have to use the public uh internet for that part or IP you can do this easy enough um you just set up the tunnels to talk direct to your web server locally in um the

cloudflare um dashboard and it gives you instructions how to install the cloudflare D agent which is basically the wi guard tunnel on your origin server um you give it the commands it has an authentication token in there um and it just links back um and you'll see your tunnel activate in the cloud ey dashboard you'll also see that a cname record gets added um so that when you look up legit Target it's actually a c name record on the CDN side that goes to Tunnel id. CF argot tel.com argot tunnel is what they used to call class tunnels before they rebranded it um so see that this is the way that they link it back

to your tunnel to know where to actually send the traffic now that we're not using your public IP right um and that's a good thing um to know but yeah On the Origin server uh just to make sure you're good to go you should be dropping all the inbound traffic um on your firewall just your standard firewall just don't use that public IP now but thinking about back to that c name and the way that we set that up um is it possible to do some sort of like imposter C name DNS record so following in the same way that we did the Imposter domains can we do like an imposter tunnel and use a um tunnel that's

associated with someone else's Zone um in your attacker controll zone I'm not sure exactly how someone would get the tunnel ID um but if you somehow manage to do this we can test it out and see whether it's equivalent to getting an origin IP um can we just connect our Target um domain to the uh Tacker controlled zone so we can set up our Tacker domain um and instead of putting the public IP in here um we're going to put the C name to that um CF arot tunnel. comom uh address and see if we can resolve that so if we browse to our TCH control domain um we see that cler does refuse to um route this and then we get an

argot tunnel error so it appears that cler doesn't resolve the tunnels across zones so if you set up a tunnel um in your Zone it will be tied back to your zone so um feeling pretty good about this one this is a a good uh protection to have asra takes a little bit more of a traditional approach to this rather than using the tunnels with the agent that CL does um and suggest you set up a VPN Gateway the VPN Gateway then puts you onto a virtual net um within Azure and you could link that to your front door um profile using those um private links or in the virtual public Cloud uh private Cloud

rather you also don't need the IP because you're getting that VPN connection as well so similar concept um slightly different implementation AWS follows the same way that aure does they give you um VPN they give you a few different options um you want to establish that VPN connection and drop all the other public IP traffic the next one is dedicated IPS um this is if you go back to the idea of allow listing IPS um but you want to try and find an IP that's unique to your tency right so this is a CDN IP but not one that's used by workers or other um shared services on the city and one that's specific to your Zone um that

way you could safe firewall off all the IP traffic except for the one that comes from this very specific IP um this seems only to be available in Cloud flare um they have a product here that's available if you have an Enterprise plan um but Azure in AWS uh don't typically support uh applying a static IP to profiles or distributions and the last one here um with the direct links uh applies to all three of them is another way um to secure yourself so here we have the attacker getting the domain for CDN the CDN peers directly with a data center or some sort of um connectivity provider you can do it yourself if you have a

networking team or you're large enough to do it um and then you can have a private Network between the origin server and the data center that way you can get from the origin server to the CDN using those um private links rather than using the public internet so you don't have to have your public IP exposed again so no need for the public IP is a connection goes directly to the CDM provider rather than over the public internet and therefore our bypass doesn't work uh they call it a variety of names CL they call it the network interconnect Azure call express route is Direct Connect um usually Enterprise only or you have a data center that sort of

works the connection out for you um so perhaps a bit more of a difficult setup for a lot of um smaller places but definitely an option if you have access to and now we'll talk a bit about the last part which is the mitigations in the summary so the first thing you should take note of as a takeaway is that you can assume that your origin server IP can be found passive Recon makes it fairly easy um you should be rotating your IP after you unboard it because the historical records might give you away having said that you can still do some informed guessing um particularly if you have a similar IP block or an ASN we still may

have some idea where the origin server is likely to be right and you can't help it if you have some services like the ssh1 or FTP or whatever that that point to your origin server if you have any records that point to your origin server they may be able to be found but even if the passive Recon doesn't um find anything active Recon still may be possible so it might be leaked out in KS and cookies and maybe you can fix those um but there may be exploits such as the serice side request fores we can coers the server into making a request and observe the IP uh or there could be legitimate functionality that's dis inherent such

as call backs and pingbacks and things that just exist and you can't really help it um maybe some things you could try um dep proxy the traffic out of a different IP um but may or may not be possible depending on your setup second thing to keep in mind is that allow listing IPS uh restricts to the CDN IPS and not to your tency specifically so it allow listing the IP is only confirmed to traffic comes in CDN but you need to make it specific to your tency or your domain right follow the documentation um otherwise you'll potentially open yourself up to some risk um if it tells you to use an official IP list you should use that

list um don't try to create your own or use the ASN or do anything too fancy um just follow the official IP lists if you're using something like azra AWS and they give you the service tags um and say that IPS are only associated with cloudfront or front door whatever it is um use that limited set of ips um so that you don't have the other services being able to talk to your origin server in any case you should validate your host headers on your origin side um the origin server ultimately is the thing that you control as a Defender um so if you validate those host headers you can make sure if you're using those

other imposter domains techniques that give you the weird host headers of the attacker control domain if you're validating with the legitimate one that technique will no longer work and you may want to um complement this with other protections as well such as the secret headers that can be used as another option to tie it back to your tency just included in front door and recommended and fairly easy to set up for cloud front and Cloud flare uh where you validate On the Origin server such as through your reverse proxy um that that header has um gone through your zone or your profile distribution whatever you want to call it um and is unique to you it can possibly be leaked

so it's something to keep in mind as well if you're using authenticated origin pools just note that they're not tency specific by default um it only confirms that traffic comes from cloud flare by default and not that it came from your specific Zone if you want to do that you need to use a custom certificate um and tie it to your certificate Authority that way it's not easily available on other providers um although you may find outs side of the three that we've looked at maybe it is provided as well but ideally what you want to do is Ditch your public IP alt together if you can so tunnels and vpns can be used to

make the private Network links and then you can just stop using your public IP all together direct links are also an option if you have the infrastructure such as the data center link uh and you can onboard your origin server effectively to your private clouds and you just block all the impound traffic coming to your public IP or just take that interface off if you can and the last one was about the dedicated IPS is also an option if you're a cloud forare Enterprise um customer using CL product to get a tendency specific IP this may be useful if you have some Legacy um products so you're limited in the uh projections that you can Implement for whatever

reason um and that will give you a tendency specific IP however it doesn't appear to be available on other providers um and just keep that in mind that it might not always be available to you and that leads us to the end of the presentation so if we have time I guess I'll pass over and maybe answer a question or two thanks

great talk do we have any questions from the audience just raise your hands if our Runners can see you

cheers uh over here sorry did I hear correctly that you mentioned it was non-routable to the uh sort of private internal DNS from a different Service uh sorry what was that question you mentioned it there's a very minor point but I think the Details Matter having worked on the space bit um it sounded like when one you tested in Cloud flare if I set up another Cloud flare service that's not on the same account but I know the uh Cloud flare seemed to have a ability to set up a tunnel which had its own DNS for how that was resolved yeah you're saying you couldn't resolve that from a different Service uh in the different so I believe

this has to do with the tunnels um when the tunnels uh the question is about um tunnels and whether you can resolve that in a different zone so you set up your tunnel um going for your Zone and you get given a tunnel ID um and that's basically a cname record on the cdns um part if you would grab that CD that c name record and try and make the same DNS configuration on a different Zone uh like your attacker controlled Zone uh it doesn't allow you to do that CL they would just give you back an error and it won't route it um I'll see if I can find the slide um thanks that's super interesting because

there's technical reasons that surprises me behind the scenes um yeah second question if I may I don't see too many other hands but uh did you see Mutual TLS for the back end like come up cuz I I didn't see that as one of the options uh so the mutual TLS uh we saw in the form of authenticated origin PS on um flare uh that had a way of putting it in for the other providers there wasn't any native way um to implement it that I could find um you could implement it by using like Azure functions or a Lambda um using the custom solution to do it um but I couldn't see anything sort of

built in so depends on your provider but um the the cloud Mutual TLS is the authenticator that fits other providers provideed too that's fine I think uh I don't want to detract too really great talk as someone that knows the space really well um I would recommend if anyone here in the room at least hears this verify your accounts have MFA cuz take up was incredibly low amongst the customer base and it really shouldn't be and make sure everyone logs what they should from their CDN because it's incred incredibly powerful but thanks great talk uh I don't see the runners to give them mic back Co thank you he