GF - Prepare for the Appocalypse - Exposing Shadow and Zombie APIs

yeah thank you everyone am I audible Perfect all right so thank you so much for joining um my name is Amit we will talk about H Shadow apis and zombie apis today um I've been an um API exterminator for PayPal for the past year and a half um I've been developing tools Frameworks to that will that validates H Expose and help Drive remediation uh for this uh issue and uh yeah let's get on to it so um first I'll of course Define what uh Shadow and zombie apis are and and I'll show you an example a few examples actually and we'll talk about the preventative um sdlc on how organizations should Implement uh measures to to stop Shadow apis from being an

issue and uh finally some techniques for major uh Frameworks and the demo of course so in this talk let's set up some some ground uh rules whenever I will say the word API what I mean by that will be uh um a service from a micros service or a method and endpoint or um a query for a graphql or any resource that is available through uh API consumers so um like a form uh URL that that that that gets uh some information through a post request or um any API request basically so so everything I will call Api um so first of all Shadow apis uh basically there are the unidentified um sorry the official

undocumented endpoints so um that basically are being exposed through a configuration issues or um some human error or llm by mistakes and they are still externally available through uh through the Gateway um most of the issues are with Shadow apis are that they're not part of the security review process and often lead to areas of of the organization that are not being tested against H by by the security teams zombie apis in they're similar in their uh risk that they pose and um but they were originated from from actual working apis but they were simply forgotten so um and I've encountered a lot of those kind of of zombie apis where the developers they know about

some issues but yeah they saying yeah we're going to we're going to decommission it soon or we're going to Sunset the application or yeah yeah it's inactive but still exposed those are still a a issues that are posing risks to the company um basically um in in if you want to categorize the this class of issues there are improper Inventory management um and by this definition um they they they leave some uh areas of of of what we call attack surface uh so the the weakness itself can be categorized as um well in the definition it say old API versions or endpoints left running unpatched that's a risk right outdated documentation makes um uh some sensitive

information that might be available uh through the through the those old apis um sometimes they host real data so let's say that and we'll see some examples of that but but if we by mistake as a developer if I a put out a QA with real data and I forget to uh delete it after or and it's still being exposed then then then those are real risks some outdated systems will be a risk because they host some vulnerabilities um and there are many CW that H uh can be considered um um as part of this weakness but that depends on on the the the the business logic or how it's being implemented so okay uh one

second so uh anyone's familiar with the Resident Evil the the the movie and games right I see I see some heads right so um let's dive into this this fictional company of Umbrella Corporation and let's see some of the developers tasks and um the actions that they took to perform the task and what was the result uh you can you can imagine this Corporation as having a um multiple services that um Expo been Expo are exposed through several servers API gateways right going through firewalls and everything just this this pretty standard right you don't have to go into the details just understand that they might have many services that are being exposed and some of them should not be

exposed um so so this is Matt he's a senior developer right he was tasked with uh upgrading uh the API spec from version one to version two uh while while he was implementing those some new features in version two he left the old version one end points active so he was trying to make sure that the transition was smooth uh the old version one apis were never properly decommissioned leading to Shadow apis so the version one um was still accessible with many vulnerabilities in it uh that that like I mentioned before could also happen uh on subdomains and basically any resource that resource that it is still exposed okay so let's go to Alice she's

a full stack engineer H she uh integrates a third party debugging tool to troubleshoot some issue with her stack uh the tool registered some additional end points um with the framework API router so because it needed that to operate for example like a Swagger UI or um health checks right and um she forgot about it and um they were left those end points were left exposed in production and uh that that's that's a major example of Shadow API um James here he was uh uh working on U exposing some of the end points in his AP I and he was using U um um a dependency for for for T script right an npm package um to expose those those uh uh

endpoints the package itself became outdated he forgot about it did not update did not do an npm audit or whatever and um those endpoints were left accessible in production so the the even code that we don't see even code that we took from other developers might expose those kind of um Shadow apis right and um you know it's a general idea a good idea to check with sneake advisor when you're installing external dependencies um so uh this is rain she's a jial developer um she was tasked with configuring the API router uh to make sure that the API itself would handle uh some some end points she was she used a wild card pattern so I'm not sure it's

probably too small I'm sorry about that H but uh she defined the path with a wild card meaning that anything that's um inputed into this body parameter would pass through to the um service so um some end points that might occupy the same path uh route basically would also be exposed so this is a misconfiguration um yeah so I I'll Rush a bit because I'm I'm I'm bit a bit out of time here uh so um so so Spence here should have took the time and decommission so apis but he basically forgot because he was rushing to push to deployment and um here is an example uh by by Kaplan of of deploying a some sort

of Gateway that Bridges between Downstream services and um basically what what he did was he forgot to put some authentication to um to limit the access of some Downstream services that were inadvertently exposed um so yeah missing documentation is a is the major issue this this this this is what helps with um you know identifying those apis and um you you need to have someone competent enough to uh make sure that those apis are properly documented and I wanted to give an example by by an llm so the Red Queen is in the in the movie She's she's a a a artificial intelligence and um in in this example she was tasked with working on a feature

she she took some time um but the branches she was working on um drifted from um the uh the branch that other developers were working on and between those some apis were between those in a diversion of of of those branches so some apis were H unintended uh unintendedly exposed right um so this is like a list a shopping list of how adversaries uh identify and discover apis in general so boot Force running a word list and trying everything until it works uh less than optimal but even blue team use this to identify some endpoints um you can you can identify domain names and the old hosts if you have a history of DNS and uh that could also lead so to some

exposure um you can decompile some mobile applications that link to some apis uh that also give some information about it bugs error messages uh logs and uh traffic and uh some some even back Bounty Scopes might leak this okay so we we see that this is like a major a major issue uh and and how we deal about how we deal with it so there's a thing called uh open API Swagger specification basically H that is supposed to be a contract between the consumer of the API and what the API um exposes and uh allows um we use this uh specification to have tools and integration that validate uh what endpoints we have and um um we use this to make security

scalable right so having um a detail specification of each and every API that we expose reduce the attack surface that we are not aware about and there's a thing in Swagger in in in open API called Swagger extension uh which is just a extra parameter you can add to the specification and um the recommendation is to add the uh visibility into to it and validate against it in the API level and validate against it uh in the cicd right so if we know about an API that should be internal we would never add it to the Gateway as being exposed from external sources sources and um you can use um open API spec with many of the uh asms and

uh major gateways uh um like uh like Kong and um um there's a lot like Microsoft Google they own have their own everyone has their own tools Amazon and that that ingest open API spec and basically um you know stop some access to some apis based on the specifications um the most important thing that I've encountered while dealing with this issue is uh is listed here basically to know what we have to have the tools to have the scripts that are running automatically and and and and identifying those in the logs in in external traffic in all of the tools that we have H open Telemetry is a major thing that can be used to identify those

um and ownership of the of the of the service itself um who do we call to in the middle of the night to tell him that there's an issue right and um and actually finding this in a centralized place is really important um I wanted to speak a bit about thread modeling I don't think I have a lot of time so um basically thread modeling means that we visualize our Network we find the potential risks involved in the architecture and um and in this case what can be exposed and what not and what can be exposed through various services like if a service routes the traffic back to another service they should be considered within

the same level of exposure and um I really suggest if you have if you have any interest in threat modeling uh to go to this link threat modeling uh Manifesto there's a ton of information there there's also a contest in in uh in Vegas this week that are regarding to uh thread Ming and uh yeah so also OHP is a great tool for any cheat sheet or information regarding uh web application security so in this demo I will uh show um uh one technique among many to identify services for um expressjs which is a node uh package used to expose web applications um but but you can use source code analysis dependency injection code injection um

framework query simply asking the the the framework with CLI what are the endpoints are um using traffic that you've identified in any of your other tools and of course boot Force U always works um and um yeah for spring boot you have many options um like actuators if you're using Jersey then you can use the web application descriptor language that gives an XML and you can um use even Sam grip or other static analysis tools to obtain the annotations that defin them the apis and of course the actuator which is a debugging tool um and um there's there's a few examples on how to achieve this but let's move on so uh yeah in flask for example you you can simply use

uh the command flask routes and it will give you all of the routes that are exposed even the ones that you did not Define in the code like any dependencies um yeah I see people taking pictures like at the end of the H of the talk I have a cure with a a link to um to a repository with all of these tutorials like a workshop all right so you'll have everything and uh all right so let's let's move on to to a to the H demo real quick okay so I'll I'll probably skip a bit so um here we H run a nodejs application we use the npm Run start this application I've created

3 years ago it's it's it hosts it's a damn vulner application that hosts many vulnerabilities and um exposes some weaknesses such as Shadow API and we we run the application we save the debuging log you can see sorry you're not seeing the I'm really sorry how do I stop this all right okay let me start again so we are running this application um we use the npm Run start to um R application and save the logs the logs saved into a a file inside the file it defines every instance of an API registry in the register in the router sorry that registers um a method and a path and uh we use this log with in combination with

the Swagger file um to and this is just example of the Swagger right and uh some of it's apis and U we use use this file and a script that I've also provided to parse the logs into a a file that basically contains all of the endpoints so the list you see here is all of the endpoints the application actually exposes even the ones that the developer did not intend to expose in this example we will see a few uh end points and um so you take this and then you copy it to the folder uh this folder contains a spectral rule which is a tool used to lint open API specifications so it goes through each

and every path of the spectral file and it will normalize the paths to H um identify endpoints that basically share body parameters so let me see the let me show you the example here so the API H if you can see the delete user um user email the user email is a variable it's a it's called a path variable H and uh basically it it can has many names both on the API specs and both on the um routes that the API registered registered and um so we normalize this and then we compare uh we simply run the linter using spectral lint Swagger Json and you can see that it provided us if you can

see to your to your right there with the API that are exposed but are not registered properly so those register form and a V1 API docs and um yeah so as I mentioned all of these uh um scripts and tools are available uh let me just open the last slide all of these uh apis and tools are available um through this GitHub repository and um so this is just an example but in this repos they have many examples for like Jango flask and and more so um yep that's it thank you very much for listening I have time for about one question while we let everyone scan the QR code um any questions for me all right thank you so much all right

thanks everyone [Music]