Practical Vulnerability Analysis For Development Teams - Michael Helwig and Alvaro Martinez

good uh Welcome to our talk um help my application is vulnerable but how bad is it um we are going to look at vulnerability uh prioritization and Remediation strategies and yeah what you can do to uh get a bit um structure in the mess of vulnerabilities that you might encounter during your um scanning and everyday work so um what is the problem with vulnerabilities what is a vulnerability very quickly vulnerability is a weakness basically in your system software code or configuration um that can be exploited by an attacker and that's basically a weakness um that compromises confidentiality Integrity or availability of the system um there are different types of vulnerabilities that you might encounter so one category are publicly disclosed

vulnerabilities and these are vulnerabilities for example in open source components or dependencies um that you might get through scannings software composition analyzes um there are vulnerabilities in commercial of the Shelf software like sdks as well um vulnerabilities in software as a service products or Cloud platforms and then of course there are vulnerabilities in the code that you or your team produces yourself so in your own custom written development code your infrastructure as code your configuration and so on and for the first category of vulnerabilities the publicly disclosed ones you usually get a cve entry and we are going to have a closer look um what that means um from a developer perspective vulnerabilities are always a bit of a

challenge so um you see the title here don't worry remote code execution is not a feature um this was a response that I got from one of the product managers basically when telling him well you have a remote code execution in your product he said don't worry it's not a feature so everything is good um so you see there is still a problem of understanding vulnerabilities and what they actually mean in the system and if we look at um categories like how much information is actually available to developers and where you have action required um you see that usually the the most familiar they are with vulnerabilities in their own custom code um then there is also information

available on components and dependencies and the least information you usually get if it's like a software as a service product so because that's usually with a vendor to fix it and hopefully you do not have a lot to do here but it is um done for you um looking at understanding of vulnerabilities by developers you get a bit of mixed picture um the Linux Foundation has made a study like SEC software development education survey and uh the result was that 27% of developers are not aware of secure development practices which in return Then means basically that 73% are so you think okay that's a good result but um then the second answer um that they

raised was that 86% of the developers find it challenging to practice secure coding so 70% are but they are not um in a position to actually Implement secure coding which makes it um still challenging to deal with vulnerabilities um dealing with vulnerabilities brings more challenges for V vulner for developers the typical are that our vulnerabilities often appear out of the blue it's often not just one vulnerability so especially if you think of Legacy software you have a lot of vulnerabilities that you might have to deal with um it's hard to reasonably prioritize vulnerabilities especially if you do not understand them there might be a lack of clear remediation process which is also linked to um a lack of understanding and um you

might have too many false positives and irrelevant vulnerabilities there especially if you think of cod scans SS scans and so on um that might frustrate developers so the question is how can we prioritize and respond quickly and efficiently to vulnerabilities that we do not easily understand but that may require our action and luckily we have some tools for this um some tools are the cve entries um the CVSs ratings The CF list and nowadays also the epss ratings uh the CVS rating cve entries those are probably familiar to uh most of you this is uh tracked in the National vulnerability database they are have a scoring from 0 to 10 where everything above 7.0 is high or critical

um and on the on the right side here you see a typical entry for such cve so this is the things we usually deal with um if you look at how the statistics develop it's a kind of a bleak picture so it's not very happy we see that the number of cves increase by year so that means um every year we see more and more um cves unfortunately we also see more and more like critical cves you can see this in the um statistics on the bottom of the page um and if you look at the overall distribution of vulnerabilities by CVSs scores we see that the weight at average CVSs score is 7.5 so that's definitely

on the high side and we see that 55% um of vulnerabilities are high or critical so we see a tendency towards more and more higher and critical vulnerabilities um if we look about around what what it means to have like a high CVSs scoring so I'm not sure um many of you might also know this this score metrics calculator which you can use on the website and um I know that there are some companies who really have the developers go through this scoring calculator um you see that there is base score metric which are most often used which contains the attack Vector attack complexity page required user interaction and scope um and this is the

input used to calculate the score there are other metrics that are not used to calculate the Bas score that you get in the cve ENT and this is a temporal score metrix like code maturity or exploit code maturity remediation level report confidence and environmental metrics that means everything that is tailored to your organization or organizational environment um the de teams might now have the CVSs scorings but then what huh we just received info about a crit critical 9.8 in one of our libraries what to do do we drop everything what do we do um second we did a component scan and now we're sitting on 30 critical 100 findings which is not an uncommon situation for legacy code if you start

scanning at all so what else can we do um the shortcomings here of the CVSs cve are that you have don't have a context usually unless you add it manually yeah so in the calculator you can add your organizational context but by by default you only have the base course so they are generic and not tailored to the context of your application that does not consider architecture configuration or reachability or what role the system plays in your environment you don't have Strat Intel CVSs base scores do not take existence of actual exploits or exploitation activity into account um also if you're looking deeper into the topic there is a delay in adding vulnerabilities entries in the nvd might

be delayed typical it takes seven days for a zero day to be added to the nvd and sometimes there is a sign a significant delay in analysis that means nvd is not always up to date there might be significant lags in data enrichment yeah then you might see the entry but you have no details to that vulnerability what can help um what definitely can help is to understand what vulnerabilities are actually exploited in the wild and this is the information that you can add with a CF list the key the non exploit vulnerabilities provided by the Caesar and the main criteria for adding a vulnerability the CF catalog is whether the vulnerability has been exploited or

is under active exploitation so to be added here um you need reliable evidence evidence that the execution of malicious code was performed by an actor on a system without permission of the system cor um and the suggestion is that organizations should use the CF catalog as an input to their vulnerability management prioritization so think about it it makes definitely sense you have the um criticality of vulnerabilities with the CVSs scoring and um you know what exploit what vulnerabilities are exploited by adding um the looking at the CF list um what would be nice however if you uh take this um ah wait here's a typical CF entry so um it's not very fancy but you can see it just uh for

example here vulnerability for for image magic huh um we now have the CVSs score we have the key entry and what would be nice if we actually would get a probability for exploitation so this is the next step basically in the in the evolution of this kind of prioritization um that we get a score from 0 to one How likely is it that we will see exploitation activity and this score is something that epss adds to the picture so there is a system called exploit prediction scoring system and this is a daily estimate of the probability of exploitation activity being observed over the next 30 days so the idea is here to quantify the likelihood of exploits in the wild and

um tell you are there is there going to be exploitation activity in the next 30 days or rather not and of course you can then go on and try to prioritize those vulnerabilities where you will see a high exploitation probability um the nice thing is um it's a score between zero and and one uh one means 100% probability so it's good that it's uh it's practical that it's a numeric score and the epss score are calculated for cves and updated daily and you have a nice API where you can request those scorings for the cves um there is a bit of science behind all this and this is a bit more complex oh see there's a bit of an old slide but

still um the idea or the question is what makes a good vulnerability remediation framework and there are two um two criteria the one is that you want to have a high cover coverage you want to fix as many of the exploitable vulnerabilities as possible and you want to have a high efficiency that means you want to fix only the exploitable vulnerabilities because usually you might have a lot of vulnerabilities but only a small subset of them is exploitable and this is where epss can help you um to make sure that you are remediating exactly those vulnerabilities um that are being exploited so the idea is really find the subset of all vulnerabilities in your organization

where it is likely that you will see exploitation activity and if you're using epss that leads to a high increase um an efficiency and efficiency means that you remediate only the exploitable vulnerabilities and do not spend your time on um fixing vulnerabilities that might never be exploited and um you can see a comparison between different strategies um for example if you go with the um CVSs score in fixing vulnerabilities that have um a CVSs value above 9.1 um you are remedi remediating like 50.1% of cves with a coverage of 33.5% but only an efficiency of 6.1 um% while if you um use the epss value and fix vulnerabilities that have a probability of exploitation above 2.2%

you get an efficiency of 24.1% so way better and you can just see this in the picture basically it's the overlap between the blue and the red uh Circle here yeah the better this matches um the better is in the end your coverage then of fixing the right vulnerabilities so this sounds pretty nice we can now find out what vulnerability are actually relevant um based on the score and it takes a lot of data into account it takes into account CVSs metrics cve list publicly available exploit code offensive security twits and scanners um age of vulnerability and so on and it throws all this information into a machine-based algorithm a cross gradient boost algorithm and generates

this value between zero and one for you huh so this is pretty nice we have an probability value that we get out of this which is also nice if you do some risk management or quantif quantifiable calculations based on this um and gives you an indication what vulnerabilities have a high priority at least at the moment um and what you can focus on the issue however is that the epss score can change as we learned it is calculated daily so it can be recalculated daily and in some cases this leads to quite some interesting effects so here you can see in this table that a vulnerability um which had a very low exploitation value of um of um this

0.15% has a new a um apss score suddenly of 93.7 1% so from no likelihood basically you go to nearly 100% likelihood of seeing exploitation activity uh within yeah a bit more than a week here or something so this is this means you basically always have to to apply still monitoring to those kind of vulnerabilities that you deprioritize for remediation um so still advantages of epss it leverages real vo World exploit data it incorporates a variety of data sources it provides a probabilistic value um it significant signicantly increases coverage and efficiency and it's quite low effort to use actually because you can just request this value from from an API um the downside here a bit model

opacity so it's a machine learning model so as you might know if you're if you're deal with machine learning it's always hard to predict the reasons for a specific outcome from a machine learning model so it's a bit opar um the databases it's not 100% clear if the list of exploitation activity is really complete there might always be exploitation activity that goes unnoticed um limitations in scope and application so it relies on an existence of a cve ID and as we learned there might still be this kind of delay in the cve IDS um and we have seen the fluctuation in the epss scoring which also means that you still need to monitor your vulnerabilities um going back to our

developers now if we think about it what could be a small process first we check our cve scores we might fix what is easily fixable and for the rest we go into detail and check the epss scoring and then only when we have an epss score that is relevant for our organization we go into more detailed analyzis here and see if we really need to get active and for the rest of the vulnerabilities we might just go on and monitor them okay and now I'm handing over to ivaro who will show a bit in the practice how how this can be

used hi everyone um so I'm going to talk now a little bit of um practical case of how we deal with vulnerabilities we have uh especially big list um normally we obtain list like that when we do uh um vulnerability scanners other it's like container scanning or SCA for example and uh what we first do is we exract the CVSs and epss you can do that easily with the apis the official apis you can also use tools like CB map that will automatically grab the the ratings and will uh output it via command line terminal and once we have these ratings we sort them out and then we normally get what is closer to um critical around

nine and we also focus on those highs that have uh very high epss as well because that will tell us that it's uh easily exploitable and with that information we can kind of build a short list and in this s list we normally have to go uh through individual analysis because um otherwise we cannot just rely on CVSs or epss to to prioritize or not a vulnerability so we're going to take as an example this CV 2023 uh 53164 which is uh strats path traversal maybe some of you know this was a critical vulnerability that was uh released end of last year and it basically fails at validating uh file upload parameters and it allows uh pth traversal characters

and that can make or is um an attack like uh for example changing the expected directory for an upload um under certain circumstances it can lead to remote code execution and something important to understand uh when dealing with the vulnerability is the attack vector and for this case um this would ability allows to upload the file name of the of an upload so normally in an upload you control the attacker will control the the file name it will control the uh file content and the content type as well and um the attack Vector here could be something like we see there on the screen on red um it actually could introduce the dot dot slash uh uh like as many times as as he

want for example to to go up directories and then CH change the intended directory um for this example to place it under web apps where is the uh where typically an application is serving files and um here we see websell uh because this is uh typically used for remote code execution it's it's like a web view uh where you can control basically the underlying uh operating system by uh passing query parameters with commands and then you will get in the response uh the response of the common line so we'll go first quickly through the Cyber kill chain the Cyber kilon is a framework that it's used to understand U basically how a Cyber attack works and um we have like the

recognizance phas where the attacker gathers information about the the application either like manual exploration or like vulnerability scanners to find uh flaws Orab is then the weaponization phase is where the attacker uh basically Builds an exploit uh that will deliver in the next phase and then we'll have exploitation phase where if the um system is vulnerable uh then the attacker will will gain control um the installation fa phase is to maintain control uh by installing that exploit and then come and control phase um the attacker will basically uh just control machine and action and objectives will will for example do things like uh run someware or like stealing information or whatever uh the objectives are um applying the Cyber

kills in for strats um uh first it will be like the recogniz and pH so what will happen here is that the attacker will find an upload functionality that is using strats 2 and um then it will go to the second phase which is building a GSP webs IP stands for Java server page because starts to um it's Java uh Library vulnerability and then the attacker will deliver this DSP web so how he does this in this case it's very straightforward because there is an appla functionality which is vulnerable so he will just do Post request manipulating the file name and introducing the uh dot dot slash that we mentioned before and here is where we go through the exploitation

phase and this is um where strats 2 is is failing so what is happening here um we have there as an example temp uploads is where the server is supposed to save files to process them um and what is happening is when it's calculating the destination file path it's not validating the file name and then it's going to director's app meaning it's going to the root directory and then it's changing directory to optom web apps where is uh in this case where the application is serving files and then the access the attacker could access them in the next phase sorry installation phase so here we see JSP is available uh the website already and here the attacker can just make a get

request to retrieve this uh JSP and that will give him basically uh access to the underlying uh operating system having remote code execution so having understood this uh now we have to look how can we deal with this like what can we do as developers to to solve this so for recognizance phase there is not much you can do but um there are some cont meurs like for example uh do not return Barbo server messages that can leak for example you're using uh strats vulnerable version um or you can run software composition analysis to find the flaw before the attacker does uh then we will have um the delivery phase so here is one of the

most important and it's one of the questions we always do to developers uh like are you using strats 2 framework and if you're using it are you using it for upload uh functions that are exposed to un trusted users uh because if that's not the case then you directly are not vulnerable um then we can also think uh think about stuff like uh normalization of file path so are you uh removing this um dot do SL sequences or are you also like doing input validation or sanitization in the file name or not uh that would be for the delivery pH then we have exploitation uh here there is a very um important check that could actually be a

cter message to mitigate this which is checking if the destination file path that has been calculated is using the canonical path for the upload directory that means if the destination file path starts with uh the if we go to this slide we can see it clearly the destination file path which will be upt web as it starts with temp uploads in this case is false then you will automatically block the the upload and uh then uh there is other things like file type validation you can check uh extensions for example this allow GSP or PHP if you only need to process for example P PDF files then you can disallow everything that is not a PDF

extension and uh for the installation phase um there is also the possibility to limit right access outside this uh uh destination uh directory the designated directory so that if the attacker tries to write a file somewhere else it's it's going to it's not going to it's not going to be possible and uh then there are also some other Solutions like Baseline monitoring uh to check Integrity changes on the system and those would be basically the three phases where where the developer uh the developers can do something uh to mitigate this then uh the other two phases we can talk about some other uh mitigate factors for Comm and control on actions and objective like KS CM uh

Network segmentation or least privilege principle but uh normally are in those three phases where we focus on and we check if we have some of these Texs in place and then we can maybe move on to another vulnerability or we really need to like update this library or Implement something in the code to to defend um the application and that's it from our side we leave here some resources like CV map is what you can use to extract the cbss and epss from the cbes also the official apis and there is also a small uh proof of concept there in my GitHub where you can see um and if you want like a test this

uh this vulnerability Strat soell and that's it from our side thank you very much [Applause]

so uh we have time for one question who has a question for our illustrious speakers you were close thank you thanks uh thank you for the talk um I wonder how is the correlation between a vulnerabil vulnerability being in the Kev and having a high epss scores so is it like the same information or is it a different information look the the K I think has a more narrower set of criteria so you need to have a reliable proof of exploitation for to be added to the key V list in my understanding but the epss score is takes more criteria into account it takes into account um like is there exploit code on G GitHub and some

like this it I think it monitors um social media activity around exploitation discussions um it m takes the age of the vulnerability into account and things like this so it's a broader set of criteria um that they um factor into this machine learning algorithm so because it's all data driven I think they have the possibility to add way more than do it manually with like um the the key V list where you really need um to have a case of exploitation okay thanks so please give Michael and Alvaro a big round of applause thank you so much thank you very much