CVSS Is Sh*t! Or Is It?

um thanks for having me uh so I'm going to be talking a about CVSs um covering out why I think the industry thinks it's rubbish what my personal view of it is and then also covering off CVSs 4 that's coming out soon so just out an interest here um who thinks CVSs is rubbish okay great I have some convincing to do um this talk is relevant um so just a bit about myself uh so I work at feature space uh self plug if you're interested in roles we have a couple rols out um if you are interested come speak to me afterwards um but other than that uh so I my principal application security engineer

future space I do other loads of things that bottom logo there is probably out of date now ASD but oops um um but yeah I'm involved try to get involved as POS as much as possible within the community so quite active on Twitter X whatever the heck you call it these days um try do streaming although my streaming is kind of Tak Pune lately but try give her uh actively involved in the community and help as much as possible so one a cover of secur uh CVSs I think it's important for us to take a step back and ask us all what is security what are we actually doing when we say hey we making things more secure

I work in security what does my role actually mean to an organization so one of security important things is about risk right you trying to take risk for an organization and reduce it to a level that is acceptable for that organization I think this is really important because my if you even view it on a personal level my personal view of a risk I'm not Dr scy because I'm terrified of HS so to me that risk is way up there where someone ask I love scy and they're like it's fine I'm going to go jump out of the plane every weekend so to me I'm going what the hell is wrong with you to them 75 so our risk

levels are totally different and the same happens for an organization each organization has their own risks their own things that they would accept as the different levels of risk simply security is about managing risk managing risk for an organization well that's all great what is risk most of us not all of us in this room have seen this equation and risk is simply what is the impact of a security flaw or vulnerability and How likely is that to occur so if this thing happens what does it mean for my organization and also is it going to happen and one of the things I love is this kind of uh diagram or picture or whatever you

want to call it um and that's kind of for me PS depict risk right right so at the top there you pretty much guaranteed to walk in the thing in the ground whatever the hole in the ground but the impact is very very very low so you're going to walk through there but the chance of something bad happening to you might sprin your ankle or nothing happens so therefore the risk is very low similarly it could be a massive big hole but the Gap is very small so the chances of you falling into the hole not impossible you might trip and fall and cause yourself great injury but the chance of that happen is very low so

again combine the two you have very very big hole L spars at the bottom and a a very wide Gap well chances are you're going to fall into the hole and possibly done pretty bad risk right and another example I was thinking about like trying to use real world typ scenarios and ready going from one extreme to the other so most of everyone in this room drive or most of us drive in this room right how many of you ever given a c think well I'm not going to drive today because I'm not get involved in the accident and you know I'm not ready willing to take that risk [Music] for many people um it's because the

likelihood and the the impact is going to be relatively low right so there are chances that you're going to get involved in ch accident but it doesn't necessarily mean that that's going to be instant there on my other hand asteroid hidden Earth the impact is literally huge uh if the asteroid is big enough and the uh enough to take out entire planet well the impact is significant but how many of us put your hand up in your if you think about an asteroid hidden Earth the Tes of that happening are so insignificant that it's just become negative and also the other important thing not about that is there's nothing we can do about it well at the moment

there are trying to work ways of trying to solve for this uh some time ago they smashed a satellites into meter last some time but still the asteroid we have no chance of stopping it again the risk is very very very low because the chance of Happ very low where if you're getting a car like I'm going to drive back tomorrow there's a chance I get involved in an accident tomorrow touch wood doesn't happen um but and something significant could happen to me but it's still a risk that I'm going to take because the risk of facing my other half 10 is not going to be wor it um so what does this all have to do with cvsis

why am I talking about risk and impact and all this well cvsis is a really important thing say you have this vulnerability right in your organization yeah a vulnerability and a software product you use or some process that you have how do you quantify that vulnerability how do you go to the business and say well this vulnerability is something that we have to look at whereas this other vulnerability is something that we can live with so what other things that we can do well one of the things is we can kind of finger so can go well I kind of guess going to be this or a kind of guess it's going to be that um or you can come up

with your own metrics so coming up with own metri is fine but then how do you start applying that to software or vulnerabilities that you have no control over say a s provider or third party software provider that you don't have a lot of information about how do you then assess that risk and this is where CVSs starts coming into place so it is a industry standard way of scoring vulnerabilities I'm going to hesitate and say not necessarily risk but it is a way of scoring and I'll going into a bit more detail of more the impact of a vulnerability but the important thing is it's a common language that we can use across the board Microsoft has a

vulnerability that will use this metric someone doing an open source project as a vulnerability they'll be using the same metric so therefore we hopefully and it's not always the case but genuinely measuring things on the same kind of way so just a bit of history about CVSs um first came out in 2005 um was version one and then it evolved to version two I think I was long enough in the career for that to be around just as I was turning to version three um so version two to version three wasn't much of a difference and then we went from 3 to 3.1 uh recently in 2019 are great well there's a lot of debates

online whether they actually provide value um they do have their fors but personally I believe they have value um and if you want to go see hate CVSs hate put something like that oh oops thing there left out the v um but the point is you still uh if you go put an or whatever you'll get results and also teaches me to double check 1 things my opinion is people use cbss all right I've seen many tools so J is another classical example many people go oh J's horrible I hate J I love J that could be because I'm weird um I like teams so yeah probably have weird but um my view of CVSs and is they're both

Fantastic Tools they're both fantastic metrix it's just not used correctly like many things if you don't use it correctly it's not going to do what it's supposed to do and there yes it's going to be rubbish so take for example of anyone not familiar with log forj the vulnerability that came out well just in case anyone's watching um just give a very brief overview what it was so log forj is a loging framework or or library for Java making it clear log forj is not Java when people transf there was vulnerability F that could you could do bad things with it basically using Java name directive JD with acony stands for another thing I am

is horrible with acronyms um but allowed an attacker to basically execute commands on our system running the vulnerable version of log forj and this is a screenshot or portion of a screenshot from the nvd scoring of uh the for vulnerability well the first because when it came out there was a fix and then there was a vulnerability and there was a fix this is if you for for for sh sh for what was called um that's a CV that he found and everyone to cbss 10 right and everyone went oh my gosh the world's ending we are all going to um suffer these major data breaches and I remember saying seeing some sensationalist things like this is the

worst s vulnerability ever found and blah blah blah and when we look at it now it's not in the room so um wasn't that bad no do we have Brees yes we suffered breaches from it but wasn't that significant was it the worst s security incident or vulnerability ever no a far more likely that's going to happen for your organization is you're going to have a user for for fish go read a really interesting book The Lazarus H I found it very fascinating it's about how the North Korean operatives working for the north well supposedly working for the North Koreans uh North Korean government a Lazarus group um how they infiltrated different banking organizations to steal money

from it what is the the single entry point from most not all this fishing look for J vulnerability was getting someone to CLI on ack so here we have these people going oh my gosh vulnerability is the worst thing CVSs 10 ignoring all this other stuff and if we come back to what I was talking about is managing R so how is something like CVSs 10 not impact or unlock sorry CVSs 10 with unlock event happening more important than something that you know could have big impact if I came to you and just said hey this vulnerability is number 10 right what would you do see we but I just give you another number 10 it's cly me right it's and I put a TW

that for this a score without context is meaningless it's just a number how are you supposed to assess anything is a network access exploitable what's the complexity like does it involve uh tempering with data or modification data if you're looking your see triangle so what we should be doing is rather focusing on the score one thing I always say about TS says focus on the vector the score it can kind of help but for me far more important is a vector I can work out or is a network exploitable how hard is it to exploit what does it mean if it's actually been ex that to me is going to mean far more when I'm assessing the vulnerability

rather than going oh it's top score must stop everything and fix it so when you compare those two you get far more information you can make a more informed decision and that's the important bit making that decision based on information that's relevant to your organization and when you break it down um so one of the nice things with the NBD calculator is you can see all the different GRS and that but you can then also C oh sorry one of the things with CVSs is again when people don't do it properly is they just go Bas for they don't actually modify it for the environment so in this case for one of the things that drove me

up the wall was yeah someone could get a remote code execution on the box is the C now you're running a custom application that level of complexity for them to be able to actually exploit it is significantly more because they most likely need access to the logs to in order to see what the payloads do otherwise they'll just be completely blind sending all these requests is those fields that they can control are those even belonged P not it so now you start uh modifying the score and adding in things like environmental support metrics and that's what it was designed for making sure that you can customize the score um so you can score it for your own

environment so now you're starting to tweak the score so you can have more relevant information when you go to security teams or C or other teams saying hey this is why we need to fix this or this is why this isn't such an issue that we have to worry about or even talking to your customers some of them might be for you have to fix not you go hold on hold on let's look at the schore this is why we don't think it's such an issue because it's standard you're now speaking on that same language this is the other thing that draws me up the wall when trying to assess vulnerabilities here works with trying

to assess vulnerabilities basically vulnerability management this is a x this is taken from nvd for the vulnerability for work I kid you not that was the description how are you supposed to do anything with that when you're trying to evaluate your the risk that that poses you yeah so this is me personally looking at okay what does this actually mean then I get frustrated because I'm trying to assess this vulnerability and I just can't figure out what the heck I need to do with this and then I just give up and just go final Point going just c as I mentioned at the beginning I kind of alluded to this so when we view risk

it's an impact on the CVSs unfortunately is very focused on impact not so much like the only real field they have there is difficulty in exploiting and that you could kind of relate to Black sure you you could also take things maybe the T Vector is a network network exploitable local so on so forth but in reality most of it um more B towards wait CVSs 4 is coming so the new CVSs version is coming out soon now so is it going to actually solve everything and be our magical B well if we compare some of the differences between CVSs 3 and CVS S4 um I've just listed the basic sections in there I was trying to figure out a way I

was trying to show this in a non boring way just going round the text so hopefully this will help but there are now more sections which I think is good I'm going to a little bit of it in detail shortly so one of the big differences is there's the base metrix then there's environmental but then you have those kind of exploitable vulnerable subsequents so the idea behind the vulnerable system metrix and the subsequent metrix is okay well this system's been exploited what can attacker then do from that system so in the past it was just very much a binary yes or no it was the the I think scope changed or unchanged now there's a

bit more scor to say okay if an attacker exploited this vulnerability can they then use that to that effect the other thing is we have the supplemental metrix and this is a really cool thing I think so what happens here is typically the supplier producer of that software or system or whatever would typically fill that out is one of the more difficult things is we didn't have a context from those who know the system right those who should be able to answer all the real hard questions of like how bad is explo what does it mean if it's exped that's now hopefully f up by the SP and then you can treat it like you did in

the cbss 3 with your environmental scool so that's to be complet devel consumer some of the interesting uh metrics that I noticed so an interesting one there is they now building in safety so this is about human safety I guess this is something that we've always kind of not really considered so if this vulnerability is exploited what does it mean for actual safety of humans I imagine this having a very big impact especially in medical related value density is another one so what resources will the attacker actually gain control again further information of what theability um and then some other ones there uh vulnerability response effort so how much effort is it going to take

to resolve the the vulnerability exploited uh how urgent it is from the provider perspective so the provider going oh you have to drop everything fix this probably something you want to listen to um and then the environmental kind of touched on there's also the security the scoring is also really interesting so want to show you that score uh CVSs 10 right everyone normally goes for the base score but if someone provided a different score it was very hard to tell what did what score are they providing now they have this n nomenclature um kind of scoring mechanism so when someone provided you the score they can go well that's the CVS says b score and then you can know

what it actually means which I found pretty interesting so what does this all mean well let's take an example um and I think this is really important because the highlights how this extra information is is meaning that you get a more accurate score so if we take off for J for example what I did is I scored the that one that I showed originally so I did use the base score and then modified for a system and result in 7 7 I then took the same system and rated it for cbss 4 and it suddenly went down which is good because in my view that is a the medium was a more reflective uh measure of that R for that

specific vulnerability we that an attacker would have to go through a lot Hoops in order to exploit it the exploit might have not been that significant you also get a lot more information now so you can see that that back stren top one there bottom one far far more so cvsis what does that mean um so in July this year they started the public preview so you off you can actually go see it on the first side the the whole specification play around with the calculator um the comments are still open for four more days I think pretty quick you can get them in oh no actually I think we close they just reviewing them um and then the go live is the end

of October so million dollar question was CVSs solve by all our problems head up if you think of

all so it it does a far better job of impact measuring the impact um but unfortunately the likelihood it's still a bit yeah it's it's still not answering uh which is disappointing like when I first saw CVS come out I was yes this is going to be our Saving Grace we're going to be able to score things on CVSs 4 be able to have this awesome metric and yeah but un but I still think it's really important and there are some other things that we can leverage to help and these are the ones that help with more of the likelihood aspect so epss is exploits prediction scoring system so that measures I think a scale

of 0 to one of How likely a vulnerability is to be exploited by there's a tool out there called dependency track that takes the PSS and the CVSs schol put them on two matrixes so I think it's the CVSs schol on the X access and the epss on the Y AIS so you now basically have quadrants can go and say well if it's in the top quadrant yeah we kind of need to look at those and then you work your way down the other really really really uh useful tool um that's becoming more plugging up my eil plugging up my email box more and more these days which is woring is the ca exported vulnerabilities K and that's is

something that's maintained by the US government and it's about the vulnerabilities that attackers are actually exploiting out there in the world so the idea behind this is if you got any infrastructure that you suspect attacker may be able access and you get one of these um notifications for vulnerability in that drop what you do fix it because it's not a of may do this is doing this so the million dollar question I'm actually interested to see from the audience hopefully I convinced those at the beginning um do you think CVSs is a worthy or worthwhile scoring me anyone who feel disagrees okay I've failed I'm done I'm going to go home I mean it's going to take a lot of

convincing um there is a perfect how is it going to solve our problems I know but I think it's moving in the right direction so if we look from where I came from CS one CSS there's been a lot of changes cious is called I hope it said this will not be the last one right of all one of the things I want to see happen more is the likely Factor being incorporated into the sco I think that is the key aspect of cbss um it's too focus on the impact with that finished way earlier than I was expecting so it's a lot of time back any

questions

[Music]

yeah

it's not um so organizations are been out I mean trying to score and work out different using different scoring to Hope measure and it's still very hard one of the things I found initially when I was doing is well how do you put metrics so how do you measure this against that try make as objective as possible why exploring mechanisms it's because it's objective you don't go well I guess it's kind of that or I feel it's like that or one day I decided that I Chang my mind today because I had a bad day or something um and I use the thre Matrix to try assess that even that very much well I kind of guess that so yeah

um if we can somehow have automation where a system does it a system will pretty job of do consistent I think that's othery thing consistency when we RW vulnerabilities process buch People review vulnerabilities my thought process to a vulnerability or something en to explo it is different to someone else we will then be different to someone else so you got all these different people thinking or assessing it differently if we had a system it'll be a lot more consistent [Music]

it that's yeah so having that as much as possible I think another really important thing is so I work in application security one of the the hardest things is I've seen it before a security team goes to the de team oh this is a cbss 10 fix we don't use that why and no it's a cbss 10 you have to fix it and I have to step and go why that getting collaboration your technical people have the context behind how the systems used the way your security have the context of what that exploit means put the two together and then you collect this is a really important thing collect collectively and collaboratively agree on this is a high or medium or low and

then using some of those metrics there to help drive you towards that I

[Applause] think

um again that's

kinds so something I need to look further another time I know like by introduction to it was by dependency train where that's kind of where I first came across and as I said you do if you combine those two and kind of get quadrant the other problem with epss and this is one of the big ones is not every vulnerability has score and in fact not many of them actually have that especially historical ones so that's certainly

[Music]

[Music] exactly and what's what's under system so one of things I like using as an example um so I have a personal BL right I could have a vulnerability that goes okay well Leakes all the information system well our so what would that mean theability obviously it's like credentials to Lo in in but I mean say it was behind the pay or in my case there is pay there that I but I don't so if they exploited that they w't gaining public yeah exactly right about the and that's why I think it's so important because people just go okay she has the score we have to rate that or we have to have the score and then s of

vulnerabilities based on that and they don't actually go take that vulnerability and to the envirment score for their environment why I think I think

yeah and the very dangerous thing about that you're going to be spending all your time fixing all that and you might have thatting that's what used to reach but

hey that's the other thing and even if you have the capacity there are some really complex systems out there especially for Organization for Aller infrastructure that it's just not impossible and this is why I hate Google's um their Google projects Z with a STI oh it's 90 days and we'll make it public well it's not always that simple sometimes it's really complicated to fix the team's trying to do the right thing they just can't do it in 90 days it's not possible so these kinds of things are not black and white there's a lot of gray areas CH of green blue and whatever you want to all color any other comments yeah um so C

this for going to improve the description updates known exclusive but they don't say how where it's Sol it it will help provide for the context a bit I'm not going to say like a bigger but it's going to help provide because the vectors not growing add in more information especially from the supply side you're not going to have to assume as many things prior so hopefully my hope is that it will uh add that additional context to at least prove things enough um how much so I don't know but yeah um Oracle is famous for this because I used to have to review Java and their descriptions were absolutely rubbish you get the very basic vector and you have to then assess

what does this mean foration and the default was then we have to assume the wor hat not if if the vulnerability is not already being exploited then the supplier doesn't want to disclose that and give the attack Advantage Apple doesn't tell you anything about so if if Microsoft doesn't choose to utilize the additional pieces in cbss 4 then we just have to go back to the for the updates on y exactly

yeah well there's a certain I'm not going to name them there's a certain s provider he's only recently [Music] switch

they lower down alphabetical order but yeah that that's again that's going to be interesting on cbs4 see how quickly EV is adopted I'm not holding my breath and nor do advis my M probably Suffocation for you um but hopefully I'm wrong some of the new technology like the scap on the cloud uh configuration with vulnerability with u then you start to going look at the full into to end risk and the challenge with getting it back on premise is now you have to have a CB and that's management platforms that are all up to dat and accurate and build in theability pieces it becomes very challenging Cloud we may be able to solve this with ML and AI in the very

near future that's actually interesting point because I was going to ra it earlier is vulnerability management data the amount of times I've going to review Rel I have to collect all this data the more data I have the more so certainly time your thread your your live running system that's all really helpful information you have to make that choice is this critical or Ed or so and the thread end I think going to be one of the really important parts in really really helpful let's hope the bad guys don't call that yeah all right well thank you all for coming um and hope was somewhat helpful may have changed slightly a little B maybe break it down a little bit more

but yeah