Reassessing 50k Vulnerabilities: Insights from SSVC Evaluations in Japan's Largest Telco

uh thank you very much uh to come to my presentation so uh today I'm talking about reassessing 50,000 V abilities Insight from ssvc evaluation in Japan R Teo uh first of all uh it's about me so I'm hero from Japan uh I'm a sock manager at nme that is NTTS group company so uh in my 10 Years background in cyber security I I read instant response BB management and I also worked for uh MSS actually this is my first time to present abood so I'm little bit nervous but I'm really uh happy to be here thank you and it's about n East uh n East is the largest Telecom carrier having 13 million customers in Japan so we cover

uh 99% area of Eastern Japan so and we provide ftth a fixed telephone line and Leist line and VPN and web services uh to operate these Services uh we have hundreds of servers and network devices in open and closed Network it's a trend in the number of vulnerability the number of vulnerability increase uh continues to increase year by year uh it increased 50 6% in 5 years especially uh in this portion Brock and Portion a critical and high account for over 60% to deal with a huge number of vulnerability uh we use to assess vulnerabilities with our original method uh we equally dealt with uh public and close Network that's a big problem and so Anonymous number of

pabilities required action and B manage like these people so uh we are started considering a new method for bability management uh two years ago and so the new uh requirements are this so first one is prioritize and decide action appropriately based on the cility of varability itself and attack possibility and attack surface and asset value and also asset logic is accountable the last one is most important the number of high priority B abilities requiring immediate action is a realistic number to respond this uh this is our first approach uh it's actually did not work so uh we try to use a CVSs version 3.1 more effectively uh to in addition to Bas metric we we try to use a temporal

metrics such as uh exploit code M maturity and environmental metric such as CIA importance of each service and impact from uh impact to CIA then after considering the each metric vulnerabilities have low CVSs score in the close network uh may not have to be respond ear uh let's see how it works from next slide so this is a uh picture of CVSs calculator so if you want you can try from this URL and this is one example of a vulnerability that has 8.6 uh vulnerability uh CVS score it's a little bit higher and uh let's see the effect of temporal metrics to consider exploit code maturity uh I changed High functional uh proof of concept and

unprovable as you can see uh everyone is still high the next one is uh effective environmental metrics so to consider a confident confidentiality effect because uh this vity has high uh confidentiality so uh I try to uh change high medium and low but it's still high the last one is temp uh temporal metrics and uh environment metrics so finally I Chang the change unproven to the uh exod exploit code maturity and then set LW to the confidentiality requirement uh finally I got uh medium so as I can as I explained our my first approach didn't work well because uh it was not effective enough to pro pro pro ortize vulnerability read the number respond so uh because considering each

metric is not effective enough to change the CVSs score and its severity the CVSs is a good indicator uh to to understand the basic accessability of B vity itself on the other hand uh importance important parameters are positioned as option items such as exploit availability attack surface and business impact and CVSs score uh 0.10 is obtained but no guideline exists for actions uh from the score and also Al algorithms have not been openly discussed and justified by the stand standardization groups so to reconsider the uh V read management for us uh there are several method for B decent method one is K and epss and ssvc and CVSs version four so uh this time I we focused on ssvc this

time what is ssvc so uh I explain a little bit about it uh a framework this is a framework proposed by kegi melon University to address the problem of CVSs and the decision Tre makes a decision considering four key ke factors such as exploitation exposure and utility and human impact and then it output actions to be taken such as uh immediate out cycle and scheduled and defer and also different uh decision tree clearly shows what decision were made this is the entire picture of uh decision three so very small but uh if exploitation is active and exposure is open is super efficient and uh human impact is human impact is uh uh very high or high uh we can get immediate

sorry for very small picture and to use ssvc method to our actual uh V management so we created our own criteria to determine each parameter as input considering our actual environment and the set that line based on uh the output from our ssbc decision tree the first one is ex exploitation the definition and uh criteria and decision no exploit code uh found the decision is none exploit code exists but not LED on the exploit DB the decision is PC when exploit C is public on the exploit DB and available for exploitation the output is active the second one is exposure it's a a system in the close Network as such as management servers uh decision is small

and The Limited system in the fixed Telecom network uh it's controlled and the systems published to the internet or Telecom Network to customers such as web subers the decision is open the utility is a different uh little bit different from others it can be decided uh combination of aable and uh value density for aable uh we use a vector of CVSs so if attack Vector is network and uh required pre level is none and a required engagement level is none the decision is yes and other than that no for Value density impact and exective resources when a successful attack such as web server data database authentication servers and highly uh sensitive data the decision is

controlled other than that diffuse from combination of this output uh the UT can be decided to uh laborious efficient super effective the last one is human impact the development environment a system for internal use only this decision is LW the system to maintain the uh telom services and medium and Telecom Service uh services and R Services other than the following the decision is high the criteria uh critical uh Telecom and socially important service the very high this is the actual example uh for uh actual V VD a spring for Shell the CVSs square is 9.8 for public web services public web service uh exploitation is active uh exposure is open utility is super effective impact

is high the output is immediate in my company actually it's completed immediation on the same day and on the other hand uh Telecom related services in the cloud Clos Network exploitation is active uh exposure is small because it's it's uh Closs Network and the utility is efficient and impact is high the output is uh out of cycle to evaluate uh the method uh we reassessed uh 50,000 bilities for Target data we extracted a list of 50,000 vality from our management database and it's acquired based on the software components information from our actual hundreds of Services over the past few years it's actually uh it actually have been assessed and responded is necessary in our previous work

for and to evaluate these uh bilities uh we uh applied our assess new ssvc method and uh it carried out uh by two people over 10 days it's a a result of the assessment so it's divided into a public net netw work and uh close or control Network and there uh immediate output cycle uh schedule and differ so first of all so immediate is just 8% and out of cycle is 9% the total number of this is 70% so uh we can say uh it's much more more realistic number to theond next one uh let's see the uh attack surface so in the public network a immediate and out of cycle and schedule account for 98% and defy just

2% and in the inclose orol network no immediate on the other hand uh scheduled and differ account for 92% so it effective pro pro prioritize considering open and control or close Network let's focus on immediate so I we checked what ssbc security included and the critical is 15% and high is 50% so the total with this is 65% that descent and on the other hand uh medium and low account for 35% that's interesting if we uh respond to the C CV a CVSs score a critical High stability we overlook this portion so why uh low severity is included in the immediate let's see so this is because of the structure of ssvc decision Tre so even if CVSs score

is low if a foring barity can be immediate exposure is active and the Expo uh exploitation is active exposure is open and human impact is uh high or very high that's why it happened the last one is uh uh attack difficulty so uh attack Vector is if attack Vector network is 67% and uh for attack complexity low and medium account for 97% high is only 3% that's decent so I can say effectively prioritize considering attack difficulty so that's the conclusion our CVSs method effectively Pro prioritize parability considering attack possibility attack surface and a business impact the number of high priority requiring immediate action is 8% that's much more realistic number to respond our current and future work is how to

determine the important parameter so this is the most important one so uh I think we should reconsider this one and other future work is to automate and optimize entire bability management thank you for

listening do you have any questions

oh are you applying this to software that you develop internally as well or is it only for uh third party vulnerabilities in production so so could you say yeah sure uh do you apply this methodology to software you develop internally as well as third-party and open- Source uh components that you run in production uh you mean uh what so uh simple answer is partyware ah third party software thank you for thank you for helping yeah first party versus third party so this is all third party okay so we use so many uh open sources or uh commercial sources to operate our services such as the services so we use many kind of softwares

what do you have any

question um did you get any push back from management moving away from CVS it seems to like CVS is very you know established especially with regulators and that sort of thing they expected oh sorry could you say yes I'm so sorry oh um did management U mind moving away from CVS were they a bit hesitant to um use a a new kind of approach oh uh sorry no so like uh uh Regulators are big fans of CVSs um and I imagine you guys very regulated uh so you mean why I we decided to use CVSs yeah so at the um at your level you know of course it looks like an interesting thing CVS networ but um maybe the

management um attached to it okay so we can we can decide it uh by ourself so it's not regulation uh but so to deal with so many uh bares we started using this yeah it's not regulation respond quickly so the is the mar we know that they want

to did you have any difficulty in automating checking things like the human impact of the vulnerability as you score the 50,000 vulnerabilities yes so we have some hundreds of systems and we decide what what is important and what is not important and uh we try to uh decide by ourself and decide and so it's just manual yeah we cannot manually decide this what is important system or not were there any other places where you used an approach like that to help in automating um the scoring or the categorization sorry what were there any other places where instead of choosing the decision tree for individual vulnerabilities you chose it for like a group of systems or something like that

to help automate uh so the last one just that's my our future work so the B can can be can be acquired uh based on the uh software component information so we try to extract the software information automatically and uh and and uh so yeah that's that's uh sorry that's the last last page system yeah perfect thank you yeah have you uh implemented this yet at the company sorry have you implemented the strategy at the company and seen results of rep prioritizing things so you're focused on reprioritizing these vulnerabilities to be fixed yeah have you implemented this at the company and seen tangible results yet of the prioritization maybe uh uh less lower active amount of open

vulnerabilities anything like that Sor is just a study really did this right yeah this is actual data have you yeah that's actual data have you have you seen the impact of this study have you implemented at your company oh yeah it's already implemented so yeah we we use ssvc to our actual uh parp management so to to decide uh uh to use ssvc or we check how it works yeah have you seen uh like a I don't if you guys measure how many active vulnerabilities you have on a monthly or yearly cycle have you seen a decrease in the number of criticals and highs that are outstanding or like your mean time to repair any of those metrics

pop up as a side effect or a product of this side side effects of something um you have prioritized everything has that prioritization yielded uh quicker fix times for the higher priorities or you now have less outstanding criticals and highs because of

it ah yeah okay so yeah actually the immediate under cycle is about 8.9% so it reduced so yeah so we decided uh uh sorry we decided set that line for each output and uh uh for immediate and out cycle we just we have to respond earlier so it's just about 70% so yeah other than that we we can uh do it uh so s way so sure yeah okay thank

you with the deadlines you guys set do they really hit that yeah I we actually said that the TR but I can not share because of our company's information say sorry yeah I I I can tell you after that yeah it's on YouTube so yeah