The Art of Concealment: CVE’s Challenge with Transparency

All right, welcome everybody. Good afternoon. Welcome back to Bides Las Vegas and the Common Ground. This talk is The Art of Concealment, CVE's Challenge with Transparency by Jerry Gamblin. And a few announcements before we begin. We'd like to thank our sponsors, especially our diamond sponsors, Adobe and Iikido, and our gold sponsors, Drop Zone AI and Run Zero. It's their support along with our other sponsors, donors, and volunteers that make this event possible. These talks are being streamed live to YouTube. And as a courtesy to our speakers and audience, we ask that you check to make sure that your cell phones are silent. If there is time after the talk, I will be passing around the mic for any

audience questions. And just as a quick reminder, as per the besides cell phone policy, there is no photography allowed. And with that, I think we'll kick it off and hand it over to Jerry. Take it away. >> Thank you so Let's give him a welcome. >> Thank you. We'll get right into this. We only have 20 minutes today, so I think we'll be able to get through it in plenty of time. But a little bit about me. Um, my name is Jerry Gamblin. I come from a government background. Spent my first 10 years in the government. Um, I spent a lot of time running VM programs. I went to Carfax after my government term and then I worked for a small

startup called Kenna Security who, you know, kind of pioneered riskbased vulnerability management. I then went to Cisco where I'm at presently. Uh I'm a principal engineer there in their threat detection and response. So I handle the CVE data that we get in for our customers. So I'm on the other side of the board where you'd think of Cisco's at, right? Like we have our PERT that is for Cisco that publishes all the CV for the routers. We have Talos who publishes all their independent work and then I'm kind of the third leg over here complaining about the data. So, I showed them this talk before uh before I'm showing it to you guys. So, any

questions you ask me cannot hurt my feelings after what they said about the talk. So, so we are we're good to go. Um I'm a member of the EPSS SIG, too. Uh I like to throw that in there because that's an open SIG and if you're interested in exploit prediction and data science, it's a great place to kind of to get your feet wet there. Jay Jacobs runs that. He's in the room. I'm sure he'll be happy to talk to you guys afterwards if you're interested in that. So, I am moving some stuff around. I have officially moved all of my open-source vulnerability intelligence software kind of projects under Rogo Labs because I got I have four of them

now and that's too many to to just run under my personal GitHub. and GitHub who we also have in the room was super nice enough to give me a free co-pilot organization. So, so now I have that. So, it's all under there. So, CVE.ICU is under there. It just pulls in the data and runs through that every day. I have CVE forecast.org because I'm a big nerd and I want to know how many CVEes are going to be published each year. So that goes through every every night and predicts how many CVs are going to be published by the end of the year. Um and I have patch this.app which is just a simple answer to the CISA Kev. Um I

thought the CISA Kev was missing a few places where there were data so I just went out and pulled all those in from metas-ploit etc. So it just builds a list there about 8,000 CVEes in there now that you should use. Um, it does use the CISA Kev. I am a big fan of the CISA Kev, but I don't think it it presents a full picture of all the exploited vulnerabilities on the internet. So, the promise of the CVE, there are a lot of very smart CVE people in this room. So, I so talking about this here is is a little weird, but in 1999, it started and it simple thing was we were going to put all the CVE in one place.

We're going to give them a standard name. They're all going to be CVE with some identifier and then another identifier, right? And so it started out super simple. There was, you know, 300 of them in 1999, so not a big deal. Um, last year we broke 40,000 and I got a lot of press saying we broke 40,000 and then 30 people went and rejected the CVEEs that they published. So we ended the year with 39,970 published CVEes. So it was close uh rounding error basically and a common schema. The common schema was supposed to say what this vulnerability does and kind of give you some information on it. Todd's in the room. He'll tell you it's

not how you're supposed to fix the vulnerability. It was just for publishing. But that that's a different question. So I feel that there the broken promise of the CVE program is this right the poor data quality. So I was first going to give this talk when I when I submitted it. I was going to talk about how CNAs like Microsoft have over 125 CVE published this year with the exact same description name. Um but I didn't think that that was worthy of the full talk. So I thought I would so I changed it up a little bit. Um it also cripples automation. You started noticing this after NVD had their funding issue uh in 2024.

That's when people finally realized where where the most majority of the CVE data they rely on came from. And it wasn't the CNAs or the CVE program. It was the NVD. So when that broke, we started to see that. And then we have research fatigue. And I'll talk about this a little bit later. Uh there are 460 individual organizations now that can publish CVEes. That means there are 460 different data definitions that you would need to learn and understand to be able to easily understand the data that's coming into CVES because everybody says, "Oh, we do it this way. We do it a little different here. Here's where you get this information." So it really broke the

promise of a standard language for CVEEs. So, people are going to disagree with me on this, but as somebody who spent my life in in the blue team, a CVE needs four things to be a CVE. It needs what kind of weakness it is. We use C.WE for that, right? We use what's affected, what product this is, right? We use CPE. Everybody hates CPE, but there's nothing better. It's the standard. I will put Pearl on on this list as soon as the CVE board approves the schema change that has been in progress for three months at a minimum. Yeah. Um, how to fix it. I know people will say this isn't part of the original

thing, but as somebody who needs to know how to fix it, I want to know what the patch information is and the severity. Once again, you can all tell me how much CVS sucks and that it's not great, etc., etc., but everybody uses it and everybody needs it, right? So, the first pillar is obviously the CWE and it is needed for people to understand what the overall vulnerability situation is in their network. It might seem like a little thing, but mature vulnerability management programs track this information so that they can tell where they're most likely to be vulnerable and to decide where to spend their money externally outside of the of the VM program, right? So if I know that I have

a bunch of DDoS based vulnerabilities that I might not be able to patch, I can then say, hey, maybe I need to increase my DDoS protection through Cloudflare or whatever. Cross- site scripting is the same way. Um, that's why CWE is important. CWE is also ran by MITER and they've done an amazing job over the last three years. They're actually up to 80% of all CVEes now have a C.WE on there. Um, I think that's the tipping point. I think if we have 80% of them, that should easily become a required field, right? Like just click it over and say now we require these all to have them. The product, this is this one gets me all the time.

CPEs are the only machine read machine readable way to get product information out of a CVE record today. Um, but they're not required to publish a CVE. Uh, and most people do not do it. Does anybody want to guess how many what percentage of CVEes in the last six months from a CNA has a CPE on it? >> 2%. So CNAs are only providing 2% of the CVs they publish with CPS. um they'll tell you it's not required. They'll tell you they put that information in the description. They'll tell you they put it in the affected field. Um very smart CVE board members will tell you that you really don't have to have this information, but it can be

in the re like there are only three required fields to publish a CVE one of and they will tell you that you can munge one of them into the description and not have it. Um my next question to them is always how do you check that the data is in there before they publish it? Because the schema validation for CVE description is one character. That that's all it that's that's all the schema validation for CVEES as of today. The severity CVSS does anybody in here like CVSS scores? All right. Couple of people. All right. So, it's not bad, but you know, most people if you run a PCI shop, you know, you got to patch everything

that's high and above, right? And we need that information in the CVEEs. How many people who need CBSS scores have had a hard time finding them recently since NVD stopped producing that data? Right. Same deal. We're getting close. I think it was 70% last time I looked. more and more CNAs are starting to put that data in there. It's really skewed towards the bigger CNAs are doing it, but but we're missing that data as a normal data point that we need. And the last is the patch info, which I know is probably the most, you know, the one that most people want to fight me on. It's because they say the data is there somewhere, but it's

not in a reference in the CVE record so that I can just look and say, okay, here's the link I click for the patch, right? I have to look at eight or 10 different links in the references and figure out which one of these is it. I got to know that Cisco puts their patches here, Microsoft puts their patches here, and it really doesn't need to be that hard. The CVE program has a patch tag that everybody should be using. So [snorts] let's just go to the impact on security operations. It's without CPEs, most of your scanning tools break. It's it's as simple as that. The NVD was propping up a billion plus dollar a year

industry for security scanners because they would fill in that data and then scanners would take that data and scan your networks and says here's what it is. I was part of those conversations. I was part of that scramble last year to figure out how do we do this? and everybody came up with a different answer and none of them were really great. So we're now back to relying on CNAs to do that and even NVD to some point. We have to have that data to be, you know, to be valuable and we have wasted time and resource strain. I know that I know that people say the CBE program is a volunteer program and that people publish these voluntarily,

but this information is not consumed voluntarily. It's consumed by people who need to know where the data is and how to get to it. So, incomplete records, half records, buried records make it super hard for people to do their job in a day in dayout basis. So the CNA ecosystem problem, if you were at the CVE panel, you heard Chris say that from the last 10 years, I think from 2016, they tried to increase it and they've done an amazing job. They went from 30 CNAs to over 460 CNAs this year. That's great. But nobody here has time to learn 460 different data models and understand how each one of these CNAs are going to publish this

data.

So a path forward, it's been said, I couldn't even figure out who to say this to, but you can't improve what you can't measure. And we've never measured CNA quality in a way that makes sense. So about four months ago, I started to build CNA scorecard. Um I went to work on it. I built a first model. It used machine learning and AI and it was really really cool. And then Bob Lord sent me an email and said, "Hey, can you explain what this 82.67% means?" And I'm like, uh, not in an easy way, right? Like, like I could sit down and do it. So, I went back and said, we're we're going to step

away from that and we're just going to count fields at first, right? Like if you have the field, we're going to say it's good. We're not going to look at quality yet. We'll get there. But we've started to build a project to help manage that. So, today I'm super happy to announce the public launch of CNAcorecard.org. It's a website that gets updated three or four times a day using GitHub actions. What it does is it goes pulls down the CVE v5 list uh processes all the CVEEs published in the last five years, lists them and then tells you or last sorry last six months and then tells you which of those four four objects they're missing, right? Do

they have the CWE? Do they have the patch info? do they have the CVSS score etc. You can go in and you can tell by CNA which ones are filling in which information and I only have two minutes left so I am going to jump right to there. So here is the overall as of today right I wanted to be nice so we have 100% for completeness right this is the minimum stuff you have to have to publish a CVE right so everybody has published is great right root cause awareness it was better than I thought 87.4% 4% of all CVEes published has have a C.WE. Only 2% of all CVE published in the last

6 months have a CPE. If you need that data to figure out which vulnerabilities on your network, that number should scare you. Um 88% have CVSS. Super great. Um that one's pretty wide. So it's from 3 31 and four. I break it down on the site a little more in depth if you want to look on who's doing what. Um, and the last one is only about 5% of CVEes have an easy way to tell where the patch is for the vulnerability. That that should be be easy table stakes for people is to say here's exactly where I go to get this patch. Here's what a little breakdown looks like. Octa who is leading today. Um

they have you can tell they do all the foundation awareness, all the root quality, all the CPEs. They do sorry they don't do any CPEs but they get all the other four. So they get a 90% overall score. So that's great. So you can go to the website right now, type in which CNA you want to look up and it'll tell you here's how they're performing. And well, this is interesting. You guys are all consumers in here mostly and you buy stuff from these companies. If I typed in somebody's name here and they weren't in the top 10% and they were asking me for, you know, a renewal of $150,000 or whatever, along with my request for I

need this feature, I should also have this request like, hey, I need this, right? Like I would love to have better CVE records. That's part of the product you guys pay for when you buy software is the vulnerability records. It's all part of the life cycle. And I need more people to help me hold CNAs, not accountable because that's the wrong word, but to let them know that we're looking at the data they're providing and that while in the past the CVE program has been optional and you didn't need to publish stuff and like it was just people doing it in their spare time, in 2025 we're beyond that and this data is serious. It helps protect our

companies. It helps protect our infrastructure and it helps protect our nation. Um, outside of that, um, that is about it, I would love for you guys to look at the CNA scorecard and take a look and check it out and let me know what you guys think. It's open source and it's open source on purpose. I would love for issues. If you think that I'm wrong, put it in the issues and we can have a community discussion about it. Uh, if you need find a bug, let me know. I'll try to fix it, right? Like I'm trying to build this stuff in open publicly so people can see the data and see where it's coming from and help improve it. I

I don't want spoken mirrors. I I just want to fix this problem for everyone. And with that, I think we have time for maybe one question.

Uh earlier you talked also about the EU cyber resilience act and how that would impact uh the number of CVS being released. Um in terms of uh quality of the data as well because there's now a mandate for that how many new vendors will come into this uh new area and just put out information with the minimum requirements and let's let the rest sort itself out. Yeah, I that that's a very important question and and something that we have to figure out, right, with the EU's new regulations on publishing. If the publishing bar is so low that that basically, you know, a simple sentence and a link will get you published, that's all that they have to to provide. It's going to

be it's going to have both teams are going to have to work together, right? the EU legislature gonna have to say, "Hey, you're going to need these more informations and the CVE program needs to raise the bar about where publication is, right? Like especially when you get to that point where you have 80 plus% of CNAs providing a a field that should be an easy field to switch to to mandated, right? Like you've already you've already made critical NAS there and you just flip it over. So, I think that I think that there's going to be a lot of maturing in the vulnerability database space in the next five years. Um, I'm giving another talk at EPSC Village at

Defcon on Friday afternoon at 1 that really dives into that. If you're interested, I would love to see you guys there. So, with that, thank you guys very much for your time and [applause] I'll let the next person have the room.