The bogus CVE problem
Please consider subscribing to LWNSubscriptions are the lifeblood of LWN.net. If you appreciate this content and would like to see more of it, your subscription will help to ensure that LWN continues to thrive. Please visit this page to join up and keep LWN on the net.
The "Common Vulnerabilities and Exposures" (CVE) system was launched late in the previous century (September 1999) to track vulnerabilities in software. Over the years since, it has had a somewhat checkered reputation, along with some some attempts to replace it, but CVE numbers are still the only effective way to track vulnerabilities. While that can certainly be useful, the CVE-assignment (and severity scoring) process is not without its problems. The prominence of CVE numbers, and the consequent increase in "reputation" for a reporter, have combined to create a system that can be—and is—actively gamed. Meanwhile, the organizations that oversee the system are ultimately not doing a particularly stellar job.
A recent incident highlights some of the problems inherent in the system. CVE-2020-19909, which is an integer-overflow bug in the curl tool and library for URL-based data transfers that was only reported to the project in 2023. In a blog post describing the mess, curl maintainer Daniel Stenberg said that a message to the curl-library mailing list on August 25 alerted the project that the CVE had become public the week before.
The year in the CVE number (2020 in this case) is meant to indicate when the bug was reported to one of the more than 300 CVE numbering authorities (CNAs) that hand out CVE numbers. Under normal circumstances, a new bug showing up with a CVE number would have 2023 in it, but sometimes CVEs are given out for older bugs that somehow slipped through the cracks. That appears to be what happened in this case, as Stenberg was able to track the problem back to a bug report from Jason Lee in mid-2019.
The report was for a legitimate bug, where the
‑‑retry‑delay option value was being multiplied
by 1000 (to milliseconds) without an overflow check. But what it was
not was a security
bug, Stenberg said; giving insanely large values for the
option might result in incorrect
delays—far shorter than requested—but it is not a security problem to make
multiple requests in a short time span. If it were, "then a browser
makes a DOS [denial of service] every time you visit a website — and curl
does it when you give it two URLs on the same command line
", he said in
a followup
post.
The problem was duly fixed, a test case was added, and Lee was credited with the report in the commit message. In September 2019, curl 7.66.0 was released with fix, which was mentioned in the announcement; also notable are the two CVEs mentioned at the top of the bug fixes listed. As Stenberg noted, the curl project works hard to ensure that it fully documents the (real) CVEs that get filed for it; his exasperation with CVE-2020-19909 coming out of the blue is evident:
In the curl project we work hard and fierce on security and we always work with security researchers who report problems. We file our own CVEs, we document them and we make sure to tell the world about them. We list over 140 of them with every imaginable detail about them provided. We aim at providing gold-level documentation for everything and that includes our past security vulnerabilities.That someone else suddenly has submitted a CVE for curl is a surprise. We have not been told about this and we would really have liked to. [...]
The National Vulnerability Database (NVD) tracks CVEs and "scores" them using the Common Vulnerability Scoring System (CVSS), which is a ten-point scale that is meant to give an indication of the severity of a vulnerability. For the curl bug, which should probably not be scored at all, NVD initially came up with a "9.8 critical", scoring an integer overflow in a delay parameter as one of the most severe types of vulnerability possible. Stenberg, who has tangled with NVD over scoring before, is even further exasperated:
It was obvious already before that NVD really does not try very hard to actually understand or figure out the problem they grade. In this case it is quite impossible for me to understand how they could come up with this severity level. It's like they saw "integer overflow" and figure that wow, yeah that is the most horrible flaw we can imagine, but clearly nobody at NVD engaged their brains nor looked at the "vulnerable" code or the patch that fixed the bug. Anyone that looks can see that this is not a security problem.
In fact, the pull request for the fix was attached to the report, but that apparently made little difference in the assessment from NVD. Back in March, Stenberg decried the NVD scoring process and, in particular, the NVD practice of re-scoring CVEs that have already had severity levels attached to them. NVD uses CVSS, but the curl project long ago rejected that scoring system:
In the curl project we decided to abandon CVSS years ago because of its inherent problems. Instead we use only the four severity names: Low, Medium, High, and Critical and we work out the severity together in the curl security team as we work on the vulnerability. We make sure we understand the problem, the risks, its prevalence and more. We take all factors into account and then we set a severity level we think helps the world understand it.
His example in that case is a double-free in curl that the project determined had a "medium" severity, while NVD scored it as "9.8 critical", as can be seen in the GitHub advisory database. Since then, NVD apparently had a change of heart after Stenberg's complaint as the CVE is now scored "8.1 high". In a followup on NVD "brokenness", Stenberg gave another example of a CVE that was initially scored "9.8 critical", but eventually was reduced to "5.9 medium" after complaints—though the curl project rates it as "low". He also noted that there is a set of projects that never report low or medium CVEs that they find, in order to avoid these kinds of scoring woes.
One could perhaps wonder if this is all just a problem for the curl project and not more widespread, but there is a fair amount of evidence of a variety of problems in CVE-land. For example, the PostgreSQL project had a similar problem with a CVE "from" 2020 that appeared recently—and is not a security vulnerability at all, according to the project. In June, the KeePassXC password manager project had a bogus CVE filed for the tool; there are other examples as well.
Each of these bogus CVE filings, which can apparently be made anonymously and without much in the way of backing information, require that the project notice its existence, analyze the problem (or "problem"), and, if necessary, dispute the existence or score of the CVE. As noted, several curl CVEs have had their scores reduced rather substantially due to requests from the project. The delay parameter overflow that was initially scored 9.8 has been reduced to "3.3 low", marked as "disputed", and had a link to Stenberg's blog post added to the NVD entry.
Keeping up with all of that is a lot of work, which Stenberg said he is going to try to avoid in the future by applying to become the CNA for curl. Several other open-source projects are CNAs, which gives them some notification of a reported problem along with ways to try to ensure that the problem is handled sanely. He mentioned Apache, OpenSSL, and Python as some of the projects that are already CNAs; Python was just granted CNA status at the end of August.
Meanwhile, though, CVEs are used in ways that elevate their importance well beyond the level that makes sense given the amount of scrutiny that is apparently applied to them. Service-level contracts and governmental requirements mean that a critical CVE needs to be addressed in short order, so non-critical bugs that get marked that way can cause real problems. It does provide incentives for companies and others to try to downplay the severity of bugs, as well, of course, which makes for something of a "Pushmi-Pullyu" in CVE-land.
As was alluded to in our mid-August look at kernel security reporting for distributions, the CVE system is generally included in the "security circus" that kernel developers largely disdain. A 2019 talk by Greg Kroah-Hartman described multiple problems that he sees with the system as well.
All in all, the CVE system seems to be broken in various ways. It also seems to be getting more and more entrenched into "cybersecurity" handling at various levels. Given that it is effectively run by—and now for—governmental agencies, the ability to replace it with something more sensible has likely already passed us by. CVE, warts and all, will be with us for a long time to come; FOSS projects and organizations are simply going to have to figure out how to coexist with it.
| Index entries for this article | |
|---|---|
| Security | Bug reporting/CVE |
Posted Sep 13, 2023 21:10 UTC (Wed)
by geofft (subscriber, #59789)
[Link] (6 responses)
I actually think that this is, essentially, a DoS vulnerability in the CVE process itself. Anyone can submit an entry into a database that gets insufficiently validated and causes other people to have to scramble and respond and maybe makes their automated systems break. That's unauthenticated input causing denial-of-service attacks!
Also it occurs to me that CVE + CVSS actually does kind of provide a solution here. The point of CVE is that it's common, i.e., if someone else discovers the same vulnerability, there's a way to ideally give them the same CVE identifier or at least mark their new one as a duplicate. It feels like we should be able to proactively mark things as CVSS score 0, and say, yes, this is an integer overflow or stack corruption or whatever, but exploiting it rather involves being on the other side of this airtight hatchway, and so we're going to mark the fact that it exists so people who find it know it was already analyzed. And in some cases it makes sense to "fix" it, as with CVE-2020-19909. It's always better to print a real error message instead of "double free or corruption (fasttop)". (But I do wonder about the incentive system, as Woodruff mentions, of having CVEs on your résumé, and whether the existence of CVEs with CVSS scores of 0 will make things better or worse.)
Posted Sep 14, 2023 0:04 UTC (Thu)
by wahern (subscriber, #37304)
[Link] (5 responses)
The CVE system has flaws--serious flaws. But I'd argue that these flaws are less severe than the alternative where vendors can more easily waive off reports. Let's not forget from whence we came--a time when vendors didn't take these things seriously unless and until bugs were already being conspicuously exploited in the wild. And we still have problems with too many vendors not taking CVEs seriously. The current system is unfair to conscientious developers and maintainers whose time is wasted by self-aggrandizing bug reporters, but we would never have needed the current CVE system if such conscientious people were the majority of those shipping software.
* There's a reason we can't have nice things. * No good deed goes unpunished. * Etc, etc.
Posted Sep 14, 2023 7:14 UTC (Thu)
by madhatter (subscriber, #4665)
[Link]
> Specifically on the researchers’ claims of a backdoor in TEA1, Boyer added “At this time, we would like to
And I would like to point out that that’s the very definition of a backdoor.
Posted Sep 15, 2023 3:01 UTC (Fri)
by florianfainelli (subscriber, #61952)
[Link] (3 responses)
- EU CRA: https://www.european-cyber-resilience-act.com/
this will no doubt create an immense amount of work for projects, distributors, and so on, almost as if it was a jobs program.
Posted Sep 15, 2023 11:42 UTC (Fri)
by wtarreau (subscriber, #51152)
[Link] (1 responses)
... or this garbage will convince many of us that too much is too much and it's time to switch jobs, maybe become a european deputee of some other such dummy jobs where you can vote for random rules that morons will blindly apply. I mean, there's no reason we should be their slaves, we're still free to stop maintaining projects that take too much time under such stupid rules, and let the internet fall apart as well.
Posted Sep 15, 2023 13:17 UTC (Fri)
by pizza (subscriber, #46)
[Link]
This is the most likely outcome.
> we're still free to stop maintaining projects that take too much time under such stupid rules, and let the internet fall apart as well.
Ah yes, but if you stop maintaining it, you're engaging in willful gross negligence and will be on the hook for $very_massive liabilities. And that's where things are heading.
Damned if you do, damned if you don't.
Posted Sep 22, 2023 6:46 UTC (Fri)
by kunitz (subscriber, #3965)
[Link]
Since there are so many software packages, nobody has the time to look at the detail of a CVE. If there is a CVE which is high or critical you have to make it disappear, regardless whether you actually use the functionality or expose the service in any way. The financial industry has the audit and governance structures to enforce such rules.
For me it looks like that dependency reduction will become very important in the industry. How open-source will deal with the requirements of the CRA will be interesting to see.
Posted Sep 13, 2023 21:55 UTC (Wed)
by flussence (guest, #85566)
[Link] (4 responses)
If NVD/MITRE don't massively clean up their act following this, the next step is to start removing baked-in recognition of CVE numbers in the many tools that have it, and educating the general public that those organisations possess no legitimate authority. IMO this part of the industry is at least a decade overdue for a scorched-earth cleanup like what Let's Encrypt did to the CA aristocracy.
Posted Sep 14, 2023 12:03 UTC (Thu)
by hkario (subscriber, #94864)
[Link]
Posted Sep 14, 2023 17:50 UTC (Thu)
by mcatanzaro (subscriber, #93033)
[Link] (2 responses)
Most CVEs are valid. But invalid CVEs are certainly annoying. Another common problem is multiple CVEs being reported for the same issue. Recently somebody found one bug in WebKit and managed to get 6 different CVEs for it by asking MITRE instead of Apple to assign them. It was a serious vulnerability and the reporter otherwise did a good job, but 6 CVEs was unimpressive. (I think I requested these be marked as duplicate CVEs, but I don't think it actually happened.)
What's especially weird is that sometimes MITRE sides with the reporter rather than the project developer even when there is no dispute. E.g. a few years ago, somebody received two CVEs for what turned out to be one bug (I think it was also WebKit bug?) and when I requested that they be combined, MITRE decided that two CVEs was correct because there were two different test cases that trigger the bug. This was mind boggling to me.
Posted Sep 18, 2023 11:20 UTC (Mon)
by cpitrat (subscriber, #116459)
[Link]
Posted Sep 18, 2023 12:14 UTC (Mon)
by wtarreau (subscriber, #51152)
[Link]
Some ex-insiders such as Kurt Seifried and Josh Bressers in a recent talk disagreed with this suggesting that around 20% only were valid. See the link to their talk below in another comment.
Posted Sep 14, 2023 7:03 UTC (Thu)
by Trou.fr (subscriber, #26289)
[Link]
Posted Sep 14, 2023 7:25 UTC (Thu)
by wtarreau (subscriber, #51152)
[Link] (4 responses)
But yeah the process is totally broken and non-transparent. Other projects such as SQLite don't want to hear about CVE anymore for these reasons. In HAProxy we don't file them any more after an embargoed CVE was twitted with only the subject, but enough details to hurry up the process. Instead we now let distro package maintainers decide if it makes their life easier with or without one and leave it to them to file these. When they do, they fill them correctly because they're more accustomed to the process. In the end it's a clean and efficient sharing of the roles so it's not that bad.
One thing I've been tempted to do many times is to file one CVE per backported patch, at least to ensure that all backports land into distros, and not have to respond to questions about CVEs anymore. In our case it would cause a real burden to our package maintainers and since they're doing a great job I don't want to do that. It would have been different otherwise :-)
I tried to educate coworkers and other people about focusing on "bug fixes" and not "CVE" for backports, but it doesn't seem to take off. There's so heavy an industry hammering "CVE" all the day that they manage to make anyone believe this has anything to do with security, while it's just a business consisting in selling lists and automated tools checking for the presence of a tiny portion of bugs affecting software, and this business is so lucrative that it's almost hopeless to be able to open other people's eyes about what it really is. It's particularly annoying because real bugs are rarely fixed and CVE make lots of noise.
Posted Sep 14, 2023 9:37 UTC (Thu)
by ringerc (subscriber, #3071)
[Link] (3 responses)
They've decided it's too hard to classify the balance of supposed vulnerabilities, so they're just going to take the severities as fact, irrespective of where the component lives in our stack.
"Critical" vuln in a completely unrelated part of a golang executable that happens to share the same huge monorepo as a library we use? Everybody panic, rush to upgrade everything to the latest no matter what the breakage, even though we probably don't use it. Used by a 3rd pty component that didn't jump fast enough? Fork it and hack it.
Then we have all these mini forks of various 3rd party components sitting around rotting. Or they decide it's too hard to keep up with "security" for a perfectly good external component and rewrite it in-house, badly. No more CVEs because nobody's looking anymore! It's less secure, but we made the squeaky wheel go away. Go us.
They've also rolled out various mandatory and enforced static checkers and linters with centrally managed one-size-fits-all configs. The stupid things run on pull requests and they don't compare the pull request results to a scan of main with the same config and db version. So whenever the vendor or info-"security" management rolls out new config/dbs, totally unrelated changes become unmergeable. Sure you could raise a new separate PR to patch main then rebase your change. But you've got a gauntlet of Jira ticket workflow requirements, encorced pull request reviews from people who're often unavailable, slow unreliable GitHub workflows, etc. Who's got time for that? So we have PRs "fix off by one in A" that have 200 lines of completely irrelevant noise "fixing" meaningless non issues. They obscure the real changes and introduce new bugs.
Yay management of developers by non developers, and management of IT security by people who couldn't explain the difference between authentication and authorisation if you gave them a textbook first.
Posted Sep 14, 2023 15:39 UTC (Thu)
by wtarreau (subscriber, #51152)
[Link] (2 responses)
Posted Sep 20, 2023 14:21 UTC (Wed)
by bearstech (subscriber, #160755)
[Link] (1 responses)
The main flow of alerts/requests I've been handling for close to a year is composed of ... requests to add anti-XSS HTTP headers. By far. At the same time I got _zero_ status requests on the Zenbleed/Inception mayhem from this summer. The cultural gap is that wide.
In my case the problem is clearly companies who want to adopt security policies but are no trained on computer security. I can't blame them except their CTO which most of the time is totally missing the point while thinking that security is just another component that need dashboards and reports, and done.
CVE ranking is conceptually wrong, a security issue must be evaluated with context and is multi-faceted, there should never be a single figure in the first place. When talking with a dev or project manager I can educate about those issues and explain them how to handle a specific security issue in context, that's often satisfying for both parties. But then it won't bubble up to the top management and against the solidified bureaucracy they call security policy.
Posted Sep 29, 2023 4:01 UTC (Fri)
by wtarreau (subscriber, #51152)
[Link]
Posted Sep 14, 2023 11:42 UTC (Thu)
by smoogen (subscriber, #97)
[Link]
Some of the problems with CVE's is that the system was designed to replace at least 2 other systems which fell apart in the late 1990's where most software was closed source and unless you could prove undeniably that there was a true vulnerability (and could not be hand waived off with "you didn't do exactly what we said in this manual on Alpha Centauri protected by a cyberraptor ") it wasn't looked and or fixed.
Some of the problems is that everyone has a tendency to make anything they run into a game they can 'win' at. And when they can't win, look for ways to avoid playing anymore. This goes for core programmers, security researchers, software companies, etc. These are known problems which aren't solvable because they are 'human condition' issues (and every solution just becomes a new game to win or lose at).
Most of the issues are that the above are known, but because they are all hard to get N people to agree to even trying a solution, you end up punting the problem to the next year and maybe just add another number to the amount of possible CVE's for a year.
Posted Sep 14, 2023 15:54 UTC (Thu)
by MarcB (subscriber, #101804)
[Link]
One of them might be a real vulnerability, but very likely not a 7.8.
The other three are scored 5.5, but look much more like zeroes. Read buffer-overflows that do not cross any confidentiality or other security boundaries and do not have any way to exfiltrate the data anywhere.
Posted Sep 14, 2023 23:10 UTC (Thu)
by nliadm (subscriber, #94000)
[Link]
I think having a process to triage reported vulnerabilities easily is key to handling this at scale. Frameworks like SSVC are great, but there's an onus on reporting tools to be able to integrate those decisions back in and not show scary red marks to people that just want a "security" toggle.
Posted Sep 15, 2023 1:22 UTC (Fri)
by amworsley (subscriber, #82049)
[Link] (1 responses)
Posted Sep 15, 2023 10:01 UTC (Fri)
by farnz (subscriber, #17727)
[Link]
That sort of thing is already captured by the "Exploit Code Maturity (E)" component of the CVSS score; if the overall score was capped to 4 if E:U or E:X (i.e. no PoC available now), then people would focus on that.
Organisations that consume the CVSS metrics can do that today, of course.
Posted Sep 15, 2023 4:29 UTC (Fri)
by buck (subscriber, #55985)
[Link]
After all, remember the first of the 3 hardest problems in computer science.
As the key to databases (NVD, the CVE-named errata pages of RedHat, Debian security-tracker, etc.) with links to advisories and/or references, it's kinda useful to have them be content-addressable.
As long as one can ignore everything else, which I guess it's hard for somebody to do if a CVE is kind of like an unsealed indictment of your software product, ... hmm.
Maybe there needs to be a CVE-rank algorithm to score CVEs: If there are no vendor and/or distro and/or vulnerability-list and/or such links attached to, say, the NVD entry, then it should carry a low score and people should ignore it (should the score stay low). And if the score doesn't respond dynamically enough to reflect some really bad news? Well, if you're looking to track CVSS scores to tell you what advisories to pay attention to in your system, then I got nothing for you. But if you're only responding to some vulnerability scanner telling you which high-CVSS things you need to worry about, then it's the scanner's problem, not yours, and you can maybe go back to figuring out your most significant risks and how to balance them yourself.
Not sure if that would be very open to gaming (CVEO?) but also hard to see the point. I don't flatter myself that I'm very devious, though.
Posted Sep 22, 2023 8:54 UTC (Fri)
by j16sdiz (guest, #57302)
[Link]
Is there anything special about this "double-free" making this just a medium?
Depends on the malloc implementation, double-free can corrupt data. In a library like curl, this can have very bad consequence.. A "high" score is well justified.
Posted Sep 22, 2023 17:06 UTC (Fri)
by fung1 (subscriber, #144307)
[Link]
We also do not bother to dispute CVE assignments others request for bugs in our software, because it's not worth our (limited) time to do so. It's our policy that if someone has requested a CVE for one of our bugs and notified us what number got assigned, then we'll reuse that *if* we issue an advisory (instead of requesting a conflicting CVE). It's been clear for a very long time that since anyone can request and receive a CVE assignment without even talking to the people maintaining the project it's about, the mere existence of a CVE does not imply there's any sort of vulnerability, nor is there any point in wasting time trying to get them rescinded.
Posted Sep 23, 2023 7:31 UTC (Sat)
by domdfcoding (guest, #159754)
[Link] (1 responses)
Posted Sep 23, 2023 21:16 UTC (Sat)
by mathstuf (subscriber, #69389)
[Link]
William Woodruff's post from last year ReDoS "vulnerabilities" and misaligned incentives is another good look at this problem focusing particularly on people's excitement with superlinear regular expressions, and points out a very practical issue: a lot of automated security tools will report issues if any of the dependencies you're using have unresolved CVEs. This is well-intentioned, but for things that aren't actually vulnerabilities, it leads to needless churn and panic, and for things that are so much not a vulnerability that they aren't fixed, or aren't fixed except in the current development release, or similar, it may even cause deploy tools to refuse to deploy. Or, worse, it may cause well-funded corporate users to make demands of open-source projects to backport a patch that has no need to be backported. When there's an actual bug to fix, we can just lament the open-source sustainability problem, but when there isn't, it's nothing other than rude.
The bogus CVE problem
The bogus CVE problem
You do make a depressingly good point about software makers, particularly commercial ones, being incentivised to handwave vulnerabilities away. Bruce Schenier wrote this pithy response in last month's Crypto-Gram newsletter, in response to a vendor quote in a vice.com article about a vulnerability found in Tetra police radios:
The bogus CVE problem
> point out that the research findings do not relate to any backdoors. The TETRA security standards have
> been specified together with national security agencies and are designed for and subject to export
> control regulations which determine the strength of the encryption.”
The bogus CVE problem
- US EO 14028: https://www.nist.gov/itl/executive-order-14028-improving-...
The bogus CVE problem
The bogus CVE problem
The bogus CVE problem
The bogus CVE problem
The bogus CVE problem
The bogus CVE problem
The bogus CVE problem
The bogus CVE problem
The bogus CVE problem
AFAIK, nvd is exactly what is says, a database.
That's not to say the CVE/CVSS is broken, but it is still the only tool that allows people and organisations to track which vulnerability they should patch or they have patched.
And BTW the CVSS 4.0 https://www.first.org/cvss/v4-0/ is going to be published soon, it should fix some of the problems of v3 (including the impossibility to reach 10)
The bogus CVE problem
The bogus CVE problem
The bogus CVE problem
The bogus CVE problem
The bogus CVE problem
The bogus CVE problem
The bogus CVE problem
The bogus CVE problem
The bogus CVE problem
e.g. no CVSS 10 (or > 8?) if there is no PoC exploit code. So much time is wasted trying to see
if a vaguely described fault is actually a significant problem. If there is a PoC it really spells out
what is required and what can be done very quickly and efficiently.
The bogus CVE problem
The bogus CVE problem
The bogus CVE problem
The bogus CVE problem
The bogus CVE problem
The bogus CVE problem