Dan Abramov, a software engineer at Facebook, this week published a plea to silence a particularly vocal JavaScript security tool – and its creators more or less agreed there’s room for improvement. Abramov declared in a blog post:

“As of today, npm audit is a stain on the entire npm ecosystem. The best time to fix it was before rolling it out as a default. The next best time to fix it is now.”

According to Abramov, 99% of the vulnerabilities flagged by the command are false alarms in common usage scenarios. And this appears to be a fairly widespread sentiment among npm users.

More than a decade ago, Isaac Schlueter created the npm package manager and co-founded a company under the same name that would later be absorbed by Microsoft’s GitHub. In April 2018, npm version 6 was released, bringing with it the audit command, because security in the npm ecosystem had become something that could no longer be ignored. JavaScript developers using npm could thereafter type npm audit and they’d receive a security analysis of their projects’ dependency tree.

The problem is that the npm audit was overcorrected. Where a few years ago, JavaScript developers could look forward to being blindsided by security problems, npm runs its audit automatically after every npm install command and often produces a flood of vulnerability advisories that may not be easily fixable and may not really be applicable.

To some extent, the situation is unavoidable given the attack surface in the Node.js ecosystem, where the installation of an average npm package means trusting around 80 other packages due to transitive dependencies. But for Abramov, npm audit produces security warnings in contexts where the risks are not a real concern and the alert overload doesn’t help anyone involved. He wrote:

“The root of the issue is that npm added a default behaviour that, in many situations, leads to a 99+% false positive rate, creates an incredibly confusing first programming experience, makes people fight with security departments, makes maintainers never want to deal with Node.js ecosystem ever again, and at some point will lead to actually bad vulnerabilities slipping in unnoticed.”

Kat Marchán, who helped create npm audit fix and is now a senior software engineer at Microsoft, responded via Twitter, “This isn’t wrong,” while going on to explore some of the tradeoffs involved in security alerts and the decisions that led to the current state of affairs, some of which had to do with NPM’s management and labour challenges in 2018 and 2019. Marchán explained:

“The feature overall, for the company, was kind of a marketing (scare-ish) tactic to promote its private registry service. This isn’t to say it was malicious: there was, and still is, a loud clamour for this kind of security visibility/improvement of the ecosystem.”

Rebecca Turner, also involved in the creation of npm’s auditing feature and now a principal engineer at Microsoft, also responded to Abramov’s broadside, acknowledging that NPM’s need for revenue shaped some design decisions. Turner wrote in a Twitter thread:

“The start of the theatre was the report of the number of vulnerabilities found. It didn’t report the number of vulnerable modules, but the number of things ever depending on the vulnerable module, producing numbers often larger than the number of modules in the tree.”

Turner said she’d have pushed back on that at the time if she’d realized the consequences, but testing didn’t show an excessive number of advisories. Turned also said:

“No further development of this feature happened in the main-line life of npm. Priorities and resources were shifted elsewhere in the push for profitability. Discussion around how to make results more manageable was starting to happen the next year … but between the business pushing to develop those as premium features and the firing of half the CLI team for union organizing  they never went anywhere.”

It’s not quite so simple as union-busting broke npm audit. Crafting security alerts that provide just the right amount of information at just the right time in the appropriate context is a challenge. As Marchan put it,

“I personally spent a long time workshopping the CLI messages around audit to make them as unobtrusive as possible. But sometimes noise is noise, no matter how much you try to reduce it.”

Further code adjustments being considered could improve the situation by providing a manual way to resolve audit warnings, as could Abramov’s call for a way to exclude certain transitive dependencies from generating security warnings.

But calibrating the level of security concern to be appropriate for every individual and situation is a thankless task – dialling back too much on the frantic hand waving and suppressed vulnerability mitigation advice might just lead to the next SolarWinds or Kaseya compromise.