END-OF-LIFE NOTICE

On October 3, 2022, the NVD announced that they will be launching v2 of their APIs. When launched, the v1 API along with the data feeds that are downloaded and used by tools such as Dependency-Check, will be deprecated. Since the announcement, the v2 APIs have been launched indicating that the NVD will officially retire the data feeds in 12 months, or in Q4 2023.

Due to this announcement, this project is now archived. No future work is planned.

---

NIST Data Mirror

A simple Java command-line utility to mirror the NVD (CPE/CVE JSON) data from NIST.

The intended purpose of nist-data-mirror is to be able to replicate the NIST vulnerabiity data inside a company firewall so that local (faster) access to NIST data can be achieved.

nist-data-mirror does not rely on any third-party dependencies, only the Java SE core libraries. It can be used in combination with [OWASP Dependency-Check] in order to provide Dependency-Check a mirrored copy of NIST data.

For best results, use nist-data-mirror with cron or another scheduler to keep the mirrored data fresh.

Usage

Building

mvn clean package

Running

java -jar nist-data-mirror.jar <mirror-directory>

To use a proxy provide http.proxyHost / http.proxyPort system properties.

Downloading

If you do not wish to download sources and compile yourself, [pre-compiled binaries] are available for use. NIST Data Mirror is also available on the Maven Central Repository.

<dependency>
    <groupId>us.springett</groupId>
    <artifactId>nist-data-mirror</artifactId>
    <version>1.6.0</version>
</dependency>

Docker

A Dockerfile is included, and the image is available on Docker Hub as sspringett/nvdmirror. This was created to assist in debugging other issues. While the image does create an httpd instance that mirrors the NVD CVE data feeds - note that it also creates a backup for all changed files and there is currently no automatic cleanup.

$ mvn clean package
$ docker build --rm -t sspringett/nvdmirror .
$ mkdir target/docs
$ docker run -dit \
  --name mirror \
  -p 80:80 \
  --mount type=bind,source="$(pwd)"/target/docs/,target=/usr/local/apache2/htdocs \
  sspringett/nvdmirror

The httpd server will take a minute to spin up as it is mirroring the initial NVD files.

To use a proxy during build time provide the http_proxy, https_proxy and no_proxy environment variables as build arguments (e.g. --build-arg http_proxy="${http_proxy}". For the runtime you can pass the http.proxyHost and http.proxyPort values in _JAVA_OPTIONS.

For example.

_JAVA_OPTIONS="-Dhttps.proxyHost=yourproxyhost.domain -Dhttps.proxyPort=3128 -Dhttp.proxyHost=yourproxyhost.domain
      -Dhttp.proxyPort=3128 -Dhttp.nonProxyHosts="localhost|*.domain"

The image is designed to be executed as a random non-root user and can be deployed on container orchestration platforms such as Kubernetes and OpenShift.

Related Projects

• VulnDB Data Mirror

Copyright & License

nist-data-mirror is Copyright (c) Steve Springett. All Rights Reserved.

Dependency-Check is Copyright (c) Jeremy Long. All Rights Reserved.

Permission to modify and redistribute is granted under the terms of the Apache 2.0 license. See the [LICENSE] [Apache 2.0] file for the full license.

owasp dependency-check apache 2.0 pre-compiled binaries