Skip to content

Commit

Permalink
docs: add documentation on caching (#6204)
Browse files Browse the repository at this point in the history
  • Loading branch information
jeremylong authored Dec 4, 2023
2 parents 038e353 + 68e959d commit 912f553
Show file tree
Hide file tree
Showing 6 changed files with 61 additions and 10 deletions.
16 changes: 7 additions & 9 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -186,16 +186,14 @@ jobs:
prerelease: false
draft: false
body: |
- fix: check java 8 update version; minimum JRE is 8 update 251 (#6118)
- fix: add retry for failed NVD API requests (#6136)
- docs: add default values to documentation for the NVD API Delay (#6135)
- chore: Revert "build(deps): bump com.h2database:h2 from 2.1.214 to 2.2.224" (#6131)
- this is a **breaking change** for anyone that successfully created the H2 database with 9.0.0.
- fix: mute jcs logging (#6130)
- docs: update NVD notice (#6110)
- fix: Use the correct key for NVD API-Key from Maven Settings serverId (#6109)
- fix: remove virtual match string on NVD API Request (#6177)
- fix: correct meta data in report after switching the NVD API (#6154)
- fix: retry HTTP connections to NVD on 502 and 504 errors (#6151)
- fix: Gitlab report format needs severity capitalized (#6182)
- fix: improve JDK update version parsing (#6163)
- fix: mute JCS logging (again) (#6153)
See the full listing of [changes](https://github.com/jeremylong/DependencyCheck/milestone/71?closed=1).
See the full listing of [changes](https://github.com/jeremylong/DependencyCheck/milestone/72?closed=1).
- name: Upload CLI
id: upload-release-cli
Expand Down
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,17 @@
# Change Log


## [Version 9.0.2](https://github.com/jereong/DependencyCheck/releases/tag/v9.0.2) (2023-12-01)

- fix: remove virtual match string on NVD API Request (#6177)
- fix: correct meta data in report after switching the NVD API (#6154)
- fix: retry HTTP connections to NVD on 502 and 504 errors (#6151)
- fix: Gitlab report format needs severity capitalized (#6182)
- fix: improve JDK update version parsing (#6163)
- fix: mute JCS logging (again) (#6153)

See the full listing of [changes](https://github.com/jeremylong/DependencyCheck/milestone/72?closed=1).

## [Version 9.0.1](https://github.com/jereong/DependencyCheck/releases/tag/v9.0.1) (2023-11-26)

**breaking changes**: See the [upgrade notice](https://github.com/jeremylong/DependencyCheck#900-upgrade-notice)
Expand Down
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,13 @@ Without an NVD API Key dependency-check's updates will be **extremely slow**.
Please see the documentation for the cli, maven, gradle, or ant integrations on
how to set the NVD API key.

#### The NVD API Key, CI, and Rate Limiting

The NVD API has enforced rate limits. If you are using a single API KEY and
multiple builds occur you could hit the rate limit and receive 403 errors. In
a CI environment one must use a caching strategy.


### Breaking Changes

9.0.0 contains breaking changes which requires updates to the database. If using
Expand Down
32 changes: 32 additions & 0 deletions src/site/markdown/data/cacheh2.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
Caching ODC's H2 Database
=========================================

Many users of dependency-check ensure that ODC runs as fast as possible by caching
the `data` director (or in some cases just the H2 database). Where the `data`
directory exists is different for each integration (cli, maven, gradle, etc.).
However, each integration allows users to configure the location of the data directory.

Within the data directory there is a cache directory that contains temporary caches
of data requested that is not stored in the database and is generally build specific
- but can be re-used. There are two primary stratigies used:

1. Cache the H2 database

Use a single node to build the database using the integration in update only mode
(e.g., `--updateOnly` for the cli) and specify the data directory location (see
the configuration documentation for each integrgations configuration). The data
directory is then archived. Subsequent nodes that perform scanning will then
download the archived database and configure the scan to occur and in general,
the node would be configured with `--noupdate` (or the releated configuration to
disable the updates in each configuration). The database is generally updated daily
in this use case - but could be designed with a more frequent update.

2. Cache the H2 database and the cache

Some users have a slightly modified version of the above caching strategy. Instead
of only having a single update node - they allow all nodes to update. However,
the data directory is zipped and stored in an common location. Each node will execute
a scan (with updates enabled) and if succesful the updated data directory is zipped
and uploaded to the common location. This has the small advantage of being updated
faster and will store the cache between executions which can improve the performance
on some builds.
3 changes: 2 additions & 1 deletion src/site/markdown/data/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@ have a few options:

1. Configure the [proxy settings](proxy.html) so that the build server can access the Internet
2. [Mirror the NVD](./mirrornvd.html) locally within your organization
3. Use a more robust [centralized database](./database.html) with a single update node
3. Build the H2 database on one node and [cache the H2 database](./cachenvd.md).
4. Use a more robust [centralized database](./database.html) with a single update node

## CISA Known Exploited Vulnerabilities

Expand Down
1 change: 1 addition & 0 deletions src/site/site.xml
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,7 @@ Copyright (c) 2013 Jeremy Long. All Rights Reserved.
<item name="Proxy" href="./data/proxy.html" />
<item name="Mirroring NVD" href="./data/mirrornvd.html" />
<item name="Snapshotting the NVD" href="./data/cachenvd.html" />
<item name="Caching the Database" href="./data/cacheh2.html" />
<item name="Central DB" href="./data/database.html" />
<item name="Sonatype OSS Index" href="./data/ossindex.html" />
</item>
Expand Down

0 comments on commit 912f553

Please sign in to comment.