This example shows you how to:
- Install the
eslint
analysis tool. - Configure the driver for
eslint
. - Run
eslint
on the source code of selected CVEs usingbin/cli run
. - View a benchmark report for the analysis run using
bin/cli report
.
Before working through this example, you must install the CVE Benchmark tooling. For more information, see Using the CVE Benchmark.
You must install each analysis tool that you want to benchmark with bin/cli run
.
Some drivers are provided with installer scripts that install their backing analysis tool,
while other drivers require manual installation steps. When adding support for a new
analysis tool, we encourage you to provide an installer.
To see which tools are available, and which have installers,
run bin/cli tools
:
$ bin/cli tools
Configured tools in /home/user-name/ossf-cve-benchmarking/config.json:
...
Available driver installers:
- ...
- contrib/tools/eslint/installers/install.cmd
- contrib/tools/eslint/installers/install.sh
- ...
Available driver READMEs:
- ...
- contrib/tools/eslint/README.md
- ...
Configured:
...
The output shows that eslint
is available with an installer.
Run the installer, specifying a directory to install eslint
in:
$ contrib/tools/eslint/installers/install.sh /home/user-name/analysis-tools/eslint-2020-12-08
The eslint tool has been installed. Add the fragment below to a config.json file:
{
...
"tools": {
...
"eslint-default": {
"bin": "node",
"args": [
"/home/user-name/ossf-cve-benchmark/build/ts/contrib/tools/eslint/src/eslint.js",
],
"options": {
"eslintDir": "/home/user-name/analysis-tools/eslint-2020-12-08"
}
}
...
}
...
}
eslint
has now been successfully installed.
bin/cli
does not interact directly with eslint
, but rather through
a driver that executes eslint
on selected code and converts the results
to a form that benchmark reports can be generated from. So, to complete the setup,
you must also configure the driver.
To configure the driver for an analysis tool, you need to add
a tools
entry to your local
config.json
file.
For eslint
, the installer script displays the appropriate
snippet at the end of its output:
{
"tools": {
"eslint-default": {
"bin": "node",
"args": [
"/home/user-name/ossf-cve-benchmark/build/ts/contrib/tools/eslint/src/eslint.js",
]
}
"options": {
"eslintDir": "/home/user-name/analysis-tools/eslint-2020-12-08"
}
}
}
This snippet includes the identifier eslint-default
that you need
to specify with the --tool
option on the command line.
After updating your config.json
file with this snippet, check
that eslint
has been successfully
configured by running bin/cli tools
again:
$ bin/cli tools
Configured:
- eslint-default
...
The identifier eslint-default
is now listed as configured
for use with bin/cli
.
After completing the steps above, you can use
bin/cli run
to run an analysis with eslint
over
one or more benchmark CVEs. In this example we'll select two CVEs,
CVE-2018-16492 and CVE-2020-4066.
To run the analysis with eslint
, you must specify the name of
the identifier used in your config.json
file, eslint-default
,
with the --tool
option:
$ bin/cli run --tool eslint-default CVE-2018-16492 CVE-2020-4066
...
For more information about specifying data to analyze with your tool, see Selecting CVEs of interest.
After running the command, results files are generated in
JSON format. By default, the files are written to disk
in work/results/eslint-default_....json
.
Now that you have data for some runs, you can
view a benchmark report using the bin/cli report
command.
To start a report server that you can view in your browser, run the following:
$ bin/cli report --kind server --tool eslint-default CVE-2018-16492 CVE-2020-4066
Navigate to http://localhost:8080 in your browser to view the report.