Get code-coverage of fuzzing process

[centic9]

Is there a way to see code-coverage similar to the report that JaCoCoi tself provides?

It would be useful to see how to improve the fuzzing implementation to visit more code parts.

I tried the coverage functionality of libfuzzer but this is too low-level.

[fmeum]

We just merged a commit that makes Jazzer honor JAVA_OPTS. With this, you can get coverage as follows:

1. Add the relevant Jacoco settings to JAVA_OPTS just as you would with an ordinary application (e.g., use -javaagent pointing to the Jacoco jar).
2. In addition, pass --nohooks to Jazzer. This will disable all instrumentation usually performed by Jazzer.
3. If your fuzzing corpus is in /fuzzing, also pass -runs=0 /fuzzing. These arguments are passed through to libFuzzer and tell it to test every file in that directory once and then exit.

@centic9 Could you give this process a try? If it works for you, we could add it to the README.

[centic9]

Thanks for the hints, the new JAVA_OPTS is actually not even required as the --java_args option already allows to do this.

Only tricky thing is to escape the colon in the -javaagent argument to not have the java_args split by Jazzer.

The following steps did work for me with the current release 0.10.0:

# Fetch JaCoCo Agent
wget --continue https://repo1.maven.org/maven2/org/jacoco/jacoco/0.8.7/jacoco-0.8.7.zip
unzip -o jacoco-0.8.7.zip lib/jacocoagent.jar
mv lib/jacocoagent.jar build/
rmdir lib
mkdir -p build/jacoco

# Run Jazzer with JaCoCo-Agent to produce coverage information
./jazzer \
  --cp=build/libs/fuzz-all.jar \
  --instrumentation_includes=org.example.** \
  --target_class=org.example.Fuzz \
  --nohooks \
  --jvm_args="-javaagent\\:build/jacocoagent.jar=destfile=build/jacoco/corpus.exec" \
  -runs=0 \
  corpus

# now use the Ant/Gradle Task or the CLI to create the report
# see https://www.jacoco.org/jacoco/trunk/doc/

Adding something like this to the README would be nice!