Unable to use custom doclet in sbt doc build

[ikaakkola]

Setting a custom '-doclet' via sbt Compile / doc / javacOptions does nothing.

steps

1. Download sbt-javadoc-doclet.tar.gz.
2. Extract the archive somewhere
3. Run sbt clean compile doc

problems

javadoc uses StandardDoclet instead of provided '-doclet' class.

expectation

As the example project uses an invalid doclet class ("oops"), in this case a failure from the javadoc tool is expected, something like:

user@computer sbt-javadoc-doclet % javadoc -doclet oops
error: Cannot find doclet class oops

If one were to use a real doclet implementation, the expectation is that javadoc is executed using said doclet class.

notes

zinc 1.4.0 introduced "standardDoclet":

zinc/internal/zinc-compile-core/src/main/scala/sbt/internal/inc/javac/LocalJava.scala

Line 61 in d86bc5a

private[this] val standardDoclet = "jdk.javadoc.doclet.StandardDoclet"

which makes it impossible to use a custom doclet, as the jdk implementation prefers the given Doclet class:

zinc/internal/zinc-compile-core/src/main/scala/sbt/internal/inc/javac/LocalJava.scala

Line 109 in d86bc5a

clz,

over any user set doclet ("-doclet" option) (see JDK references below).

It is unclear to me why standardDoclet was introduced in #712 , to me it looks like the JDK will always fall back to StandardDoclet anyway.

Tested with both JDK11 and JDK17, which both fail in the same way (despite having different internal implementations on this code path)

The example project fails as expected when using sbt < 1.40 (1.3.13 for example)

[info] Main Java API documentation to /tmp/sbt-javadoc/target/scala-2.13/api...
[warn] Unexpected javac output: javadoc: error - Cannot find doclet class oops.
[warn] javadoc exited with exit code 2
[error] sbt.inc.Doc$JavadocGenerationFailed
[error] 	at sbt.inc.Doc$$anon$1.run(Doc.scala:72)
[error] 	at sbt.inc.Doc$$anon$2.run(Doc.scala:91)
[error] 	at sbt.inc.Doc$$anon$3.$anonfun$run$5(Doc.scala:118)
[error] 	at sbt.inc.Doc$$anon$3.$anonfun$run$5$adapted(Doc.scala:114)
[error] 	at sbt.util.Tracked$.$anonfun$inputChanged$1(Tracked.scala:150)
[error] 	at sbt.inc.Doc$$anon$3.run(Doc.scala:122)
[error] 	at sbt.Defaults$.$anonfun$docTaskSettings$3(Defaults.scala:1646)
[error] 	at scala.Function1.$anonfun$compose$1(Function1.scala:49)
[error] 	at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:62)
[error] 	at sbt.std.Transform$$anon$4.work(Transform.scala:67)
[error] 	at sbt.Execute.$anonfun$submit$2(Execute.scala:281)
[error] 	at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:19)
[error] 	at sbt.Execute.work(Execute.scala:290)
[error] 	at sbt.Execute.$anonfun$submit$1(Execute.scala:281)
[error] 	at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:178)
[error] 	at sbt.CompletionService$$anon$2.call(CompletionService.scala:37)
[error] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
[error] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[error] 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[error] 	at java.base/java.lang.Thread.run(Thread.java:829)
[error] (Compile / doc) sbt.inc.Doc$JavadocGenerationFailed

JDK source references:

JDK 11:
confirmed with debugger that https://github.com/openjdk/jdk/blob/jdk-11+28/src/jdk.javadoc/share/classes/jdk/javadoc/internal/tool/Start.java#L351-L361 is what ends up being called, which sets this.docletClass to the given docletClass. Later, when the doclet is chosen, 'docletClass' is always used: https://github.com/openjdk/jdk/blob/jdk-11+28/src/jdk.javadoc/share/classes/jdk/javadoc/internal/tool/Start.java#L719-L721

JDK 17:
confirmed with debugger that https://github.com/openjdk/jdk/blob/jdk-17+35/src/jdk.javadoc/share/classes/jdk/javadoc/internal/tool/Start.java#L345-L356 is what ends up being called, which sets this.docletClass to the given docletClass. Later, when the doclet is chosen, 'docletClass' is always used:
https://github.com/openjdk/jdk/blob/dfacda488bfbe2e11e8d607a6d08527710286982/src/jdk.javadoc/share/classes/jdk/javadoc/internal/tool/Start.java#L716