java: Populate $out/lib/jvm

[lorenzleutgeb]

Description of changes

This change should make it easier to detect available JDKs by providing symlinks at the canonical location $out/lib/jvm/$name.

In the Java ecosystem, this is a common pattern. For example, the Gradle build tool supports "crawling" many of those locations:

• /usr/lib{,64}/jvm, {/opt,/usr/{,local/}}java
• $HOME/.jdks
• $HOME/.sdkman/candidates

The Scala tool sbt follows does something very similar and an attempt to integrate with NixOS has stalled.

There's been some discussion about how to make integration of these tools with NixOS easier, see https://discourse.nixos.org/t/4677 and an earlier attempt at #76699

Since the attempts by @raboof, I have made some progress with Gradle in #119444. The approach gives more control but is not ergonomic since it requires overriding Gradle to pass in JDKs. Note that this proposal is not meant to replace passing toolchains to Gradle but proposed in addition.

Recently @kravemir has reignited the discussion and has suggested to implement detection based on environments/profiles in gradle/gradle#29214 (comment). This PR is an attempt to standardize a detection location per-derivation, namely $out/lib/jvm/$name and to enable linking these paths whenever someone sets programs.java.enable.

A hook is provided so that all JDKs can use the same code for creating the symlink.

Once this has landed, it'd be possible to have tools like Gradle and sbt crawl /run/current-system/sw/lib/jvm, ~/.nix-profile/lib/jvm, or other locations via nix-shell.

For now I have only changed four JDKs, but that should be sufficient to see what's going on and test the approach. Of course I am happy to add the hook to other JDKs too.

Alternative Considered

I also considered generating a pkgs.linkFarm in the Java module, but this solution is not as portable. By creating the link in each derivation, downstream package consumers can also make use of the mechanism, e.g. home-manager could also link them somewhere without having to reimplement linking.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/auto-detecting-java-installations/4677/11

[kravemir]

I'm only Nix beginner, so I can't review Nix files reliably.

One comment (concern) about the new location pattern to be founded here (share instead of lib).

Otherwise, the concept LGTM.

[raboof]

I'm somewhat reluctant about this change:

• for 'end-user' programs, the JDK should just be specified in the derivation, so for those you shouldn't need this module
• for 'developer' environments, the 'appropriate' JDKs might differ per project, so installing them 'globally' as a module doesn't make too much sense.

Having let this simmer for a while, I think I would prefer the approach of having each JDK contribute itself to a JAVA_HOMES environment variable (similar to how it sets the JAVA_HOME, but allowing multiple values).

That said, as this topic has been stagnant for a while, I'd be supportive of this change as well, just to get things unstuck and gain some more experience on how this works out in practice ;)

[lorenzleutgeb]

In response to @raboof:

• for 'end-user' programs, the JDK should just be specified in the derivation, so for those you shouldn't need this module

I agree. This is not the intended use-case here. I mention how it is possible to pass JDKs to Gradle, for example, which fits this use-case.

• for 'developer' environments, the 'appropriate' JDKs might differ per project, so installing them 'globally' as a module doesn't make too much sense.

This is why it is opt-in (currently tied to programs.java.enable). Note however, that the proposal also works with regular profiles, without NixOS. You don't have to use the "global" profile via /run/current-system/sw/lib/jvm, you can point your detection mechanism at an arbitrary profile/environment. For example:

$ nix profile install .#adoptopenjdk-hotspot-bin-8 --priority 7
$ ls ~/.nix-profile/lib/jvm
adoptopenjdk-hotspot-bin-8.0.372
$ nix profile install .#adoptopenjdk-hotspot-bin-11 --priority 7
$ ls ~/.nix-profile/lib/jvm
adoptopenjdk-hotspot-bin-8.0.372  adoptopenjdk-hotspot-bin-11.0.19
$ nix profile list
Index:              0
Store paths:        [...]

Index:              1
Store paths:        /nix/store/...-home-manager-path

Index:              2
Flake attribute:    legacyPackages.x86_64-linux.adoptopenjdk-hotspot-bin-8
Original flake URL: git+file:///home/lorenz/src/github.com/NixOS/nixpkgs
Locked flake URL:   git+file:///home/lorenz/src/github.com/NixOS/nixpkgs?ref=refs/heads/jvm&rev=3a9db7691302d62a6d9020c927a791f0ddbfba86
Store paths:        /nix/store/4q4d79yp3sj9310kcxajh6y7gyjg5j4q-adoptopenjdk-hotspot-bin-8.0.372

Index:              3
Flake attribute:    legacyPackages.x86_64-linux.adoptopenjdk-hotspot-bin-11
Original flake URL: git+file:///home/lorenz/src/github.com/NixOS/nixpkgs
Locked flake URL:   git+file:///home/lorenz/src/github.com/NixOS/nixpkgs?ref=refs/heads/jvm&rev=3a9db7691302d62a6d9020c927a791f0ddbfba86
Store paths:        /nix/store/yd3x5x25k66byrwrl6qq7z3caqqqips0-adoptopenjdk-hotspot-bin-11.0.19
% nix profile remove /nix/store/4q4d79yp3sj9310kcxajh6y7gyjg5j4q-adoptopenjdk-hotspot-bin-8.0.372
removing 'git+file:///home/lorenz/src/github.com/NixOS/nixpkgs#legacyPackages.x86_64-linux.adoptopenjdk-hotspot-bin-8'
$ ls ~/.nix-profile/lib/jvm
adoptopenjdk-hotspot-bin-11.0.19

I believe that this is also useful with nix-shell. Consider the following shell.nix placed at the root of nixpkgs:

let
  pkgs = import ./default.nix {};
  env = pkgs.buildEnv {
    name = "development";
    paths = with pkgs; [
      (adoptopenjdk-hotspot-bin-8  // { meta.priority = 6; })
      (adoptopenjdk-hotspot-bin-11 // { meta.priority = 5; })
    ];
  };
in
pkgs.mkShellNoCC {
  packages = [ env ];
  shellHook = ''
    echo "Your JVMs are in '${env}/lib/jvm' check it out:"
    ls -o ${env}/lib/jvm
  '';
}

This gives you:

$ nix-shell
Your JVMs are in '/nix/store/wn8c7mvnrlw83gsm6laq24haq94xp1b1-development/lib/jvm' check it out:
total 8
lrwxrwxrwx 3 root 119 Jan  1  1970 adoptopenjdk-hotspot-bin-11.0.19 -> /nix/store/yd3x5x25k66byrwrl6qq7z3caqqqips0-adoptopenjdk-hotspot-bin-11.0.19/lib/jvm/adoptopenjdk-hotspot-bin-11.0.19
lrwxrwxrwx 3 root 119 Jan  1  1970 adoptopenjdk-hotspot-bin-8.0.372 -> /nix/store/4q4d79yp3sj9310kcxajh6y7gyjg5j4q-adoptopenjdk-hotspot-bin-8.0.372/lib/jvm/adoptopenjdk-hotspot-bin-8.0.372

Having let this simmer for a while, I think I would prefer the approach of having each JDK contribute itself to a JAVA_HOMES environment variable (similar to how it sets the JAVA_HOME, but allowing multiple values).

I know that this is your preferred approach, I have linked #76699 above, but I think that what I am proposing here both fits the "Nix way" (convention on derivation outputs instead of environment variables) and also the "Java ecosystem way" (crawling magic directories).

[rhendric]

Forgive the bikeshedding, but why share? A JVM is mostly object files, so I'd have expected $out/lib/jvm (and this is a closer match to the /usr/lib/jvm path expected by Gradle and SBT).

Edit: Oops, I see this was already raised here.