Simplify introduction of optional dependencies across multiple IDEs

[JojOatXGME]

Describe the need of your request

To resolve NixOS/nix-idea#1, I would need to add optional dependencies for each IDE I want to support. I already prototyped this using version 2.0 of this plugin at NixOS/nix-idea@0d83ea5. Unfortunately, adding optional dependencies for different IDEs causes some problems:

• Making this work requires significant changes to the project layout. However, that is only a one-time act and therefore not a very big issue.
• Just building the plugin without any tests requires downloading all the different IDEs I want to support. In my experience, this can easily exceed the 14 GiB which are available on GitHub Actions, making the standard runners of GitHub Actions effectively unusable to build such project. (I think I read somewhere, that they are actually always providing 20 GiB, but not sure where that was, and it may still not be enough.) The download of the IDEs also takes a noticeable amount of time. In my experience, the build already spends a large portion of the build downloading and extracting IDEs, even without having such optional dependencies.
• My solution still seems a bit hacky:
  https://github.com/NixOS/nix-idea/blob/0d83ea56c27416fe6d94ac039f28d7cee71f94f9/core/build.gradle.kts#L23-L25
  (Note that the plugin.xml is in the core module, but the final plugin is created in the rood project of Gradle, as all the remaining modules also need to be added as dependencies. This also results in an empty Jar from the root project to be included in the plugin, which is a bit strange.)

I also made a related post on the Support Community about a month ago, but all relevant information should be mentioned by this issue as well. A few years ago, I created IJPL-5238, which would also resolve my very specific issue from a different angel, but there seems to be no interest in resolving this issue from this end.

Proposed solution

My proposed solution consists of two parts:

1. Use Maven Artifacts as compile dependencies, instead of complete IDE builds.
2. Support usage of multiple source sets, similar to Feature variants in Gradle.

1. Use Maven Artifacts as compile dependencies

As far as I can tell, many platform dependencies seem to have a set of corresponding Maven coordinates:

Dependency	Maven Coordinate
<depends>com.intellij.modules.platform</depends>	com.jetbrains.intellij.platform:core and probably other, like ... com.jetbrains.intellij.platform:editor
<depends>com.intellij.modules.lang</depends>	com.jetbrains.intellij.platform:lang
<depends>com.intellij.modules.vcs</depends>	com.jetbrains.intellij.platform:vcs
<depends>com.intellij.modules.java</depends>	com.jetbrains.intellij.java:java
<depends>com.intellij.modules.python</depends>	com.jetbrains.intellij.python:python-community?
...	...

Note: The coordinates on the right of the table are probably incomplete. I guess each platform dependency may relate to multiple coordinates in Maven, and I have only listed the obvious one.

By using these coordinates instead of downloading the entire IDEs, we can greatly reduce the amount of data we have to download to build the plugin.

• Common dependencies can be shared across the build.
• Download of unrelated files from the IDE distributions is avoided.
• No need to extract the files, which further halves the amount of required disk space.

Of course, we still need to download some IDE to run the tests. However, developers who only change the code of a specific module, can run the tests with only one IDE. On GitHub Actions, I can easily avoid exceeding the disk limitations by creating separate jobs for each IDE I want to use for testing.

2. Support usage of multiple source sets

Less important, but making the introduction of optional dependencies much easier. Not sure about the DSL, but something similar to the following might work:

registerSourceSet("java") {
    ideaModule("com.intellij.modules.java")
}
registerSourceSet("python") {
    ideaModule("com.intellij.modules.python")
}

The tests in the test-source-set could be filtered depending on the used IDE.

@EnabledOnIde(PyCharm)
final class MyTest {
  ...
}

Alternatives you've considered

Supporting the use of the maven coordinates should resolve my biggest issue. This means supporting the use of multiple source sets can be skipped. I am not sure if there are good alternatives to reduce the amount of data which needs to be downloaded.

Additional context

I know that this would be a significant change compared to the current architecture. However, it would seem like a cleaner solution to me. There are some issues related to this proposal.

I haven't seen a BOM-artifact on Maven, which means we would have to individually specify the version for all the coordinates. A BOM artifact would be nice to provide a trivial solution to align the versions of all the dependencies.

I also noticed that the organization of the maven coordinates is a bit inconsistent. For example, com.jetbrains.intellij.java:java is just the java module, but com.jetbrains.intellij.clion:clion is a binary distribution of the whole IDE. This inconsistency makes me fear that there is currently no consistent build setup between the different IDEs, which may make it more difficult to gather all the information necessary to create a correct mapping between the platform dependencies and the maven coordinates. It also means that some IDEs may not publish the modules individually, and that it could be hard to add a BOM artifact in the future.

Another issue is that the Gradle Grammar-Kit Plugin currently requires a working IDE, and it uses the compile-classpath of the Gradle project to find this IDE. This means, if not fixed, that the Gradle Grammar-Kit Plugin would stop working after making the changes listed here.

[JojOatXGME]

I just wanted to add that I am aware that this would be a big shift and I don't expect this to happen fast or soon. However, if there is interest, maybe it could be implemented as an experimental feature behind some flag. I might be willing to prototype a solution.