[GR-54613] Do not create IntelliJ artifacts by default.

PullRequest: mx/1803

docs/IDE.md

@@ -2,6 +2,9 @@
 
 ### IntelliJ
 
+*For the time being, IntelliJ support is intended mainly for editing experience. There is limited support for building
+mx based projects from within the IDE.*
+
 Download and install the latest IntelliJ IDEA Community Edition: [https://www.jetbrains.com/idea/download/](https://www.jetbrains.com/idea/download/)
 
 Change the IntelliJ maximum memory to 2 GB or more. As per the [instructions](https://www.jetbrains.com/idea/help/increasing-memory-heap.html#d1366197e127), from the main menu choose **Help | Edit Custom VM Options** and modify the **-Xmx** and **-Xms** options.
@@ -14,8 +17,6 @@ Open IntelliJ and go to **Preferences > Plugins > Browse Repositories**. Install
 * [Python Plugin](https://plugins.jetbrains.com/idea/plugin/631-python): python plugin
 * [Markdown Navigator](https://plugins.jetbrains.com/plugin/7896-markdown-navigator): markdown plugin
 
-Check that the bundled Ant plugin is enabled in **Preferences > Plugins > Installed** (you may get `Unknown artifact properties: ant-postprocessing.` errors in your project artifacts otherwise).
-
 Make sure you have [`mx`](https://github.com/graalvm/mx) installed and updated (`mx update`). Then, to initialize IntelliJ project files, go to the root of your project and invoke: `mx intellijinit`
 
 Open the folder of your freshly initialized project from IntelliJ (**IntelliJ IDEA > File > Open…**). All depending projects will be included automatically.
@@ -39,6 +40,37 @@ The value is split using spaces as delimiter and prepended to the arguments pass
 Use `mx intellijinit --help` to view all the options and flags that allow further customization
 of the IntelliJ projects generation.
 
+
+#### Building From Within IntelliJ
+
+When building Java sources, `mx build` invokes Java compiler to produce class files and then bundles those classfiles
+to jars and other distributions according to the configuration in `suite.py` files.
+
+IntelliJ is configured to build the same classfiles as `mx build` would produce. However, for the time being, the code
+that invokes Java compiler inside `mx build` is separate from the code that configures the options for Java compiler
+in IntelliJ and there may be inconsistencies leading to compilation errors in IntelliJ.
+
+Mx Java projects are represented as Java modules in IntelliJ. Java mx distributions and mx libraries are represented
+as IntelliJ "libraries". The dependencies between IntelliJ Java modules and libraries should reflect the dependencies
+on the mx side.
+
+The recommended approach is to start with manual `mx build` to build everything necessary for the project,
+then trigger a build from within the IDE, which rebuilds all the Java classfiles, because IntelliJ refuses to reuse
+classfiles built outside of IDE. After that, one can continue with edit & compile cycle in the IDE and the
+subsequent compilations should be fast and incremental (tip: you can use "build file" or "build package" to make
+them even faster). If you know which mx distributions are affected by your changes, you can manually invoke
+the right `mx archive @ABC` and skip full `mx build` (useful in combination with
+[linky layout](layout-distributions.md#linky_layout)).
+
+`mx intellijinit --mx-distributions` also generates IntelliJ "artifacts", which correspond to MX distributions.
+Those artifacts are dummy and use Ant post-processing step to delegate to `mx archive @ARTIFACT_NAME`.
+Moreover, the artifacts depend on other IntelliJ artifacts and Java modules to reflect the dependency structure
+on the mx side. However, IntelliJ seems to ignore this and always invokes the post-processing step for all the
+artifacts regardless of whether their dependencies changed or not, which makes this slow and impractical. If you
+still want to use this feature, make sure that the bundled Ant plugin is enabled in **Preferences > Plugins > Installed**
+(you may get `Unknown artifact properties: ant-postprocessing.` errors in your project artifacts otherwise).
+
+
 #### Making IntelliJ Feel Similar to Eclipse (Optional)
 
 Set IntelliJ to use the Eclipse compiler by going to *IntelliJ IDEA > Preferences > Build, Execution, Deployment > Java Compiler*

src/mx/_impl/mx.py

@@ -18173,7 +18173,7 @@ def alarm_handler(signum, frame):
 _CACHE_DIR = get_env('MX_CACHE_DIR', join(dot_mx_dir(), 'cache'))
 
 # The version must be updated for every PR (checked in CI) and the comment should reflect the PR's issue
-version = VersionSpec("7.25.10")  # GR-52545 Add 'at_least_one_mandatory_approver' new OWNERS rule type
+version = VersionSpec("7.25.11")  # GR-54613 Do not generate IntelliJ "artifacts" by default in mx intellijinit
 
 _mx_start_datetime = datetime.utcnow()
 

src/mx/_impl/mx_ide_intellij.py

@@ -65,6 +65,7 @@ class IntellijConfig:
     on_save_actions: bool = False
     refresh_only: bool = False
     do_fsck_projects: bool = True
+    mx_distributions: bool = False
     args: ... = None
 
 
@@ -79,6 +80,7 @@ def intellijinit_cli(args):
     parser.add_argument('--max-java-compliance', dest='max_java_compliance', type=int, default=16, help='Cap the Java compliance at this value. IntelliJ requires an acceptance of a legal notice for beta Java specifications.')
     parser.add_argument('--import-inner-classes', action='store_true', dest='import_inner_classes', help='Configure auto-import to insert inner class imports.')
     parser.add_argument('--on-save-actions', action='store_true', dest='on_save_actions', help='Generate On Save Actions: checkstyle format and optimize imports.')
+    parser.add_argument('--mx-distributions', action='store_true', dest='mx_distributions', help='Generate Ant powered build of mx distributions (generated Ant scripts delegate to `mx archive {DIST}, bundled Ant plugin must be enabled in IntelliJ).')
     parser.add_argument('args', nargs=REMAINDER, metavar='...')
 
     extra_args = os.environ.get('MX_INTELLIJINIT_DEFAULTS', '').split()
@@ -989,76 +991,80 @@ def processApDep(dep, edge):
                 workspaceXml.close('project')
                 mx.update_file(workspace_path, workspaceXml.xml(indent='  ', newl='\n'))
 
-            # mx integration
-            def antTargetName(dist):
-                return 'archive_' + dist.name
-
-            def artifactFileName(dist):
-                return dist.name.replace('.', '_').replace('-', '_') + '.xml'
-            validDistributions = [dist for dist in mx.sorted_dists() if not dist.suite.isBinarySuite() and not dist.isTARDistribution()]
-
-            # 1) Make an ant file for archiving the distributions.
-            antXml = mx.XMLDoc()
-            antXml.open('project', attributes={'name': s.name, 'default': 'archive'})
-            for dist in validDistributions:
-                antXml.open('target', attributes={'name': antTargetName(dist)})
-                antXml.open('exec', attributes={'executable': sys.executable})
-                antXml.element('arg', attributes={'value': join(mx._mx_home, 'mx.py')})
-                antXml.element('arg', attributes={'value': 'archive'})
-                antXml.element('arg', attributes={'value': '@' + dist.name})
-                antXml.close('exec')
-                antXml.close('target')
-
-            antXml.close('project')
-            antFile = join(ideaProjectDirectory, 'ant-mx-archive.xml')
-            mx.update_file(antFile, antXml.xml(indent='  ', newl='\n'))
-
-            # 2) Tell IDEA that there is an ant-build.
-            ant_mx_archive_xml = 'file://$PROJECT_DIR$/.idea/ant-mx-archive.xml'
-            metaAntXml = mx.XMLDoc()
-            metaAntXml.open('project', attributes={'version': '4'})
-            metaAntXml.open('component', attributes={'name': 'AntConfiguration'})
-            metaAntXml.open('buildFile', attributes={'url': ant_mx_archive_xml})
-            metaAntXml.close('buildFile')
-            metaAntXml.close('component')
-            metaAntXml.close('project')
-            metaAntFile = join(ideaProjectDirectory, 'ant.xml')
-            mx.update_file(metaAntFile, metaAntXml.xml(indent='  ', newl='\n'))
-
-            # 3) Make an artifact for every distribution
-            validArtifactNames = {artifactFileName(dist) for dist in validDistributions}
-            artifactsDir = join(ideaProjectDirectory, 'artifacts')
-            mx_util.ensure_dir_exists(artifactsDir)
-            for fileName in os.listdir(artifactsDir):
-                filePath = join(artifactsDir, fileName)
-                if os.path.isfile(filePath) and fileName not in validArtifactNames:
-                    os.remove(filePath)
-
-            for dist in validDistributions:
-                artifactXML = mx.XMLDoc()
-                artifactXML.open('component', attributes={'name': 'ArtifactManager'})
-                artifactXML.open('artifact', attributes={'build-on-make': 'true', 'name': dist.name})
-                artifactXML.open('output-path', data='$PROJECT_DIR$/mxbuild/artifacts/' + dist.name)
-                artifactXML.close('output-path')
-                artifactXML.open('properties', attributes={'id': 'ant-postprocessing'})
-                artifactXML.open('options', attributes={'enabled': 'true'})
-                artifactXML.open('file', data=ant_mx_archive_xml)
-                artifactXML.close('file')
-                artifactXML.open('target', data=antTargetName(dist))
-                artifactXML.close('target')
-                artifactXML.close('options')
-                artifactXML.close('properties')
-                artifactXML.open('root', attributes={'id': 'root'})
-                for javaProject in [dep for dep in dist.archived_deps() if dep.isJavaProject()]:
-                    artifactXML.element('element', attributes={'id': 'module-output', 'name': javaProject.name})
-                for javaProject in [dep for dep in dist.deps if dep.isLibrary() or (dep.isDistribution() and dep in validDistributions)]:
-                    artifactXML.element('element', attributes={'id': 'artifact', 'artifact-name': javaProject.name})
-                artifactXML.close('root')
-                artifactXML.close('artifact')
-                artifactXML.close('component')
-
-                artifactFile = join(artifactsDir, artifactFileName(dist))
-                mx.update_file(artifactFile, artifactXML.xml(indent='  ', newl='\n'))
+            if config.mx_distributions:
+                # We delegate to `mx archive {DISTRIBUTION}` using Ant script.
+                # The Ant script is executed as "ant-postprocessing" step of an artificial IntelliJ artifact that we
+                # create for each MX distribution. The artificial IntelliJ artifact is configured to "contain" (depend
+                # on) all the dependencies of the given MX distribution. See IDE.md for some more info.
+                def antTargetName(dist):
+                    return 'archive_' + dist.name
+
+                def artifactFileName(dist):
+                    return dist.name.replace('.', '_').replace('-', '_') + '.xml'
+                validDistributions = [dist for dist in mx.sorted_dists() if not dist.suite.isBinarySuite() and not dist.isTARDistribution()]
+
+                # 1) Make an ant file for archiving the distributions.
+                antXml = mx.XMLDoc()
+                antXml.open('project', attributes={'name': s.name, 'default': 'archive'})
+                for dist in validDistributions:
+                    antXml.open('target', attributes={'name': antTargetName(dist)})
+                    antXml.open('exec', attributes={'executable': sys.executable})
+                    antXml.element('arg', attributes={'value': join(mx._mx_home, 'mx.py')})
+                    antXml.element('arg', attributes={'value': 'archive'})
+                    antXml.element('arg', attributes={'value': '@' + dist.name})
+                    antXml.close('exec')
+                    antXml.close('target')
+
+                antXml.close('project')
+                antFile = join(ideaProjectDirectory, 'ant-mx-archive.xml')
+                mx.update_file(antFile, antXml.xml(indent='  ', newl='\n'))
+
+                # 2) Tell IDEA that there is an ant-build.
+                ant_mx_archive_xml = 'file://$PROJECT_DIR$/.idea/ant-mx-archive.xml'
+                metaAntXml = mx.XMLDoc()
+                metaAntXml.open('project', attributes={'version': '4'})
+                metaAntXml.open('component', attributes={'name': 'AntConfiguration'})
+                metaAntXml.open('buildFile', attributes={'url': ant_mx_archive_xml})
+                metaAntXml.close('buildFile')
+                metaAntXml.close('component')
+                metaAntXml.close('project')
+                metaAntFile = join(ideaProjectDirectory, 'ant.xml')
+                mx.update_file(metaAntFile, metaAntXml.xml(indent='  ', newl='\n'))
+
+                # 3) Make an artifact for every distribution
+                validArtifactNames = {artifactFileName(dist) for dist in validDistributions}
+                artifactsDir = join(ideaProjectDirectory, 'artifacts')
+                mx_util.ensure_dir_exists(artifactsDir)
+                for fileName in os.listdir(artifactsDir):
+                    filePath = join(artifactsDir, fileName)
+                    if os.path.isfile(filePath) and fileName not in validArtifactNames:
+                        os.remove(filePath)
+
+                for dist in validDistributions:
+                    artifactXML = mx.XMLDoc()
+                    artifactXML.open('component', attributes={'name': 'ArtifactManager'})
+                    artifactXML.open('artifact', attributes={'build-on-make': 'true', 'name': dist.name})
+                    artifactXML.open('output-path', data='$PROJECT_DIR$/mxbuild/artifacts/' + dist.name)
+                    artifactXML.close('output-path')
+                    artifactXML.open('properties', attributes={'id': 'ant-postprocessing'})
+                    artifactXML.open('options', attributes={'enabled': 'true'})
+                    artifactXML.open('file', data=ant_mx_archive_xml)
+                    artifactXML.close('file')
+                    artifactXML.open('target', data=antTargetName(dist))
+                    artifactXML.close('target')
+                    artifactXML.close('options')
+                    artifactXML.close('properties')
+                    artifactXML.open('root', attributes={'id': 'root'})
+                    for javaProject in [dep for dep in dist.archived_deps() if dep.isJavaProject()]:
+                        artifactXML.element('element', attributes={'id': 'module-output', 'name': javaProject.name})
+                    for javaProject in [dep for dep in dist.deps if dep.isLibrary() or (dep.isDistribution() and dep in validDistributions)]:
+                        artifactXML.element('element', attributes={'id': 'artifact', 'artifact-name': javaProject.name})
+                    artifactXML.close('root')
+                    artifactXML.close('artifact')
+                    artifactXML.close('component')
+
+                    artifactFile = join(artifactsDir, artifactFileName(dist))
+                    mx.update_file(artifactFile, artifactXML.xml(indent='  ', newl='\n'))
 
         def intellij_scm_name(vc_kind):
             if vc_kind == 'git':