From 13339f1df0948b6e55fb7643377064772ded25f3 Mon Sep 17 00:00:00 2001 From: stepan Date: Mon, 10 Jun 2024 10:56:26 +0200 Subject: [PATCH 1/3] Do not create IntelliJ artifacts by default --- docs/IDE.md | 35 +++++++- src/mx/_impl/mx.py | 2 +- src/mx/_impl/mx_ide_intellij.py | 146 +++++++++++++++++--------------- 3 files changed, 110 insertions(+), 73 deletions(-) diff --git a/docs/IDE.md b/docs/IDE.md index ba83cafe..a056ecae 100644 --- a/docs/IDE.md +++ b/docs/IDE.md @@ -2,6 +2,9 @@ ### IntelliJ +*For the time being, IntelliJ support is intended mainly for editing experience. There is limited support for building +mx based projects from within the IDE.* + Download and install the latest IntelliJ IDEA Community Edition: [https://www.jetbrains.com/idea/download/](https://www.jetbrains.com/idea/download/) Change the IntelliJ maximum memory to 2 GB or more. As per the [instructions](https://www.jetbrains.com/idea/help/increasing-memory-heap.html#d1366197e127), from the main menu choose **Help | Edit Custom VM Options** and modify the **-Xmx** and **-Xms** options. @@ -14,8 +17,6 @@ Open IntelliJ and go to **Preferences > Plugins > Browse Repositories**. Install * [Python Plugin](https://plugins.jetbrains.com/idea/plugin/631-python): python plugin * [Markdown Navigator](https://plugins.jetbrains.com/plugin/7896-markdown-navigator): markdown plugin -Check that the bundled Ant plugin is enabled in **Preferences > Plugins > Installed** (you may get `Unknown artifact properties: ant-postprocessing.` errors in your project artifacts otherwise). - Make sure you have [`mx`](https://github.com/graalvm/mx) installed and updated (`mx update`). Then, to initialize IntelliJ project files, go to the root of your project and invoke: `mx intellijinit` Open the folder of your freshly initialized project from IntelliJ (**IntelliJ IDEA > File > Open…**). All depending projects will be included automatically. @@ -39,6 +40,36 @@ The value is split using spaces as delimiter and prepended to the arguments pass Use `mx intellijinit --help` to view all the options and flags that allow further customization of the IntelliJ projects generation. + +#### Building From Within IntelliJ + +When building Java sources, `mx build` invokes Java compiler to produce class files and then bundles those classfiles +to jars and other distributions according to the configuration in `suite.py` files. + +IntelliJ is configured to build the same classfiles as `mx build` would produce. However, for the time being, the code +that invokes Java compiler inside `mx build` is separate from the code that configures the options for Java compiler +in IntelliJ and there may be inconsistencies leading to compilation errors in IntelliJ. + +Mx Java projects are represented as Java modules in IntelliJ. Java mx distributions and mx libraries are represented +as IntelliJ "libraries". The dependencies between IntelliJ Java modules and libraries should reflect the dependencies +on the mx side. + +The recommended approach is to start with manual `mx build` to build everything necessary for the project, +then trigger a build from within the IDE, which rebuilds all the Java classfiles, because IntelliJ refuses to reuse +classfiles built outside of IDE. After that, one can continue with edit & compile cycle in the IDE and the +subsequent compilations should be fast and incremental (tip: you can use "build file" or "build package" to make +them even faster). If you know which mx distributions are affected by your changes, you can manually invoke +the right `mx archive @ABC` and skip full `mx build` (useful in combination with linky layout). + +`mx intellijinit --mx-distributions` also generates IntelliJ "artifacts", which correspond to MX distributions. +Those artifacts are dummy and use Ant post-processing step to delegate to `mx archive @ARTIFACT_NAME`. +Moreover, the artifacts depend on other IntelliJ artifacts and Java modules to reflect the dependency structure +on the mx side. However, IntelliJ seems to ignore this and always invokes the post-processing step for all the +artifacts regardless of whether their dependencies changed or not, which makes this slow and impractical. If you +still want to use this feature, make sure that the bundled Ant plugin is enabled in **Preferences > Plugins > Installed** +(you may get `Unknown artifact properties: ant-postprocessing.` errors in your project artifacts otherwise). + + #### Making IntelliJ Feel Similar to Eclipse (Optional) Set IntelliJ to use the Eclipse compiler by going to *IntelliJ IDEA > Preferences > Build, Execution, Deployment > Java Compiler* diff --git a/src/mx/_impl/mx.py b/src/mx/_impl/mx.py index c50ddde9..3fc33e90 100755 --- a/src/mx/_impl/mx.py +++ b/src/mx/_impl/mx.py @@ -18173,7 +18173,7 @@ def alarm_handler(signum, frame): _CACHE_DIR = get_env('MX_CACHE_DIR', join(dot_mx_dir(), 'cache')) # The version must be updated for every PR (checked in CI) and the comment should reflect the PR's issue -version = VersionSpec("7.25.10") # GR-52545 Add 'at_least_one_mandatory_approver' new OWNERS rule type +version = VersionSpec("7.25.11") # GR-54613 Do not generate IntelliJ "artifacts" by default in mx intellijinit _mx_start_datetime = datetime.utcnow() diff --git a/src/mx/_impl/mx_ide_intellij.py b/src/mx/_impl/mx_ide_intellij.py index 1d3d5dab..bfa26fbf 100644 --- a/src/mx/_impl/mx_ide_intellij.py +++ b/src/mx/_impl/mx_ide_intellij.py @@ -65,6 +65,7 @@ class IntellijConfig: on_save_actions: bool = False refresh_only: bool = False do_fsck_projects: bool = True + mx_distributions: bool = False args: ... = None @@ -79,6 +80,7 @@ def intellijinit_cli(args): parser.add_argument('--max-java-compliance', dest='max_java_compliance', type=int, default=16, help='Cap the Java compliance at this value. IntelliJ requires an acceptance of a legal notice for beta Java specifications.') parser.add_argument('--import-inner-classes', action='store_true', dest='import_inner_classes', help='Configure auto-import to insert inner class imports.') parser.add_argument('--on-save-actions', action='store_true', dest='on_save_actions', help='Generate On Save Actions: checkstyle format and optimize imports.') + parser.add_argument('--mx-distributions', action='store_true', dest='mx_distributions', help='Generate Ant powered build of mx distributions (generated Ant scripts delegate to `mx archive {DIST}, bundled Ant plugin must be enabled in IntelliJ).') parser.add_argument('args', nargs=REMAINDER, metavar='...') extra_args = os.environ.get('MX_INTELLIJINIT_DEFAULTS', '').split() @@ -989,76 +991,80 @@ def processApDep(dep, edge): workspaceXml.close('project') mx.update_file(workspace_path, workspaceXml.xml(indent=' ', newl='\n')) - # mx integration - def antTargetName(dist): - return 'archive_' + dist.name - - def artifactFileName(dist): - return dist.name.replace('.', '_').replace('-', '_') + '.xml' - validDistributions = [dist for dist in mx.sorted_dists() if not dist.suite.isBinarySuite() and not dist.isTARDistribution()] - - # 1) Make an ant file for archiving the distributions. - antXml = mx.XMLDoc() - antXml.open('project', attributes={'name': s.name, 'default': 'archive'}) - for dist in validDistributions: - antXml.open('target', attributes={'name': antTargetName(dist)}) - antXml.open('exec', attributes={'executable': sys.executable}) - antXml.element('arg', attributes={'value': join(mx._mx_home, 'mx.py')}) - antXml.element('arg', attributes={'value': 'archive'}) - antXml.element('arg', attributes={'value': '@' + dist.name}) - antXml.close('exec') - antXml.close('target') - - antXml.close('project') - antFile = join(ideaProjectDirectory, 'ant-mx-archive.xml') - mx.update_file(antFile, antXml.xml(indent=' ', newl='\n')) - - # 2) Tell IDEA that there is an ant-build. - ant_mx_archive_xml = 'file://$PROJECT_DIR$/.idea/ant-mx-archive.xml' - metaAntXml = mx.XMLDoc() - metaAntXml.open('project', attributes={'version': '4'}) - metaAntXml.open('component', attributes={'name': 'AntConfiguration'}) - metaAntXml.open('buildFile', attributes={'url': ant_mx_archive_xml}) - metaAntXml.close('buildFile') - metaAntXml.close('component') - metaAntXml.close('project') - metaAntFile = join(ideaProjectDirectory, 'ant.xml') - mx.update_file(metaAntFile, metaAntXml.xml(indent=' ', newl='\n')) - - # 3) Make an artifact for every distribution - validArtifactNames = {artifactFileName(dist) for dist in validDistributions} - artifactsDir = join(ideaProjectDirectory, 'artifacts') - mx_util.ensure_dir_exists(artifactsDir) - for fileName in os.listdir(artifactsDir): - filePath = join(artifactsDir, fileName) - if os.path.isfile(filePath) and fileName not in validArtifactNames: - os.remove(filePath) - - for dist in validDistributions: - artifactXML = mx.XMLDoc() - artifactXML.open('component', attributes={'name': 'ArtifactManager'}) - artifactXML.open('artifact', attributes={'build-on-make': 'true', 'name': dist.name}) - artifactXML.open('output-path', data='$PROJECT_DIR$/mxbuild/artifacts/' + dist.name) - artifactXML.close('output-path') - artifactXML.open('properties', attributes={'id': 'ant-postprocessing'}) - artifactXML.open('options', attributes={'enabled': 'true'}) - artifactXML.open('file', data=ant_mx_archive_xml) - artifactXML.close('file') - artifactXML.open('target', data=antTargetName(dist)) - artifactXML.close('target') - artifactXML.close('options') - artifactXML.close('properties') - artifactXML.open('root', attributes={'id': 'root'}) - for javaProject in [dep for dep in dist.archived_deps() if dep.isJavaProject()]: - artifactXML.element('element', attributes={'id': 'module-output', 'name': javaProject.name}) - for javaProject in [dep for dep in dist.deps if dep.isLibrary() or (dep.isDistribution() and dep in validDistributions)]: - artifactXML.element('element', attributes={'id': 'artifact', 'artifact-name': javaProject.name}) - artifactXML.close('root') - artifactXML.close('artifact') - artifactXML.close('component') - - artifactFile = join(artifactsDir, artifactFileName(dist)) - mx.update_file(artifactFile, artifactXML.xml(indent=' ', newl='\n')) + if config.mx_distributions: + # We delegate to `mx archive {DISTRIBUTION}` using Ant script. + # The Ant script is executed as "ant-postprocessing" step of an artificial IntelliJ artifact that we + # create for each MX distribution. The artificial IntelliJ artifact is configured to "contain" (depend + # on) all the dependencies of the given MX distribution. See IDE.md for some more info. + def antTargetName(dist): + return 'archive_' + dist.name + + def artifactFileName(dist): + return dist.name.replace('.', '_').replace('-', '_') + '.xml' + validDistributions = [dist for dist in mx.sorted_dists() if not dist.suite.isBinarySuite() and not dist.isTARDistribution()] + + # 1) Make an ant file for archiving the distributions. + antXml = mx.XMLDoc() + antXml.open('project', attributes={'name': s.name, 'default': 'archive'}) + for dist in validDistributions: + antXml.open('target', attributes={'name': antTargetName(dist)}) + antXml.open('exec', attributes={'executable': sys.executable}) + antXml.element('arg', attributes={'value': join(mx._mx_home, 'mx.py')}) + antXml.element('arg', attributes={'value': 'archive'}) + antXml.element('arg', attributes={'value': '@' + dist.name}) + antXml.close('exec') + antXml.close('target') + + antXml.close('project') + antFile = join(ideaProjectDirectory, 'ant-mx-archive.xml') + mx.update_file(antFile, antXml.xml(indent=' ', newl='\n')) + + # 2) Tell IDEA that there is an ant-build. + ant_mx_archive_xml = 'file://$PROJECT_DIR$/.idea/ant-mx-archive.xml' + metaAntXml = mx.XMLDoc() + metaAntXml.open('project', attributes={'version': '4'}) + metaAntXml.open('component', attributes={'name': 'AntConfiguration'}) + metaAntXml.open('buildFile', attributes={'url': ant_mx_archive_xml}) + metaAntXml.close('buildFile') + metaAntXml.close('component') + metaAntXml.close('project') + metaAntFile = join(ideaProjectDirectory, 'ant.xml') + mx.update_file(metaAntFile, metaAntXml.xml(indent=' ', newl='\n')) + + # 3) Make an artifact for every distribution + validArtifactNames = {artifactFileName(dist) for dist in validDistributions} + artifactsDir = join(ideaProjectDirectory, 'artifacts') + mx_util.ensure_dir_exists(artifactsDir) + for fileName in os.listdir(artifactsDir): + filePath = join(artifactsDir, fileName) + if os.path.isfile(filePath) and fileName not in validArtifactNames: + os.remove(filePath) + + for dist in validDistributions: + artifactXML = mx.XMLDoc() + artifactXML.open('component', attributes={'name': 'ArtifactManager'}) + artifactXML.open('artifact', attributes={'build-on-make': 'true', 'name': dist.name}) + artifactXML.open('output-path', data='$PROJECT_DIR$/mxbuild/artifacts/' + dist.name) + artifactXML.close('output-path') + artifactXML.open('properties', attributes={'id': 'ant-postprocessing'}) + artifactXML.open('options', attributes={'enabled': 'true'}) + artifactXML.open('file', data=ant_mx_archive_xml) + artifactXML.close('file') + artifactXML.open('target', data=antTargetName(dist)) + artifactXML.close('target') + artifactXML.close('options') + artifactXML.close('properties') + artifactXML.open('root', attributes={'id': 'root'}) + for javaProject in [dep for dep in dist.archived_deps() if dep.isJavaProject()]: + artifactXML.element('element', attributes={'id': 'module-output', 'name': javaProject.name}) + for javaProject in [dep for dep in dist.deps if dep.isLibrary() or (dep.isDistribution() and dep in validDistributions)]: + artifactXML.element('element', attributes={'id': 'artifact', 'artifact-name': javaProject.name}) + artifactXML.close('root') + artifactXML.close('artifact') + artifactXML.close('component') + + artifactFile = join(artifactsDir, artifactFileName(dist)) + mx.update_file(artifactFile, artifactXML.xml(indent=' ', newl='\n')) def intellij_scm_name(vc_kind): if vc_kind == 'git': From 14c4e21abacb34e20daf900259e4b0a033276bcb Mon Sep 17 00:00:00 2001 From: stepan Date: Mon, 10 Jun 2024 11:47:20 +0200 Subject: [PATCH 2/3] IntelliJ docs: link "linky layout" to its documentation --- docs/IDE.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/IDE.md b/docs/IDE.md index a056ecae..d86965df 100644 --- a/docs/IDE.md +++ b/docs/IDE.md @@ -59,7 +59,8 @@ then trigger a build from within the IDE, which rebuilds all the Java classfiles classfiles built outside of IDE. After that, one can continue with edit & compile cycle in the IDE and the subsequent compilations should be fast and incremental (tip: you can use "build file" or "build package" to make them even faster). If you know which mx distributions are affected by your changes, you can manually invoke -the right `mx archive @ABC` and skip full `mx build` (useful in combination with linky layout). +the right `mx archive @ABC` and skip full `mx build` (useful in combination with +[linky layout](layout-distributions.md#linky_layout)). `mx intellijinit --mx-distributions` also generates IntelliJ "artifacts", which correspond to MX distributions. Those artifacts are dummy and use Ant post-processing step to delegate to `mx archive @ARTIFACT_NAME`. From 4cc69a50979fc0ced62ae9158653f98b3b097a3c Mon Sep 17 00:00:00 2001 From: Patrick Ziegler Date: Mon, 10 Jun 2024 11:06:36 +0000 Subject: [PATCH 3/3] Minor formatting fix --- docs/IDE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/IDE.md b/docs/IDE.md index d86965df..45f76f31 100644 --- a/docs/IDE.md +++ b/docs/IDE.md @@ -65,7 +65,7 @@ the right `mx archive @ABC` and skip full `mx build` (useful in combination with `mx intellijinit --mx-distributions` also generates IntelliJ "artifacts", which correspond to MX distributions. Those artifacts are dummy and use Ant post-processing step to delegate to `mx archive @ARTIFACT_NAME`. Moreover, the artifacts depend on other IntelliJ artifacts and Java modules to reflect the dependency structure -on the mx side. However, IntelliJ seems to ignore this and always invokes the post-processing step for all the +on the mx side. However, IntelliJ seems to ignore this and always invokes the post-processing step for all the artifacts regardless of whether their dependencies changed or not, which makes this slow and impractical. If you still want to use this feature, make sure that the bundled Ant plugin is enabled in **Preferences > Plugins > Installed** (you may get `Unknown artifact properties: ant-postprocessing.` errors in your project artifacts otherwise).