Use JLS23 as latest compliance level in AST

[gayanper]

Currently JavaCore returns the latest supported java version as Java 23, But the AST validation is restricted to Java 22, this cause the issue with vscode java debugger, which sends the compliancy level as 23 when evaluating debug breakpoints to be registered.

[gayanper]

@mickaelistria another small fix to make javac usable with debugging in vscode.

[mickaelistria]

This seems to cause ~60 new test failures: https://ci.eclipse.org/ls/job/jdt-core-incubator/job/PR-465/1/
The issue is that when we set the "latest" then many other bits of JDT expect that ECJ can support this latest too, which is not the case yet.This seems to cause ~60 new test failures: https://ci.eclipse.org/ls/job/jdt-core-incubator/job/PR-465/1/
The issue is that when we set the "latest" then many other bits of JDT expect that ECJ can support this latest too, which is not the case yet. So far, the approach of declaring the new versions but not listing them as "latest" has been the best working one. Do you know where in your use-case is JLS_Latest being used?

[gayanper]

If we check the code here https://github.com/microsoft/java-debug/blob/83403a458e0d01fba5a3871fea81af742460bdf0/com.microsoft.java.debug.plugin/src/main/java/com/microsoft/java/debug/plugin/internal/JdtSourceLookUpProvider.java#L103

as we can see it reads the latest java version from JavaCore.latestSupportedJavaVersion() which has the following implementation in our incubator

	public static String latestSupportedJavaVersion() {
		return allVersions.get(allVersions.size() - 1);
	}

allVersions has VERSION_23 as the last element.

[gayanper]

@mickaelistria How should we handle this ?

[mickaelistria]

I'm curious about "AST validation is restricted to Java 22", can you please elaborate how we can see this? I think AST validation should be working for any known AST version, independently of what's the one marked as latest.

[gayanper]

I'm curious about "AST validation is restricted to Java 22", can you please elaborate how we can see this? I think AST validation should be working for any known AST version, independently of what's the one marked as latest.

The problem happens in https://github.com/gayanper/eclipse.jdt.core/blob/81b72d0d5ee1283ae61f543bb64d6e966fd35e22/org.eclipse.jdt.core/dom/org/eclipse/jdt/core/dom/AST.java#L1181

Since the constant at https://github.com/gayanper/eclipse.jdt.core/blob/81b72d0d5ee1283ae61f543bb64d6e966fd35e22/org.eclipse.jdt.core/dom/org/eclipse/jdt/core/dom/AST.java#L546 is not pointing to the latest version provided by avaCore.latestSupportedJavaVersion() method.

[testforstephen]

I previously encountered this issue as well while using the VS Code Java Debugger to debug applications that were switched to Javac. It happens when my session enables breakpoints in some Java files. The reason is that Java Debugger needs to invoke the DOM ASTParser (e.g., ASTParser.newParser(this.latestASTLevel)) to parse the target file and identify the breakpoint context. The debugger always uses the highest AST level supported by JavaCore.latestSupportedJavaVersion() to parse the file. Since the JDT Core updated the latest version to 23, but the Javac parser does not and just supports version 22, resulting in an unsupported AST version error.

To prevent this issue from occurring in the future, we need a flag to determine the actual highest AST version supported by the underlying parser.

[gayanper]

@testforstephen @mickaelistria What if we use the above flag in the JavaCore.latestSupportedJavaVersion() so that we can override it for javac builds ?

[testforstephen]

Introducing a system property will delegate the maintenance responsibility to the integration client, that looks a bit heavy to me. Instead, the supported java versions should be a capability of the ASTParser. The ASTParser should include a new method getSupportedHighestJavaVersion() to tell the consumer the highest Java version it supports.

Additionally, we have added an extension point compilationUnitResolver to enable the DOM ASTParser to support alternative parser. I am thinking whether it would be beneficial to extend this extension point to specify the supported Java versions.

[mickaelistria]

The AST class has some symbols that can probably be changed and used for that case.
We recently had a discussion on another issue with @iloveeclipse , there are multiple possible meanings for "support", and multiple place where versions are set/used. With the current approaches to use alternative parsers, we will have to establish new practices regarding updating those versions as the current ECJ-driven workflow don't fit for Javac.

[gayanper]

@mickaelistria @testforstephen changes to jdt.core for supporting javac support will get approved ? Since the current way of jdt working is tailored to ECJ and it works fine. And if we are suggesting a new change that benefit the javac support which is not yet in jdt.core main line, I'm thinking if it will be a hard sell.

[mickaelistria]

changes to jdt.core for supporting javac support will get approved ?

The API change and extension points to allow plugging an alternative parser/compiler are already merged in. The particular Javac extension is not planned to be contributed to JDT at the moment. I would be great to contribute it later, but this is not yet a priority to us and to other JDT contributors, so let's wait a bit.
Some of the remaining changes have been only 2 days ago: eclipse-jdt#2699, eclipse-jdt#2700 , eclipse-jdt#2701 and a couple more changes still have to completed and contributed.