Snapshot versions on repo.eclipse.org have wrong version

[stephan-herrmann]

I was trying to consume ecj-I20231001-1800.jar via maven and was happy to find what looked like a corresponding artifact on https://repo.eclipse.org/service/local/repo_groups/eclipse/content/org/eclipse/jdt/ecj/3.36.0-SNAPSHOT/ecj-3.36.0-20231001.234859-60.jar -- apparently, the snapshot was deployed to repo with a few hours' delay, fine.

Unfortunately, that artifact is outdated, as witnessed in the contained MANIFEST.MF, which says:

Bundle-Version: 3.36.0.v20230923-2346

This inconsistent versioning is highly confusing.

Note, that next to -SNAPSHOT the latest version is 3.36.0.v20230923-2346 (same version as stated in the manifest). Apparently, newer uploads update the version but supply the same bytes like a week ago?

[laeubi]

Can you explain why the version is wrong? If nothing has changed, the jar is most likely replaced by the baseline/previous build but maven can't know about that fact so OSGi Version != Artifact version, but that's nothing "wrong" as both versions are independent identifiers.

[merks]

Yes, this seems expected to me. It's going to be one of these two versions:

Soon (tomorrow I expect), it should be the latest one.

Here's the log for when it happens:

https://ci.eclipse.org/releng/view/Publish%20to%20Maven/job/PublishJDTtoMaven/lastBuild/console

It gets these bytes from the I-Build. (The last while I've seen significantly long delays in changes in the download.eclipse.org server actually being visible downstream.)

[stephan-herrmann]

At 23:46 on 2023-10-01 the latest ecj was ecj-I20231001-1800.jar, still a "new" snapshot was uploaded using a version from a week earlier. Are you saying 5 hours is not sufficient for the deploy job to find the artifact from the latest I-Build?

Or is this about the differences between monday I-builds and common I-builds?

If I-builds become visible to the deploy job only once on monday, then perhaps it should be scheduled to run only on mondays to avoid all those snapshots without any changes?

[stephan-herrmann]

Asked differently, how can consumers of snapshots on repo.eclipse.org find out, which I-build a particular version corresponds to?

[merks]

Let me ask this question too, when is the I-Build composite updated to include the latest I-build?

The fact that something produced 3.36.0.v20231001-1710---note the lateness in the time---isn't in the SNAPSHOT result the next build at Build #572 (Oct 1, 2023, 7:48:20 PM) doesn't seem so shocking. I think the I-build produces result and then runs tests for seemingly endless hours and the result is not visible in the I-build until after those tests.

Indeed, looking here:

I think the most recent I-Build has the next result but the composite was updated after the maven publishing job.

There isn't much in the way of human readable information in a Maven repository. I'm not sure what one ought to expect in that regard...

[laeubi]

Asked differently, how can consumers of snapshots on repo.eclipse.org find out, which I-build a particular version corresponds to?

I think there is no direct way. If you look at the manifest you will find:

Eclipse-SourceReferences: scm:git:https://github.com/eclipse-jdt/eclipse
 .jdt.core.git;path="org.eclipse.jdt.core.compiler.batch";tag="I20230924
 -0600";commitId=a6e42a2cb58a52b88df232a7acf46f8e4cbf599e

that shows you from what commit/tag the artifact was build.

[stephan-herrmann]

Alright, the delay before completion of the I-build build (not considering tests, which are separate jobs) and update of the composite repo is a big surprise to me. Is that to hold back auto-updates in case of an unstable build (i.t.o. too many test failures)?

There isn't much in the way of human readable information in a Maven repository. I'm not sure what one ought to expect in that regard...

Here's what I expect: if a given I-build is not yet visible to the deploy job, then no "new" artifact with content from the past should be deployed. I'm aware that time stamps from different job are not perfectly in sync, but an artifact with timestamp 2023-1001 and content from 2023-0923 heavily confused me and my team, and caused some hours of debugging a failure that has already been resolved, merged, built and tested.

[laeubi]

@stephan-herrmann that why I advocated in the past that JDT simply deploys snapshots from its master build instead of waiting for "the next big ibuild" happen... also see:

• Deploy the milestones (and maybe Releasecandidates) at maven central #1002
• Deploy batch compiler milestones to maven central eclipse-jdt/eclipse.jdt.core#1454
• Deploy the ecj.jar to the eclipse-maven-repository eclipse-jdt/eclipse.jdt.core#209

[iloveeclipse]

AFAIK this job is deploying ecj, and it is triggered manually: https://ci.eclipse.org/jdt/job/copyAndDeployJDTCompiler/

What the other job is doing I have no idea.

[merks]

As far as I know, it's these jobs that do this:

• https://ci.eclipse.org/releng/view/Publish%20to%20Maven/
• https://ci.eclipse.org/releng/view/Publish%20to%20Maven/job/PublishJDTtoMaven/

But this depends on where you are looking...

In any case, it's all open source so if something is not meeting expectations one is free to change it to make it meet those expectations or one can change change one's expectations, i.e., it might up to 24 hours to get the latest published.

[stephan-herrmann]

I'm withdrawing this issue.

I initially thought it was some simple oversight that some build picked up the wrong version, and just wanted to be helpful to alert people of this.

Apparently there are many dragons involved which cannot be defeated easily.

As for ecj versions I learned to only ever trust on: java -jar ecj-xyz.jar --version.

For my current use case I had manually created an internal maven repo with content from the latest I-build, stupid, solid, works.

[sravanlakkimsetti]

@stephan-herrmann This is definitely a valid question.

This is my theory.
During the I-build there are multiple steps.
I-build repo and composite gets built and added in "Promote Update Site" step
and the maven publish is triggered in "Trigger publication to Maven snapshots repo"

The time difference between these two about 5 seconds.

Now there is caching feature with eclipse websites. Because of this new repo and addition to composite take up to 30 minutes to reflect in the eclipse web pages.

I feel this is the root cause of the problem. Easy way to verify this is to add wait of 30 minutes to "Trigger publication to Maven snapshots repo"

[sravanlakkimsetti]

Try changing this line

eclipse.platform.releng.aggregator/JenkinsJobs/Builds/I_build.groovy

Line 520 in 5ccc7e0

build job: 'CBIaggregator', parameters: [string(name: 'snapshotOrRelease', value: '-snapshot')], wait: false

To build quietPeriod: 1800, wait: false, job: 'CBIaggregator', parameters: [string(name: 'snapshotOrRelease', value: '-snapshot')]

Or add a 30 minute quiet period to CBIaggregator job