sbt 1.10.5 (was 1.9.9)

[SethTisue]

not for merge until 1.10.0 final; but let's see if CI likes it

[SethTisue]

@eed3si9n the test failures here worry me somewhat, but I might not have time this week to look into it

[eed3si9n]

Acknowledged. Thanks for starting this, @SethTisue!

[eed3si9n]

Opened sbt/zinc#1351

[eed3si9n]

FYI @bishabosha - new Zinc 1.10.x breaks scripted test added by #18880.
https://github.com/scala/scala3/actions/runs/8690208123/job/23829593105#step:9:342

[info] [error] java.lang.RuntimeException: Can't find source file for a.A.Inner_sel
[info] [error] 	at scala.sys.package$.error(package.scala:30)
[info] [error] 	at sbt.internal.inc.Analysis$.$anonfun$sources$1(Analysis.scala:105)
[info] [error] 	at scala.Option.getOrElse(Option.scala:189)
[info] [error] 	at sbt.internal.inc.Analysis$.sourceFileForClass$1(Analysis.scala:105)
[info] [error] 	at sbt.internal.inc.Analysis$.isJavaClass$1(Analysis.scala:107)
[info] [error] 	at sbt.internal.inc.Analysis$.$anonfun$sources$2(Analysis.scala:108)
[info] [error] 	at sbt.internal.inc.Analysis$.$anonfun$sources$2$adapted(Analysis.scala:108)
[info] [error] 	at scala.collection.TraversableLike.$anonfun$partition$1(TraversableLike.scala:450)
[info] [error] 	at scala.collection.immutable.HashMap$HashMapKeys.$anonfun$foreach$1(HashMap.scala:150)
[info] [error] 	at scala.collection.immutable.HashMap$HashMap1.foreachEntry(HashMap.scala:401)
[info] [error] 	at scala.collection.immutable.HashMap$HashTrieMap.foreachEntry(HashMap.scala:735)
[info] [error] 	at scala.collection.immutable.HashMap$HashTrieMap.foreachEntry(HashMap.scala:735)
[info] [error] 	at scala.collection.immutable.HashMap$HashMapKeys.foreach(HashMap.scala:150)
[info] [error] 	at scala.collection.TraversableLike.partition(TraversableLike.scala:450)
[info] [error] 	at scala.collection.TraversableLike.partition$(TraversableLike.scala:448)
[info] [error] 	at scala.collection.AbstractTraversable.partition(Traversable.scala:108)
[info] [error] 	at sbt.internal.inc.Analysis$.sources(Analysis.scala:108)
[info] [error] 	at sbt.internal.inc.IncrementalCommon$CycleState$IncrementalCallbackImpl.mergeAndInvalidate(IncrementalCommon.scala:193)
[info] [error] 	at sbt.internal.inc.IncrementalCommon$CycleState$IncrementalCallbackImpl.completeCycle(IncrementalCommon.scala:209)
[info] [error] 	at sbt.internal.inc.AnalysisCallback.getCycleResultOnce(Incremental.scala:940)
[info] [error] 	at sbt.internal.inc.Incremental$$anon$2.run(Incremental.scala:455)
[info] [error] 	at sbt.internal.inc.IncrementalCommon$CycleState.next(IncrementalCommon.scala:117)
[info] [error] 	at sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:56)
[info] [error] 	at sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:52)
[info] [error] 	at sbt.internal.inc.IncrementalCommon.cycle(IncrementalCommon.scala:265)
[info] [error] 	at sbt.internal.inc.Incremental$.$anonfun$incrementalCompile$8(Incremental.scala:409)
[info] [error] 	at sbt.internal.inc.Incremental$.withClassfileManager(Incremental.scala:496)
[info] [error] 	at sbt.internal.inc.Incremental$.incrementalCompile(Incremental.scala:396)
[info] [error] 	at sbt.internal.inc.Incremental$.apply(Incremental.scala:170)
[info] [error] 	at sbt.internal.inc.IncrementalCompilerImpl.compileInternal(IncrementalCompilerImpl.scala:534)
[info] [error] 	at sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileIncrementally$1(IncrementalCompilerImpl.scala:488)
[info] [error] 	at sbt.internal.inc.IncrementalCompilerImpl.handleCompilationError(IncrementalCompilerImpl.scala:332)
[info] [error] 	at sbt.internal.inc.IncrementalCompilerImpl.compileIncrementally(IncrementalCompilerImpl.scala:425)
[info] [error] 	at sbt.internal.inc.IncrementalCompilerImpl.compile(IncrementalCompilerImpl.scala:137)

I can' tell if this is uncovering some changes Zinc made in 1.10.0 (https://eed3si9n.com/sbt-1.10.0-beta), or uncovering some issue with the compiler bridge.

[bishabosha]

@eed3si9n on the current main branch with sbt 1.9.9, I printed out the relations after a/compile in that task and it seems some classes are missing (such as a.A.Inner_sel):

class names: Relation [
  ${BASE}/a/src/main/scala/a/A.java -> a.A
  ${BASE}/a/src/main/scala/a/A.java -> a.A$.Inner
  ${BASE}/a/src/main/scala/a/A.java -> a.A$.Inner_sel
  ${BASE}/a/src/main/scala/a/AImport.java -> a.AImport
  ${BASE}/a/src/main/scala/a/AImport.java -> a.AImport.Inner
]

here are the set of apis.allInternalClasses:

[
  a.A,
  a.AImport.Inner,
  a.A$.Inner_sel,
  a.A.Inner_sel,
  a.AImport,
  a.A$.Inner,
  a.A.Inner,
  a.AImport$.Inner
]

[eed3si9n]

@bishabosha Thanks. To clarify, you mean it's there in sbt 1.9.9, but those entries that should be coming from Java sources are missing in sbt 1.10.0-RC2, right?

[bishabosha]

@eed3si9n both those outputs are with sbt 1.9.9. From what I understand (please verify), the class names relation comes from generatedNonLocalClass callback, and allInternalClasses are just whatever was recorded by ExtractAPI phase - so should these contain the same classes?

[eed3si9n]

https://github.com/scala/scala3/blob/05114dd6bcd007a6becc83011fe9916d35ba4716/sbt-test/pipelining/Yjava-tasty-fromjavaobject/a/src/main/scala/a/A.java

If Java compiler works like Scala compiler, I guess javac only emits a.A$.Inner_sel.class, but it generates some forwarder so humans can write a.A.Inner_sel?

[SethTisue]

The scripted tests are now running afoul of SIP-51 😆 I'll update it soon-ish

[mbovel]

Out of curiosity, why not also updating the sbt version in tests and community projects as you did in 19a6990?

[SethTisue]

oh, I suppose I'd have gotten to that before undraft-ing it

anyway, I've just updated it

if CI still fails, I won't have time to look at it until October, so if somebody's in more of a hurry than that, please feel free to take it over

[SethTisue]

I don't understand why the community build failure:

[error] -- [E038] Declaration Error: /__w/scala3/scala3/community-build/community-projects/fastparse/scalaparse/test/src-jvm/scalaparse/ScalacParser.scala:46:19 
[error] 46 |      override def incompleteInputError(msg: String) = {
[error]    |                   ^
[error]    |method incompleteInputError has a different signature than the overridden declaration
[error]    |
[error]    | longer explanation available when compiling with `-explain`
[error] -- [E038] Declaration Error: /__w/scala3/scala3/community-build/community-projects/fastparse/scalaparse/test/src-jvm/scalaparse/ScalacParser.scala:49:19 
[error] 49 |      override def syntaxError(offset: Offset, msg: String) = {
[error]    |                   ^
[error]    |method syntaxError has a different signature than the overridden declaration
[error]    |
[error]    | longer explanation available when compiling with `-explain`
[error] two errors found
[error] (scalaparse / Test / compileIncremental) Compilation failed

is somehow being surfaced in this PR, as the compilation failure is reproducible outside of the community build context, even on sbt 1.6.2, at dotty-staging/fastparse@8b934380 , with ++3.6.1!; scalaparse / Test / compile

however the problem does not reproduce if I ++3.6.0! instead of ++3.6.1!

@WojciechMazur do you have any hunches about what could be going on here? why on earth would 3.6.0 vs 3.6.1 matter? why is this PR affected, but I don't see the same failure in any of the sampling of other recent PRs that I looked at? I'm willing to go digging into it myself, but I thought I'd ask you first, since you have so much experience troubleshooting these sorts of problems —  is there is anything here that is familiar to you, any clues you see?

[WojciechMazur]

Is it possible that sbt has changed the version of scala-compiler dependency? It's used directly in the test to access UnitParser which in latest version of Scala 2.13 defines override def syntaxError(offset: Offset, msg: String, actions: List[CodeAction]) which might explain the wrong signature.
It might be caused by changes to support unfreezing Scala 2 stdlib. Can you check the actual classpath used by the Scala 3 compiler?

[SethTisue]

@WojciechMazur good call. with ++3.6.0! the Scala 2.13.0 compiler is on the classpath, whereas with ++3.6.1! the Scala 2.13.15 compiler is on the classpath. and the test expects 2.13.0

but it's baffling to me why 3.6.0 vs 3.6.1 would make a difference; the scala3-library and scala3-compiler POMs for those versions differ only by version numbers

It might be caused by changes to support unfreezing Scala 2 stdlib

except the problem is reproducible even on sbt 1.6.2 😱

but anyway, my hunch is that it isn't worth getting all the way to the bottom of this

I've pushed commits that update our fork of fastparse to use 2.13.15 instead of 2.13.0, which is the straightforward fix... hopefully CI will like it

[SethTisue]

one CI failure is because I'll need to rebase to pull in #21895

[hamzaremmal]

LGTM, I based my review on #19581

[hamzaremmal]

why not 1.10.5 and not 1.10.6 here ?

[SethTisue]

it's intentional. 1.10.4 is the newest version. there isn't always a new compiler-interface for every new sbt version, as per https://github.com/sbt/zinc/releases/

[hamzaremmal]

Is this changes necessary to be included in this PR (I doubt) ?

[SethTisue]

it is needed or the community build fails. there is some discussion with Wojciech about it above. I had to upgrade the project from Scala 2.13.0 to 2.13.15.