Scala 3 port

[Katrix]

Fixes #1043

This PR starts the process of porting Shapeless 2 to Scala 3. It moves the code that uses Scala 2 macros to it's own Scala 2 specific compilation folder, and re-implements these features where possible with Scala 3 macros and features instead. Where it is not yet possible to port a feature to Scala 3, I have either removed it, moved it to be Scala 2 specific, or left it hanging without an implementation depending on how critical the feature is and what would be needed to support in in Scala 3. What remains is to fix the tests so that they compile for Scala 3, and report bugs to the Scala 3 compiler where they are found. Bellow you can find a finer breakdown of the changes that have been made, and what remains.

The build itself

Scala 2.12 has been dropped, and Scala native has been temporarily disabled until I take a further look at how to enable it for Scala 2.13 only.

This PR uses traits to mix in Scala 2/3 behavior into the objects where this is needed. Where the original type is called foobar, the mixin with the macro/ported code is called foobarScalaCompat. These files can be found in the scala-2 and scala-3 source directories.

Major changes

Here are some of the major changes made while porting Shapeless to Scala 3

• Type providers of all sorts have been removed. Think Union, Witness, Record, HList, Coproduct. To construct records and unions, the type ->> has been added as a shorthand for FieldType

• Things that transform one method call to another. Think RecordArgs, FromRecordArgs, NatProductArgs, ProductArgs, FromProductArgs, SingletonProductArgs. These macros generally convert calls to a method into calls to a different method, while manipulating the arguments. For example for RecordArgs, lhs.method(x = 23, y = "foo", z = true) becomes lhs.methodRecord("x" ->> 23 :: "y" ->> "foo", "z" ->> true).

• Lazy has been removed in favor of by-name implicits. Strict is no longer needed.

• Generic, LabelledGeneric and friends use Mirror in Scala 3. This severly cuts down on where they can be used. This is something that needs to be implemented later, or fixed in the Scala 3 compiler.

• NatWith and WitnessWith have been removed. Where they were used and could not be replaced, a much more restricted version has been introduced instead.

• Witness has been removed in favor of singleton types and ValueOf.

Unimplemented or broken things in the library

• Widen currently lays unimplemented in Scala 3
• Generic1 currently relies on a typeclass SplitCons that splits a higher kinded type H :: T or H :+: T into head and tail. This does not seem to be possible to implement currently, and needs this to work Quoting type in a pattern doesn't work with higher-kinded types scala/scala3#10864
• Split1 is currently unimplemented
• everywhere has exponential runtime with the depth of the structure used in Scala 3
• Nat math typeclasses don't want to resolve, supposedly because of diverging implicits. Need to figure out if the fault lies with Shapeless or Scala 3. It would be nice if we could leverage the Scala 3 compiletime ops for faster compilations here as well.
• Many givens are currently defined as inline because of Inline (not transparent) with match type alters typechecking scala/scala3#13250. That should be removed once the bug is fixed.
• Fix the ScalaDocs so they don't reference removed things like type providers

Tests

The tests (and examples) are the nr 1 thing why there's still a lot of work to do here. While the tests compile just fine in Scala 2, they fail pretty clearly in Scala 3. Some of this is just stuff that needs fixing. In many other cases however, it's less certain if the error is in the Scala 3 compiler, or in shapeless. Sometimes just poking and (un)inlining pieces of code can fix them, while in other cases there's a lot more work to do. That is where the majority of the work will go to finish this PR.

There is probably around 200 compile errors for the examples and tests combined. As Nat operations do not currently work (them not working technically falls under this, but I mentioned them above as they affect so much code) that number is likely a bit larger than the true amount of failing code. Each of these compile errors need to be inspected to find out if it's a Scala 3 bug, or a shapeless bug. If it is a shapeless bug, how can it be fixed.

From that further big things that need to be fixed might be found.

What actually works

While the above may seem a bit grim, there are still many things that do work. Deriving typeclasses the good old way with Generic is one of them, and probably the most important one.

[julienrf]

Is there anyone interested in reviewing this contribution?

[milessabin]

Is there anyone interested in reviewing this contribution?

For sure, but this is still marked as a work in progress. Are there specific areas that you or @Katrix would like early feedback on?

I think in many ways the real test of the success or otherwise of this project would be if you can take an existing shapeless 2 using project and build it for Scala 3. I'm not sure what a suitable one would be ... maybe scalacheck-shapeless?

[SethTisue]

looks like CI is busted?

[joroKr21]

looks like CI is busted?

I think it doesn't run until conflicts are resolved