To.safe #3767
Draft
To.safe #3767
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jto
reviewed
Apr 20, 2021
| else Some(ls.collect { case Some(x) => x }) | ||
|
|
||
| private def interpret[T: scala.quoted.Type](using Quotes): Option[Schema[T]] = | ||
| Type.of[T] match { |
There was a problem hiding this comment.
(copying my comment here since I originally made it on your fork)
Unless I'm missing something this means that the possible set of supported schema types is effectively limited to those already defined in scio.
If a user adds a given instance of Schema for type that isn't supported (say for example a Java class), the derivation will simply ignore it. This could be a problem for aliaswa support in Schema but probably something we can live with. We don't really expect users to define their own instances of Schema.
I'd be curious to see how the implementation looks like for Java beans and Avro's SpecificRecord.
* Temporarily silence other jobs * Refactor scala3 version into a variable * Create ci job testing and compiling each subproject * Disable fail-fast * Use RC1 syntax for givens * Fix isJavaBean * Fix shadowing of BitSetCoder by private class * Remove call to showAnsiColored which does not exist anymore * Fix ambiguous resolution of implicit * Disable subprojects that don't build * Disable scio-core Co-authored-by: vincenzobaz <[email protected]>
…rides scio-macros depends only on java libs and scalatest. Scalatest is now available as a 3.0.0 lib as well, so withDottyCompat is not needed anymore.
To match the Scala 2 signature
More general types must be at the bottom
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR proposes a solution to the To.safe blocker in Scala 3 on which @MaximeKjaer, @liufengyun and I worked.
It took us some time to get familiar with the API and we tested different strategies before landing on this code, but we think it shows the strengths of Scala 3 macros and reflection by combining the two APIs in a relatively concise and idiomatic function that does not need
evalYou can see a smaller diff here
The critical section is the file
scio-core/src/main/scala-3/com/spotify/scio/schemas/To.scalawhere we use Scala 3 macros to pattern match on the type argumentTofExpr[Schema[T]]to derive aSchema[T]at compile time which can then be used to invokeTo.checkCompatibility. To deal with case class and compile time derivedSchemas we rely on the reflection API which allows us to access the names and types of the fields of the case class.You can see this the implementation here
This draft pr brings along many changes, in particular to build and syntax, that we adopted to use RC2, sbt 1.5, new syntax and to reduce the number of warnings.
We would like to have your opinion on this approach.
A few steps might be needed to finalize this pr:
caseFieldsand<: Productis a strong-enough conditionmainas it progresses.