You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
ColumnPosition could be first and after, null value means last in Graviton now. In the perspective of programing it's confusing.
For SparkCatalog in Iceberg, null position is not allowed.
For end user, if ColumnPosition is not defined, GravitonServer should fill a default value ColumnPosition.LAST. ColumnPosition is null is illegal in the Graviton Server.
The text was updated successfully, but these errors were encountered:
I hold the view that Gravitino should not fill in the null position by ColumnPosition.LAST as default.
Instead, it should be the responsibility of different catalogs to handle null values.
This is due to the different ways in which catalogs handle the null position. For example, Hive defaults to adding new columns to the end of non-partitioned columns(#871 did as the Hive default behavior), while Iceberg adds them to the last column (native Iceberg throws an exception, but #383 made an implicit conversion).
A uniform default value fill would make it difficult for catalogs to distinguish between "Last" and null. @jerryshao What's your opinion?
### What changes were proposed in this pull request?
Introduce a default column position for unspecified positions when
adding a column.
### Why are the changes needed?
Passing null values in code carries risks and uncertainties. Using
default values can solve this problem and improve the robustness of the
code.
Fix: #401
### Does this PR introduce _any_ user-facing change?
no, users can still use null, but it will be internally converted to the
default value.
### How was this patch tested?
adding new UTs
What would you like to be improved?
ColumnPosition could be
first
andafter
, null value meanslast
in Graviton now. In the perspective of programing it's confusing.For SparkCatalog in Iceberg, null position is not allowed.
How should we improve?
For end user, if ColumnPosition is not defined, GravitonServer should fill a default value ColumnPosition.LAST. ColumnPosition is null is illegal in the Graviton Server.
The text was updated successfully, but these errors were encountered: