From 0be8eaebcf27afae9ecda8ab79db63c214426561 Mon Sep 17 00:00:00 2001 From: Benchao Li Date: Mon, 6 Nov 2023 14:56:12 +0800 Subject: [PATCH] [CALCITE-6047] Release Calcite 1.36.0 Close apache/calcite#3505 --- README | 2 +- site/_docs/history.md | 348 +++++++++++++++++++++++++++++++++++++++++- site/_docs/howto.md | 4 +- 3 files changed, 345 insertions(+), 9 deletions(-) diff --git a/README b/README index 8f12ab9c9a8..f94b69fc27e 100644 --- a/README +++ b/README @@ -1,4 +1,4 @@ -Apache Calcite release 1.35.0 +Apache Calcite release 1.36.0 This is a source distribution of Apache Calcite. diff --git a/site/_docs/history.md b/site/_docs/history.md index 8aa9fdbd366..79db5b38098 100644 --- a/site/_docs/history.md +++ b/site/_docs/history.md @@ -29,10 +29,10 @@ Downloads are available on the [downloads page]({{ site.baseurl }}/downloads/). + +## 1.36.0 / 2023-11-XX +{: #v1-36-0} + +This release comes 3 months after [1.35.0](#v1-35-0), +contains contributions from 30 contributors, and resolves 125 issues. + +Among other new features, it's worth highlighting the adding of 30 new SQL functions in various libraries such as BigQuery and Spark, many improvements hardening `TABLESAMPLE` , and also the following features: +* [CALCITE-129] + Support recursive `WITH` queries +* [CALCITE-6022] + Support `CREATE TABLE ... LIKE DDL` in server module +* [CALCITE-5962] + Support parse Spark-style syntax `LEFT ANTI JOIN` in Babel parser +* [CALCITE-5184] + Support `LIMIT start, ALL` in MySQL conformance, equivalent to `OFFSET start` +* [CALCITE-5889] + Add a `RelRule` that converts `Minus` into `UNION ALL..GROUP BY...WHERE` + +In addition to new features, it's also worth highlighting the integrating of SQL Logic Test suite + +Contributors to this release: +Benchao Li (release manager), +Cancai Cai, +Claude Brisson, +Evgeniy Stanilovskiy, +Hanumath Maduri, +Hongyu Guo, +Itiel Sadeh, +Jerin John, +Jiajun Xie, +Julian Hyde, +Kaustubh Beedkar, +Lei Shen, +Leonid Chistov, +Michael Mior, +Mihai Budiu, +Mingcan Wang, +Oliver Lee, +Ran Tao, +Ruben Quesada Lopez, +Runkang He, +Tanner Clary, +Thomas Rebele, +Tim Nieradzik, +Wegdan Ghazi, +Wenrui Meng, +Xiaogang Zhou, +ZhangJian He, +Zhengqiang Duan, +Zoltan Haindrich, +zoovwang. + #### Breaking Changes {: #breaking-1-36-0} @@ -51,18 +127,278 @@ other software versions as specified in gradle.properties. #### New features {: #new-features-1-36-0} +* Supporting new SQL functions in BigQuery, Hive and Spark libraries: + * [CALCITE-5826] + Add `FIND_IN_SET` function (enabled in Hive and Spark libraries) + * [CALCITE-5979] + Enable `REGEXP_REPLACE` function in BigQuery library + * [CALCITE-6077] + Add `FACTORIAL` function (enabled in Hive and Spark libraries) + * [CALCITE-5918] + Add `MAP` function (enabled in Spark library) + * [CALCITE-5825] + Add `URL_ENCODE` and `URL_DECODE` function (enabled in Spark library) + * [CALCITE-6021] + Add `CURRENT_DATETIME` function (enabled in BigQuery library) + * [CALCITE-5993] + Add `CODE_POINTS_TO_STRING`, `TO_CODE_POINTS` function (enabled in BigQuery library) + * [CALCITE-5978] + Add `REGEXP_INSTR` function (enabled in BigQuery library) + * [CALCITE-5935] + Add `CODE_POINTS_TO_BYTES` function (enabled in BigQuery library) + * [CALCITE-5933] + Add `SAFE_DIVIDE` function (enabled in BigQuery library) + * [CALCITE-5821] + Add `FORMAT_NUMBER` function (enabled in Hive and Spark library) + * [CALCITE-5911] + Add `REGEXP_EXTRACT_ALL` function (enabled in BigQuery library) + * [CALCITE-5910] + Add `REGEXP_EXTRACT` and `REGEXP_SUBSTR` functions (enabled in BigQuery library) + * [CALCITE-5766] + Add `SAFE_NEGATE` function (enabled for BigQuery library) + * [CALCITE-5770] + Add `SAFE_SUBTRACT` function (enabled in BigQuery library) + * [CALCITE-5848] + Add `BIT_GET` and `GETBIT` functions (enabled in Spark library) + * [CALCITE-5644] + Add `CONTAINS_SUBSTR` function (enabled in BigQuery library) + * [CALCITE-5640] + Add `SAFE_ADD` function (enabled in BigQuery library) + * [CALCITE-5830] + Add `ARRAY_INSERT` function(enabled in Spark library) + * [CALCITE-5873] + Add `REGEXP_CONTAINS` function (enabled in BigQuery library) + * [CALCITE-5827] + Add `IS_INF` and `IS_NAN` functions (enabled in BigQuery library) + * [CALCITE-5831] + Add `SOUNDEX` function (enabled in Spark library) + * [CALCITE-5735] + Add `SAFE_MULTIPLY` function (enabled for BigQuery) + * [CALCITE-5820] + Add `PARSE_URL` function (enabled in Hive and Spark library) + * [CALCITE-5851] + Add `LEVENSHTEIN` function (enabled in Hive and Spark library) +* [CALCITE-129] + Support recursive `WITH` queries +* [CALCITE-6011] + Add `FilterWindowTransposeRule` to push a `Filter` past a `Window` +* [CALCITE-6038] + Remove `ORDER BY ... LIMIT n` when input has at most one row, n >= 1, and there is no `OFFSET` clause +* [CALCITE-6022] + Support `CREATE TABLE ... LIKE DDL` in server module +* [CALCITE-6031] + Add the planner rule that pushes `Filter` past `Sample` +* [CALCITE-4189] + Simplify `p OR (p IS NOT TRUE)` to `TRUE` +* [CALCITE-6009] + Add optimization to remove redundant `LIMIT` that is more than input row count +* [CALCITE-5570] + Support nested map type for `SqlDataTypeSpec` +* [CALCITE-5962] + Support parse Spark-style syntax `LEFT ANTI JOIN` in Babel parser +* [CALCITE-5940] + Add a `RelRule` to merge `Limit` +* [CALCITE-5971] + Add `SampleToFilterRule` to rewrite bernoulli `Sample` to `Filter` +* [CALCITE-5994] + Add optimization rule to remove `Sort` when its input's row number is less or equal to one +* [CALCITE-5836] + Implement Rel2Sql for `MERGE` +* [CALCITE-5889] + Add a `RelRule` that converts `Minus` into `UNION ALL..GROUP BY...WHERE` +* [CALCITE-5944] + Add metadata for `Sample` +* [CALCITE-5941] + Support `LITERAL_AGG` in `Interpreter` +* [CALCITE-985] + Validate `MERGE` +* [CALCITE-5870] + Allow literals like `DECIMAL '12.3'` (consistent with Postgres) +* [CALCITE-5916] + In `RelBuilder`, add `sample()` method (equivalent to SQL `TABLESAMPLE` clause) +* [CALCITE-5184] + Support `LIMIT start, ALL` in MySQL conformance, equivalent to `OFFSET start` + #### Dependency version upgrade {: #dependency-1-36-0} +* [CALCITE-5763] + Increase minimum Guava version to 21.0, maximum version to 32.1.3-jre, and stop building on Guava 19.0 +* [CALCITE-5938] + Update HSQLDB to Version 2.7.2 (using JDK8 JAR, default supports JDK11+) +* [CALCITE-6004] + Replace deprecated mongo-java-driver dependency +* [CALCITE-5966] + Upgrade commons-dbcp2 to 2.9.0 + #### Bug-fixes, API changes and minor enhancements {: #fixes-1-36-0} +* [CALCITE-6088] + `SqlItemOperator` fails in `RelToSqlConverter` +* [CALCITE-5863] + Calcite rejects valid query with multiple `ORDER BY` columns and constant `RANGE` bounds in window functions +* [CALCITE-5984] + Allow disabling field trimming in `Prepare` via `SqlToRelConverter.Config#isTrimUnusedFields` +* [CALCITE-5990] + Explicit cast to numeric type doesn't check overflow +* [CALCITE-6052] + `SqlImplementor` writes `REAL`, `FLOAT`, or `DOUBLE` literals as `DECIMAL` literals +* [CALCITE-6041] + `MAP` sub-query gives `NullPointerException` +* [CALCITE-6037] + The function category of `ARRAY`/`EXTRACT_VALUE`/`XML_TRANSFORM`/`EXTRACT_XML`/`EXISTSNODE` is incorrect +* [CALCITE-6024] + A more efficient implementation of `SqlOperatorTable`, backed by an immutable multi-map keyed by upper-case operator name +* [CALCITE-5949] + `RexExecutable` should return unchanged original expressions when it fails +* [CALCITE-6013] + `RelBuilder` should simplify plan by pruning unused measures +* [CALCITE-6040] + The operand type inference of `SqlMapValueConstructor` is incorrect +* [CALCITE-6030] + `DATE_PART` is not handled by the `RexToLixTranslator` +* Following [CALCITE-5570] + Support nested map type for `SqlDataTypeSpec` +* [CALCITE-6050] + Add interface `ImmutablePairList` +* [CALCITE-5950] + `DEFAULT` expression is ignored during `INSERT` +* [CALCITE-6006] + `RelToSqlConverter` loses charset information +* [CALCITE-5948] + Use explicit casting if element type in `ARRAY`/`MAP` does not equal derived component type +* [CALCITE-5989] + Type inference for `RPAD` and `LPAD` functions (BIGQUERY) is incorrect +* [CALCITE-5982] + Allow implementations of `CalciteMeta` to return extra columns in their responses to `DatabaseMetaData.getTables` and `getColumns` requests +* [CALCITE-6007] + Sub-query that contains `WITH` and has no alias generates invalid SQL after expansion +* [CALCITE-6003] + `JSON_ARRAY()` with no arguments does not unparse correctly +* [CALCITE-6026] + MongoDB: Column is not quoted in `ORDER BY` clause and throws JsonParseException +* [CALCITE-6005] + `POLYGON` string representation is different on Apple silicon +* [CALCITE-5974] + Elasticsearch adapter throws `ClassCastException` when index mapping sets `dynamic_templates` without `properties` +* [CALCITE-5995] + `JSON_VALUE`, `JSON_EXISTS`, `JSON_QUERY` functions should cache generated objects between calls +* [CALCITE-5960] + `CAST` throws NullPointerException if `SqlTypeFamily` of targetType is null +* [CALCITE-5997] + `OFFSET` operator is incorrectly unparsed +* [CALCITE-5961] + Type inference of `ARRAY_COMPACT` is incorrect +* [CALCITE-5999] + `DECIMAL` literals as sometimes unparsed looking as `DOUBLE` literals +* [CALCITE-5988] + `SqlImplementor.toSql` cannot emit `VARBINARY` literals +* [CALCITE-5996] + `TRANSLATE` operator is incorrectly unparsed +* [CALCITE-5862] + Incorrect semantics of `ARRAY` function (Spark library) when elements have Numeric and Character types +* [CALCITE-5931] + Allow round decimals like `1.00` in window ranges +* [CALCITE-5732] + `EnumerableHashJoin` and `EnumerableMergeJoin` on composite key return rows matching condition `NULL = NULL` +* [CALCITE-5967] + `UnsupportedOperationException` while implementing a call that requires a special collator +* [CALCITE-5952] + `SemiJoinJoinTransposeRule` should check if JoinType supports pushing predicates into its inputs +* [CALCITE-5953] + `AggregateCaseToFilterRule` may make inaccurate `SUM` transformations +* [CALCITE-5861] + `ReduceExpressionsRule` rules should constant-fold expressions in window bounds +* [CALCITE-5965] + Avoid unnecessary String concatenations in the `RexFieldAccess` constructor to improve the performance +* [CALCITE-5914] + Cache compiled regular expressions in SQL function runtime +* Refactor: In `ReflectUtil`, add methods isStatic and isPublic +* Refactor: In `RexImpTable`, ensure that every method is in BuiltInMethod +* [CALCITE-5922] + The SQL generated for the `POSITION` function(with 3 input arguments) by the `SparkSqlDialect` is not recognized by Spark SQL +* [CALCITE-5920] + Reset `PERCENTILE_CONT`/`PERCENTILE_DISC` to reserved keywords +* [CALCITE-5946] + `TimeString` should allow fractional seconds ending in zero +* [CALCITE-5906] + JDBC adapter should generate `TABLESAMPLE` +* [CALCITE-5895] + `TABLESAMPLE (0)` should return no rows +* [CALCITE-5813] + Type inference for sql functions `REPEAT`, `SPACE`, `XML_TRANSFORM`, and `XML_EXTRACT` is incorrect +* [CALCITE-5908] + Refactor: Remove unnecessary null checks in `CalciteSchema` +* [CALCITE-5843] + Constant expression with nested casts causes a compiler crash +* [CALCITE-5885] + `SqlNode#toSqlString()` does not honor dialect's `supportsCharSet()` flag on nested types +* [CALCITE-5869] + `LEAST_RESTRICTIVE` does not use inner type of `MEASURE` for comparisons +* [CALCITE-5903] + `RelMdCollation` does not define collations for `EnumerableLimit` +* [CALCITE-5882] + Compile-time evaluation of `SPLIT` function returns incorrect result +* [CALCITE-5879] + `AssertionError` during constant reduction of `SPLIT` expression that returns null +* [CALCITE-5875] + Remove unnecessary null checks in Redis adapter +* [CALCITE-5859] + Compile-time evaluation of `LEFT(NULL, n)` should not throw `RuntimeException` +* [CALCITE-5837] + `RexUtil#pullFactors` output's order should be deterministic even when the `RexNode` kind is `OR` +* [CALCITE-5877] + `AssertionError` during `MOD` operation if result scale is greater than maximum numeric scale +* [CALCITE-5841] + Improve singleton implementation for `ChinookAvaticaServer` in calcite-plus +* Following [CALCITE-5688] + Eliminate warnings in server parser + #### Build and test suite {: #build-1-36-0} +* [CALCITE-5921] + `SqlOperatorFixture.checkFails` and `checkAggFails` don't check runtime failure +* [CALCITE-5923] + `SqlOperatorTest` using `safeParameters` are not using overridable fixture +* [CALCITE-6014] + Create a `SqlOperatorFixture` that parses, unparses, and then parses again before executing +* Incorrect test fixture used by `SqlOperatorTest.testLeastFunc` +* [CALCITE-5980] + QuidemTests are not effectively executed on Windows +* [CALCITE-5615] + Run SQL Logic Test suite using Calcite's HSQLDB JDBC adapter +* [CALCITE-5909] + `SqlParserTest.testNoUnintendedNewReservedKeywords` fails in IDE while passes in command line +* Refactor `RelDataTypeSystemTest` to use test fixture +* Add various lint checks +* Code style: lint +* In `Puffin`, allow an action to test whether it is looking at the last line of a source + #### Web site and documentation {: #site-1-36-0} ---> + +* Site: Add Runkang He as committer +* Site: Add Hongyu Guo as committer +* Site: Add Lei Shen as committer +* Site: Add Ran Tao as committer +* Site: Add Mihai Budiu as committer +* Site: Add Apache Wayang (incubating) to powered-by page +* [CALCITE-5884] + `ARRAY_TO_STRING` function should return `NULL` if its `nullValue` argument is `NULL` +* [CALCITE-6075] + Site: Cloning source code from GitHub using git protocol fails +* Add example for `MAP` type in reference docs +* [CALCITE-6033] + Correct broken links on adapter page +* [CALCITE-6017] + Update the GitHub link of released versions +* [CALCITE-5905] + Documentation for `CREATE TYPE` is incorrect +* Remove mentions of binary distribution from README +* Update broken link in `RelMetadataProvider` Javadoc +* The parameter names of `SqlTypeMappingRules.Builder#add` are misleading ## 1.35.0 / 2023-07-26 {: #v1-35-0} @@ -504,8 +840,8 @@ xinqiu.hu, Zou Dan. #### Breaking Changes diff --git a/site/_docs/howto.md b/site/_docs/howto.md index 476ee2b7797..0a2057a9e3f 100644 --- a/site/_docs/howto.md +++ b/site/_docs/howto.md @@ -39,8 +39,8 @@ Unpack the source distribution `.tar.gz` file, then build using Gradle: {% highlight bash %} -$ tar xvfz apache-calcite-1.35.0-src.tar.gz -$ cd apache-calcite-1.35.0-src +$ tar xvfz apache-calcite-1.36.0-src.tar.gz +$ cd apache-calcite-1.36.0-src $ gradle build {% endhighlight %}