[CALCITE-6348] ARRAY_OVERLAP with a NULL argument crashes the compiler #3746
Conversation
| @@ -7126,6 +7126,9 @@ void checkRegexpExtract(SqlOperatorFixture f0, FunctionAlias functionAlias) { | |||
| "Cannot apply 'ARRAYS_OVERLAP' to arguments of type 'ARRAYS_OVERLAP\\(" | |||
| + "<INTEGER ARRAY>, <BOOLEAN>\\)'. Supported form\\(s\\): 'ARRAYS_OVERLAP\\(" | |||
| + "<EQUIVALENT_TYPE>, <EQUIVALENT_TYPE>\\)'", false); | |||
| f.checkNull("arrays_overlap(null, null)"); | |||
There was a problem hiding this comment.
@mihaibudiu Are you sure spark supports this writing method? My test failed in spark 3.4.
There was a problem hiding this comment.
I am not sure what you mean by that. This is a legal expression which shouldn't crash the compiler.
There was a problem hiding this comment.
I am not sure what you mean by that. This is a legal expression which shouldn't crash the compiler.
It seems to be a bug in spark. I can help you test it on the latest code branch of spark to see if it can be fixed.
There was a problem hiding this comment.
@mihaibudiu I tested it on the spark master branch. It seems that the parameters in arrays_overlap are not allowed to be null, which will fail at compile time
Is this a bug in spark sql?
There was a problem hiding this comment.
which program crashes?
There was a problem hiding this comment.
Sorry. There may be something wrong with my expression. It should not be said that the program crashed, but that the execution failed.
What do you think？
There was a problem hiding this comment.
Whoever wrote that code is confusing the type "NULL" with the type "VOID". These are not the same thing at all. NULL is a type which has one value, whereas VOID has no values at all.
So there is at least one bug in their handling of types.
We can also reject this expression at compilation time in Calcite.
What happens if both arrays are NULL?
There was a problem hiding this comment.
The situation of arrays_overlap(null, null) is as follows
There was a problem hiding this comment.
then it's easy, we just reject arguments of type NULL for this function.
I will rework the implementation.
Signed-off-by: Mihai Budiu <[email protected]>
|
No description provided.