Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[feat](metatable) support table$partitions for hive table (#40774) #41230

Merged
merged 3 commits into from
Sep 25, 2024
Merged

[feat](metatable) support table$partitions for hive table (#40774) #41230

merged 3 commits into from
Sep 25, 2024

Commits on Sep 24, 2024

  1. 1

    @morningman
    morningman committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    4f35d30 View commit details
    Browse the repository at this point in the history

  2. [feat](metatable) support table$partitions for hive table (apache#40774) 

    Support new grammar:  `table_name$partitions`

User can query partition info by using:

```
select * from table_name$partitions
```

`table_name$partitions` is a special meta table corresponding to the
table.

Its schema is the partition columns of the table, and column type is
always "String".
And the value is the partition column's value

You can use `DESC table_name$partitions` to get schema:

```
mysql> desc partition_values_all_types$partitions;
+-------+----------------+------+------+---------+-------+
| Field | Type           | Null | Key  | Default | Extra |
+-------+----------------+------+------+---------+-------+
| p1    | boolean        | Yes  | true | NULL    | NONE  |
| p2    | tinyint        | Yes  | true | NULL    | NONE  |
| p3    | smallint       | Yes  | true | NULL    | NONE  |
| p4    | int            | Yes  | true | NULL    | NONE  |
| p5    | bigint         | Yes  | true | NULL    | NONE  |
| p6    | date           | Yes  | true | NULL    | NONE  |
| p7    | datetime(6)    | Yes  | true | NULL    | NONE  |
| p8    | varchar(65533) | Yes  | true | NULL    | NONE  |
| p9    | decimal(9,2)   | Yes  | true | NULL    | NONE  |
| p10   | decimal(18,10) | Yes  | true | NULL    | NONE  |
| p11   | decimal(38,9)  | Yes  | true | NULL    | NONE  |
+-------+----------------+------+------+---------+-------+
```
Where `px` are partition columns of table `partition_values_all_types`;

```
mysql> select * from partition_values_all_types$partitions order by p1,p2,p3;
+------+------+--------+-------------+----------------------+------------+----------------------------+--------+-------------+----------------------+------------------------------------------+
| p1   | p2   | p3     | p4          | p5                   | p6         | p7                         | p8     | p9          | p10                  | p11                                      |
+------+------+--------+-------------+----------------------+------------+----------------------------+--------+-------------+----------------------+------------------------------------------+
| NULL | NULL |   NULL |        NULL |                 NULL | NULL       | NULL                       | NULL   |        NULL |                 NULL |                                     NULL |
|    0 | -128 | -32768 | -2147483648 | -9223372036854775808 | 1900-01-01 | 1899-01-01 23:59:59.000000 | NULL   | -9999999.99 | -99999999.9999999999 | -99999999999999999999999999999.999999999 |
|    0 |  127 |  32767 |  2147483647 |  9223372036854775807 | 9999-12-31 | 0001-01-01 00:00:01.321000 | boston |  9999999.99 |  99999999.9999999999 |  99999999999999999999999999999.999999999 |
+------+------+--------+-------------+----------------------+------------+----------------------------+--------+-------------+----------------------+------------------------------------------+
```

Currently, this grammar can only be used for partition table's in Hive
Catalog.
Table in other catalogs or non partition table can not use this grammar.

Internally, it is implemented as a table valued function:
`partition_values`

```
mysql> select * from partition_values("catalog" = "hive", "database" = "multi_catalog", "table" = "partition_values_all_types");
+------+------+--------+-------------+----------------------+------------+----------------------------+--------+-------------+----------------------+------------------------------------------+
| p1   | p2   | p3     | p4          | p5                   | p6         | p7                         | p8     | p9          | p10                  | p11                                      |
+------+------+--------+-------------+----------------------+------------+----------------------------+--------+-------------+----------------------+------------------------------------------+
|    0 |  127 |  32767 |  2147483647 |  9223372036854775807 | 9999-12-31 | 0001-01-01 00:00:01.321000 | boston |  9999999.99 |  99999999.9999999999 |  99999999999999999999999999999.999999999 |
|    0 | -128 | -32768 | -2147483648 | -9223372036854775808 | 1900-01-01 | 1899-01-01 23:59:59.000000 | NULL   | -9999999.99 | -99999999.9999999999 | -99999999999999999999999999999.999999999 |
| NULL | NULL |   NULL |        NULL |                 NULL | NULL       | NULL                       | NULL   |        NULL |                 NULL |                                     NULL |
+------+------+--------+-------------+----------------------+------------+----------------------------+--------+-------------+----------------------+------------------------------------------+
```
    @morningman
    morningman committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    63d7efa View commit details
    Browse the repository at this point in the history

  3. fix case

    @morningman
    morningman committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    212cdc1 View commit details
    Browse the repository at this point in the history