Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feat](metatable) support table$partitions for hive table (#40774) #41230

Merged
merged 3 commits into from
Sep 25, 2024

Conversation

morningman
Copy link
Contributor

bp #40774
and pick part of #34552, add isPartitionedTable() interface in TableIf

Support new grammar:  `table_name$partitions`

User can query partition info by using:

```
select * from table_name$partitions
```

`table_name$partitions` is a special meta table corresponding to the
table.

Its schema is the partition columns of the table, and column type is
always "String".
And the value is the partition column's value

You can use `DESC table_name$partitions` to get schema:

```
mysql> desc partition_values_all_types$partitions;
+-------+----------------+------+------+---------+-------+
| Field | Type           | Null | Key  | Default | Extra |
+-------+----------------+------+------+---------+-------+
| p1    | boolean        | Yes  | true | NULL    | NONE  |
| p2    | tinyint        | Yes  | true | NULL    | NONE  |
| p3    | smallint       | Yes  | true | NULL    | NONE  |
| p4    | int            | Yes  | true | NULL    | NONE  |
| p5    | bigint         | Yes  | true | NULL    | NONE  |
| p6    | date           | Yes  | true | NULL    | NONE  |
| p7    | datetime(6)    | Yes  | true | NULL    | NONE  |
| p8    | varchar(65533) | Yes  | true | NULL    | NONE  |
| p9    | decimal(9,2)   | Yes  | true | NULL    | NONE  |
| p10   | decimal(18,10) | Yes  | true | NULL    | NONE  |
| p11   | decimal(38,9)  | Yes  | true | NULL    | NONE  |
+-------+----------------+------+------+---------+-------+
```
Where `px` are partition columns of table `partition_values_all_types`;

```
mysql> select * from partition_values_all_types$partitions order by p1,p2,p3;
+------+------+--------+-------------+----------------------+------------+----------------------------+--------+-------------+----------------------+------------------------------------------+
| p1   | p2   | p3     | p4          | p5                   | p6         | p7                         | p8     | p9          | p10                  | p11                                      |
+------+------+--------+-------------+----------------------+------------+----------------------------+--------+-------------+----------------------+------------------------------------------+
| NULL | NULL |   NULL |        NULL |                 NULL | NULL       | NULL                       | NULL   |        NULL |                 NULL |                                     NULL |
|    0 | -128 | -32768 | -2147483648 | -9223372036854775808 | 1900-01-01 | 1899-01-01 23:59:59.000000 | NULL   | -9999999.99 | -99999999.9999999999 | -99999999999999999999999999999.999999999 |
|    0 |  127 |  32767 |  2147483647 |  9223372036854775807 | 9999-12-31 | 0001-01-01 00:00:01.321000 | boston |  9999999.99 |  99999999.9999999999 |  99999999999999999999999999999.999999999 |
+------+------+--------+-------------+----------------------+------------+----------------------------+--------+-------------+----------------------+------------------------------------------+
```

Currently, this grammar can only be used for partition table's in Hive
Catalog.
Table in other catalogs or non partition table can not use this grammar.

Internally, it is implemented as a table valued function:
`partition_values`

```
mysql> select * from partition_values("catalog" = "hive", "database" = "multi_catalog", "table" = "partition_values_all_types");
+------+------+--------+-------------+----------------------+------------+----------------------------+--------+-------------+----------------------+------------------------------------------+
| p1   | p2   | p3     | p4          | p5                   | p6         | p7                         | p8     | p9          | p10                  | p11                                      |
+------+------+--------+-------------+----------------------+------------+----------------------------+--------+-------------+----------------------+------------------------------------------+
|    0 |  127 |  32767 |  2147483647 |  9223372036854775807 | 9999-12-31 | 0001-01-01 00:00:01.321000 | boston |  9999999.99 |  99999999.9999999999 |  99999999999999999999999999999.999999999 |
|    0 | -128 | -32768 | -2147483648 | -9223372036854775808 | 1900-01-01 | 1899-01-01 23:59:59.000000 | NULL   | -9999999.99 | -99999999.9999999999 | -99999999999999999999999999999.999999999 |
| NULL | NULL |   NULL |        NULL |                 NULL | NULL       | NULL                       | NULL   |        NULL |                 NULL |                                     NULL |
+------+------+--------+-------------+----------------------+------------+----------------------------+--------+-------------+----------------------+------------------------------------------+
```
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@morningman
Copy link
Contributor Author

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.14% (9343/25853)
Line Coverage: 27.67% (76767/277426)
Region Coverage: 26.43% (39393/149047)
Branch Coverage: 23.22% (20054/86364)
Coverage Report: http://coverage.selectdb-in.cc/coverage/63d7efa3a5cc8a374a160aa57be4c6f52c284c36_63d7efa3a5cc8a374a160aa57be4c6f52c284c36/report/index.html

@morningman
Copy link
Contributor Author

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.15% (9346/25853)
Line Coverage: 27.68% (76785/277427)
Region Coverage: 26.43% (39401/149049)
Branch Coverage: 23.22% (20057/86366)
Coverage Report: http://coverage.selectdb-in.cc/coverage/212cdc120be2cdafc5c38ef57990e89e9b3ca1cf_212cdc120be2cdafc5c38ef57990e89e9b3ca1cf/report/index.html

@morningman morningman merged commit 5b3b2ce into apache:branch-2.1 Sep 25, 2024
19 of 21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants