[VL] Refine cmake flags to decrease normal build time #3485
Conversation
|
Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues? https://github.com/oap-project/gluten/issues Then could you also rename commit message and pull request title in the following format? See also: |
|
Run Gluten Clickhouse CI |
Yohahaha
changed the title
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
dev/builddeps-veloxbe.sh
Outdated
| @@ -103,7 +103,7 @@ if [ "$SKIP_BUILD_EP" != "ON" ]; then | |||
| cd $GLUTEN_DIR/ep/build-velox/src | |||
| ./get_velox.sh --enable_hdfs=$ENABLE_HDFS --build_protobuf=$BUILD_PROTOBUF --enable_s3=$ENABLE_S3 | |||
| ./build_velox.sh --enable_s3=$ENABLE_S3 --build_type=$BUILD_TYPE --enable_hdfs=$ENABLE_HDFS \ | |||
| --enable_ep_cache=$ENABLE_EP_CACHE --build_benchmarks=$BUILD_BENCHMARKS | |||
| --enable_ep_cache=$ENABLE_EP_CACHE --build_tests=$BUILD_TESTS | |||
There was a problem hiding this comment.
Why do we remove build_benchmarks option here?
There was a problem hiding this comment.
I find we never use this option, and Gluten seems not need too, BUILD_TESTS is enough.
| --build_benchmarks=*) | ||
| ENABLE_BENCHMARK=("${arg#*=}") | ||
| --build_tests=*) | ||
| ENABLE_TESTS=("${arg#*=}") |
| COMPILE_OPTION="-DVELOX_ENABLE_PARQUET=ON" | ||
| if [ $ENABLE_BENCHMARK == "OFF" ]; then | ||
| COMPILE_OPTION="$COMPILE_OPTION -DVELOX_BUILD_TESTING=OFF -DVELOX_BUILD_TEST_UTILS=ON" | ||
| COMPILE_OPTION="-DVELOX_ENABLE_PARQUET=ON -DVELOX_BUILD_TESTING=OFF -DVELOX_BUILD_TEST_UTILS=OFF -DVELOX_ENABLE_DUCKDB=OFF" |
There was a problem hiding this comment.
I suppose parquet compilation is not needed for ORC users. Better to keep it as optional.
There was a problem hiding this comment.
previous parquet option is enabled by default, I keep it as original.
I suppose parquet compilation is not needed for ORC users
We should not separate users with file format, more is better.
| COMPILE_OPTION="$COMPILE_OPTION -DVELOX_BUILD_TESTING=OFF -DVELOX_BUILD_TEST_UTILS=ON" | ||
| COMPILE_OPTION="-DVELOX_ENABLE_PARQUET=ON -DVELOX_BUILD_TESTING=OFF -DVELOX_BUILD_TEST_UTILS=OFF -DVELOX_ENABLE_DUCKDB=OFF" | ||
| if [ $ENABLE_BENCHMARK == "ON" ]; then | ||
| COMPILE_OPTION="$COMPILE_OPTION -DVELOX_BUILD_BENCHMARKS=ON" |
There was a problem hiding this comment.
Thanks for your fix!
Considering --build_benchmarks option is being removed, do we need to keep this part?
There was a problem hiding this comment.
Thanks for review, I keep both build_benchmarks and build_tests here.
|
Run Gluten Clickhouse CI |
|
We just updated velox. Please do a code rebase. Thanks! |
|
Run Gluten Clickhouse CI |
Could you help review again? thanks! |
|
Thank you! |
|
===== Performance report for TPCH SF2000 with Velox backend, for reference only ====
|
Move duckdb and test_util targets under BUILD_TESTS flag to decrease build time, and modify related build scripts.
normalmeans without benchmark/test.