Skip to content

Commit

Permalink
address comments
Browse files Browse the repository at this point in the history
  • Loading branch information
zhli1142015 committed Nov 7, 2023
1 parent f9ff91d commit 8eae37e
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 2 deletions.
3 changes: 3 additions & 0 deletions docs/Configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,9 @@ You can add these configurations into spark-defaults.conf to enable or disable t
| spark.gluten.loadLibFromJar | Controls whether to load dynamic link library from a packed jar for gluten/cpp. Not applicable to static build and clickhouse backend. | false |
| spark.gluten.sql.columnar.force.hashagg | Force to use hash agg to replace sort agg. | true |
| spark.gluten.sql.columnar.vanillaReaders | Enable vanilla spark's vectorized reader. Please note it may bring perf. overhead due to extra data transition. We recommend to disable it if most queries can be fully offloaded to gluten. | false |
| spark.gluten.sql.columnar.backend.velox.bloomFilter.expectedNumItems | The default number of expected items for the velox bloomfilter. | 1000000L |
| spark.gluten.sql.columnar.backend.velox.bloomFilter.numBits | The default number of bits to use for the velox bloom filter. | 8388608L |
| spark.gluten.sql.columnar.backend.velox.bloomFilter.maxNumBits | The max number of bits to use for the velox bloom filter. | 4194304L |

Below is an example for spark-default.conf, if you are using conda to install OAP project.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1281,15 +1281,15 @@ object GlutenConfig {
val COLUMNAR_VELOX_BLOOM_FILTER_EXPECTED_NUM_ITEMS =
buildConf("spark.gluten.sql.columnar.backend.velox.bloomFilter.expectedNumItems")
.internal()
.doc("The default number of expected items for the velox bloomfilter" +
.doc("The default number of expected items for the velox bloomfilter: " +
"'spark.bloom_filter.expected_num_items'")
.longConf
.createWithDefault(1000000L)

val COLUMNAR_VELOX_BLOOM_FILTER_NUM_BITS =
buildConf("spark.gluten.sql.columnar.backend.velox.bloomFilter.numBits")
.internal()
.doc("The default number of bits to use for the velox bloom filter" +
.doc("The default number of bits to use for the velox bloom filter: " +
"'spark.bloom_filter.num_bits'")
.longConf
.createWithDefault(8388608L)
Expand Down

0 comments on commit 8eae37e

Please sign in to comment.