Skip to content

Commit

Permalink
[GLUTEN-3635][VL] Clean up docs related to build-arrow paths (#3709)
Browse files Browse the repository at this point in the history
  • Loading branch information
ted-jenks authored Nov 15, 2023
1 parent 36c9208 commit b2c62cd
Show file tree
Hide file tree
Showing 3 changed files with 9 additions and 32 deletions.
1 change: 0 additions & 1 deletion .github/workflows/velox_tpch_merge.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@ on:
- 'shims/**'
- 'tools/gluten-it/**'
- 'tools/gluten-te/**'
- 'ep/build-arrow/**'
- 'ep/build-velox/**'
- 'cpp/*'
- 'cpp/CMake/**'
Expand Down
32 changes: 7 additions & 25 deletions docs/developers/HowTo.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,32 +41,14 @@ To debug C++, you have to generate the example files, the example files consist

You can generate the example files by the following steps:

1. get and build Arrow
1. build Velox and Gluten CPP
```
cd gluten_home/ep/build-arrow/src
./get_arrow.sh
./build_arrow.sh
```

2. get and build Velox
```
cd gluten_home/ep/build-velox/src
./get_velox.sh
./build_velox.sh --build_type=Debug
```

3. compile the CPP
```
cd gluten_home/cpp
mkdir build
cd build
cmake -DBUILD_VELOX_BACKEND=ON -DBUILD_TESTS=ON -DBUILD_BENCHMARKS=ON -DCMAKE_BUILD_TYPE=Debug ..
make -j
gluten_home/dev/builddeps-veloxbe.sh --build_tests=ON --build_benchmarks=ON --build_type=Debug
```
- Compiling with `--build_type=Debug` is good for debugging.
- The executable file `generic_benchmark` will be generated under the directory of `gluten_home/cpp/build/velox/benchmarks/`.

4. build Gluten and generate the example files
2. build Gluten and generate the example files
```
cd gluten_home
mvn clean package -Pspark-3.2 -Pbackends-velox -Prss
Expand All @@ -87,7 +69,7 @@ gluten_home/backends-velox/generated-native-benchmark/
└── _SUCCESS
```

5. now, run benchmarks with GDB
3. now, run benchmarks with GDB
```
cd gluten_home/cpp/build/velox/benchmarks/
gdb generic_benchmark
Expand All @@ -99,15 +81,15 @@ gdb generic_benchmark
- Actually, you can debug `generic_benchmark` with any gdb commands as debugging normal C++ program, because the `generic_benchmark` is a pure C++
executable file in fact.

6. `gdb-tui` is a valuable feature and is worth trying. You can get more help from the online docs.
4. `gdb-tui` is a valuable feature and is worth trying. You can get more help from the online docs.
[gdb-tui](https://sourceware.org/gdb/onlinedocs/gdb/TUI.html)

7. you can start `generic_benchmark` with specific JSON plan and input files
5. you can start `generic_benchmark` with specific JSON plan and input files
- If you omit them, the `example.json, example_lineitem + example_orders` under the directory of `gluten_home/backends-velox/generated-native-benchmark`
will be used as default.
- You can also edit the file `example.json` to custom the Substrait plan or specify the inputs files placed in the other directory.

8. get more detail information about benchmarks from [MicroBenchmarks](./MicroBenchmarks.md)
6. get more detail information about benchmarks from [MicroBenchmarks](./MicroBenchmarks.md)

## 2 How to debug Java/Scala
wait to add
Expand Down
8 changes: 2 additions & 6 deletions docs/developers/NewToGluten.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,12 +58,8 @@ You can just refer to [build-gluten-with-velox-backend](../get-started/Velox.md#
If you need to debug cpp code, please compile the backend code and gluten cpp code with debug mode.

```bash
## compile velox
./build_velox.sh --build_type=Debug
## compile arrow with tests required library
./build_arrow.sh
## compile gluten cpp with benchmark and tests to debug
cmake -DBUILD_VELOX_BACKEND=ON -DBUILD_TESTS=ON -DBUILD_BENCHMARKS=ON -DCMAKE_BUILD_TYPE=Debug ..
## compile velox backend with benchmark and tests to debug
gluten_home/dev/builddeps-veloxbe.sh --build_tests=ON --build_benchmarks=ON --build_type=Debug
```

If you need to debug the tests in <gluten>/gluten-ut, You need to compile java code with `-P spark-ut`.
Expand Down

0 comments on commit b2c62cd

Please sign in to comment.