Skip to content

Commit

Permalink
Fix
Browse files Browse the repository at this point in the history
  • Loading branch information
PHILO-HE committed Aug 20, 2024
1 parent ca0181c commit a9887ac
Show file tree
Hide file tree
Showing 6 changed files with 25 additions and 22 deletions.
20 changes: 10 additions & 10 deletions .github/workflows/velox_docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,8 +63,8 @@ jobs:
with:
path: |
./cpp/build/releases/
./cpp/build/velox/udf/examples/
./cpp/build/velox/benchmarks/
# ./cpp/build/velox/udf/examples/
# ./cpp/build/velox/benchmarks/
key: cache-velox-build-centos-7-${{ hashFiles('./cache-key') }}
- name: Build Gluten native libraries
if: ${{ steps.cache.outputs.cache-hit != 'true' }}
Expand All @@ -86,7 +86,7 @@ jobs:
# path: ./cpp/build/velox/benchmarks/
- uses: actions/upload-artifact@v2
with:
name: arrow-jar-centos-7-${{github.sha}}
name: arrow-jars-centos-7-${{github.sha}}
path: /root/.m2/repository/org/apache/arrow/

run-tpc-test-ubuntu:
Expand Down Expand Up @@ -127,7 +127,7 @@ jobs:
- name: Download All Arrow Jar Artifacts
uses: actions/download-artifact@v2
with:
name: arrow-jar-centos-7-${{github.sha}}
name: arrow-jars-centos-7-${{github.sha}}
path: /root/.m2/repository/org/apache/arrow/
- name: Setup tzdata
run: |
Expand Down Expand Up @@ -198,7 +198,7 @@ jobs:
- name: Download All Arrow Jar Artifacts
uses: actions/download-artifact@v2
with:
name: arrow-jar-centos-7-${{github.sha}}
name: arrow-jars-centos-7-${{github.sha}}
path: /root/.m2/repository/org/apache/arrow/
- name: Update mirror list
run: |
Expand Down Expand Up @@ -283,7 +283,7 @@ jobs:
- name: Download All Arrow Jar Artifacts
uses: actions/download-artifact@v2
with:
name: arrow-jar-centos-7-${{github.sha}}
name: arrow-jars-centos-7-${{github.sha}}
path: /home/runner/.m2/repository/org/apache/arrow/
- name: Setup java and maven
run: |
Expand Down Expand Up @@ -395,7 +395,7 @@ jobs:
- name: Download All Arrow Jar Artifacts
uses: actions/download-artifact@v2
with:
name: arrow-jar-centos-7-${{github.sha}}
name: arrow-jars-centos-7-${{github.sha}}
path: /home/runner/.m2/repository/org/apache/arrow/
- name: Setup java and maven
run: |
Expand Down Expand Up @@ -488,7 +488,7 @@ jobs:
- name: Download All Arrow Jar Artifacts
uses: actions/download-artifact@v2
with:
name: arrow-jar-centos-7-${{github.sha}}
name: arrow-jars-centos-7-${{github.sha}}
path: /root/.m2/repository/org/apache/arrow/
- name: Update mirror list
run: |
Expand Down Expand Up @@ -555,7 +555,7 @@ jobs:
- name: Download All Arrow Jar Artifacts
uses: actions/download-artifact@v2
with:
name: arrow-jar-centos-7-${{github.sha}}
name: arrow-jars-centos-7-${{github.sha}}
path: /root/.m2/repository/org/apache/arrow/
- name: Setup tzdata
run: |
Expand Down Expand Up @@ -729,7 +729,7 @@ jobs:
- name: Download Arrow Jars
uses: actions/download-artifact@v2
with:
name: arrow-jars-centos-8-${{github.sha}}
name: arrow-jars-centos-7-${{github.sha}}
path: /root/.m2/repository/org/apache/arrow/
- name: Update mirror list
run: |
Expand Down
1 change: 1 addition & 0 deletions dev/vcpkg/ports/simdjson/portfile.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ vcpkg_cmake_configure(
-DSIMDJSON_BUILD_STATIC=${SIMDJSON_BUILD_STATIC}
-DSIMDJSON_DEVELOPMENT_CHECKS=OFF
-DSIMDJSON_VERBOSE_LOGGING=OFF
-DSIMDJSON_SKIPUTF8VALIDATION=ON
${FEATURE_OPTIONS}
)

Expand Down
4 changes: 2 additions & 2 deletions docs/developers/UsingGperftoolsInCH.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
layout: page
title: Developer Overview
title: Debug for CH Backend with gpertools Tool
nav_order: 11
has_children: true
permalink: /developer-overview/
parent: /developer-overview/
---
We need using gpertools to find the memory or CPU issue. That's what this document is about.

Expand Down
4 changes: 2 additions & 2 deletions docs/developers/UsingJemallocWithCH.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
layout: page
title: Developer Overview
title: Use Jemalloc for CH Backend
nav_order: 12
has_children: true
permalink: /developer-overview/
parent: /developer-overview/
---
We need using jemalloc to find the memory issue. That's what this document is about.

Expand Down
4 changes: 2 additions & 2 deletions docs/developers/VeloxUDF.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
layout: page
title: Developer Overview
title: Velox UDF and UDAF
nav_order: 13
has_children: true
permalink: /developer-overview/
parent: /developer-overview/
---
# Velox User-Defined Functions (UDF) and User-Defined Aggregate Functions (UDAF)

Expand Down
14 changes: 8 additions & 6 deletions docs/developers/velox_backend_CI.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
layout: page
title: CPP Code Style
title: Velox Backend CI
nav_order: 14
parent: Developer Overview
---
Expand All @@ -9,14 +9,16 @@ parent: Developer Overview
GHA workflows are defined under `.github/workflows/`.

## Docker Build
We have a weekly job to build a docker based on Dockerfile.gha for CI verification, defined in docker_image.yml.
We have a weekly job to build a docker based on `Dockerfile.gha` for CI verification, defined in `docker_image.yml`.

## Vcpkg Caching
Gluten main branch is pulled down during docker build. And vcpkg will cache binary data of all dependencies defined under dev/vcpkg.
These binary data is cached into `/var/cache/vcpkg` and CI job can re-use them in new build. By setting `VCPKG_BINARY_SOURCES=clear` in env., reusing cache can be disabled.
These binary data is cached into `/var/cache/vcpkg` and CI job can re-use them in new build. By setting `VCPKG_BINARY_SOURCES=clear` in env.,
reusing vcpkg cache can be disabled.

## Arrow Libs Pre-installation
Arrow libs are also pre-installed in docker, assuming they are not actively changed and not necessarily to be re-built every time.
Arrow libs are pre-installed in docker, assuming they are not actively changed, then not necessarily to be re-built every time.

## Updating Docker
Two Github secrets `DOCKERHUB_USER` & `DOCKERHUB_TOKEN` can be used to push docker to docker hub: https://hub.docker.com/r/apache/gluten/tags.
## Updating Docker Image
Two GitHub secrets `DOCKERHUB_USER` & `DOCKERHUB_TOKEN` can be used to push docker image to docker hub: https://hub.docker.com/r/apache/gluten/tags.
Note GitHub secrets are not retrievable in PR from forked repo.

0 comments on commit a9887ac

Please sign in to comment.