From 821a80510e5c5be100bd12a28b32879f605aec3e Mon Sep 17 00:00:00 2001 From: Xun Liu Date: Tue, 17 Oct 2023 20:23:44 +0800 Subject: [PATCH] [#492] feat(doc): Document website framework (#512) ### What changes were proposed in this pull request? Add launch document website script into `docs/launch-docs-website.sh` + You can execute `./launch-docs-website.sh` command to launch the docs website your locally, and you can open `http://localhost:1313/` to visit. + If you add or modify any documents in the docs directory, you need execute `./launch-docs-website.sh update` to update document website. ### Why are the changes needed? We are considering using hugo to generate the Graviton project document framework. + [hugo](https://github.com/gohugoio/hugo) is a popular tool for Documentation Sites, an Active community with better functionality. + gohugo is an Apache 2.0 license, After Gravtion to open source, no modifications are needed. Fix: #492 ### Does this PR introduce _any_ user-facing change? N/A ### How was this patch tested? N/A --------- Co-authored-by: Jerry Shao --- .gitignore | 3 +- docs/README.md | 40 +++++++++++++ docs/how-to-build.md | 10 ++-- docs/integration-test.md | 10 ++-- docs/launch-docs-website.sh | 107 ++++++++++++++++++++++++++++++++++ docs/publish-docker-images.md | 12 ++-- 6 files changed, 168 insertions(+), 14 deletions(-) create mode 100644 docs/README.md create mode 100755 docs/launch-docs-website.sh diff --git a/.gitignore b/.gitignore index 9eb2d6567a..7d98520a8d 100644 --- a/.gitignore +++ b/.gitignore @@ -36,4 +36,5 @@ out/** distribution server/src/main/resources/project.properties -dev/docker/hive/packages \ No newline at end of file +dev/docker/hive/packages +docs/build \ No newline at end of file diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000000..a8305ea890 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,40 @@ +--- +title: "Gravitino Document" +date: 2023-10-03T09:03:20-08:00 +license: "Copyright 2023 Datastrato. +This software is licensed under the Apache License version 2." +--- +# Gravitino Document + +## Launch Gravitino document website + +Execute `./launch-docs-website.sh` scripts, it will create the Gravitino document website locally, it follows these steps: + +1. Create a `build` directory within the `docs` directory. +2. Automatically download the [Hugo](https://github.com/gohugoio/hugo) binary executable file into the `build` directory. +3. Use the `hugo new site web` command to create the website project within the `build` directory. +4. Copy all markdown files into `web/content/docs` to generate the HTML for the document website using Hugo. +5. Copy the entire assets directory into `web/static` to include the images referenced in the document website HTML. +6. Automatically replace the Markdown embedded image addresses with Hugo website absolute paths. + > This is necessary because Markdown embedded images use relative paths, whereas Hugo website images require absolute paths. + > Replace `![](assets/` with `![](/assets/` in all markdown files. +7. Execute `hugo server` to launch the website. +8. Open `http://localhost:1313` in your browser to view the Gravitino document website. + +## Add or modify a document + +To add a new document to the Gravitino website, We should follow these steps: + +1. Create a new markdown file in the `docs` directory. +2. For the markdown file, we should include the following header at the beginning because `Hugo` use it create website link list. + ``` + --- + title: "Article Title" + date: Writing date + license: "Copyright 2023 Datastrato. + This software is licensed under the Apache License version 2." + --- + ``` +3. To insert an image into a markdown file, you must save all the referenced images in the `docs/assets` directory. + Additionally, you must use the following format: `![](assets/...` +4. Execute `./launch-docs-website.sh update` scripts to update the Gravitino document website. diff --git a/docs/how-to-build.md b/docs/how-to-build.md index 018ba33453..d84d808971 100644 --- a/docs/how-to-build.md +++ b/docs/how-to-build.md @@ -1,7 +1,9 @@ - +--- +title: "How to Build Gravitino" +date: 2023-10-03T09:03:20-08:00 +license: "Copyright 2023 Datastrato. +This software is licensed under the Apache License version 2." +--- # How to Build Gravitino ## Prerequisites diff --git a/docs/integration-test.md b/docs/integration-test.md index 8db2cf3862..198c12581b 100644 --- a/docs/integration-test.md +++ b/docs/integration-test.md @@ -1,7 +1,9 @@ - +--- +title: "How to Run Integration Tests" +date: 2023-10-02T09:03:20-08:00 +license: "Copyright 2023 Datastrato. +This software is licensed under the Apache License version 2." +--- # How to Run Integration Tests ## Introduction diff --git a/docs/launch-docs-website.sh b/docs/launch-docs-website.sh new file mode 100755 index 0000000000..e37c1d39cb --- /dev/null +++ b/docs/launch-docs-website.sh @@ -0,0 +1,107 @@ +#!/bin/bash +# +# Copyright 2023 Datastrato. +# This software is licensed under the Apache License version 2. +# +set -ex +bin="$(dirname "${BASH_SOURCE-$0}")" +bin="$(cd "${bin}">/dev/null; pwd)" + +USAGE="Usage: launch-docs-website.sh [update]" + +OS=$(uname -s) +if [ "$OS" == "Darwin" ]; then + HUGO_OS_NAME="darwin-universal" +elif [[ "$OS" == "Linux" ]]; then + HUGO_OS_NAME="linux-amd64" +else + echo "Currently only support macOS or Linux." + exit 1 +fi + +# Download Hugo +HUGO_VERSION="0.119.0" +HUGO_PACKAGE_NAME="hugo_${HUGO_VERSION}_${HUGO_OS_NAME}.tar.gz" +HUGO_DOWNLOAD_URL="https://github.com/gohugoio/hugo/releases/download/v${HUGO_VERSION}/${HUGO_PACKAGE_NAME}" + +# Because Markdown embedded images are relative paths, +# Hugo website images are absolute paths. +function hugo_image_path_process() { + for file in ${1}/*; do + if [[ -d ${file} ]]; then + hugo_image_path_process "${file}" + elif [[ -f ${file} ]]; then + # Replace `![](assets/` with `![](/assets/`, so that the images can be displayed correctly in the Hugo website. + sed -i '' 's/!\[\](assets\//!\[\](\/assets\//g' ${file} + fi + done +} + +# Copy all Markdown files to the Hugo website docs directory +function copy_docs() { + # Create a new Hugo website docs directory + if [[ ! -d ${bin}/build/web/content/docs ]]; then + mkdir -p ${bin}/build/web/content/docs + fi + + # Copy all markdown files to the Hugo website docs directory + # 1. Copy all root directory markdown files + rsync -av --exclude='README.md' ${bin}/*.md ${bin}/build/web/content/docs + + # 2. Copy all subdirectory markdown files + subDirs=$(find "${bin}" -type d -mindepth 1 -maxdepth 1) + for subDir in ${subDirs}; do + if [[ ${subDir} != ${bin}/build && ${subDir} != ${bin}/assets ]]; then + cp -r ${subDir} ${bin}/build/web/content/docs + fi + done + + # 3. Copy all assets to the Hugo website static directory + cp -r "${bin}/assets" "${bin}/build/web/static" + + # Replace the Markdown embedded images with Hugo website absolute paths + hugo_image_path_process "${bin}/build/web/content/docs" +} + +# Launch the Hugo website +function launch_website() { + # Prepare download packages + if [[ ! -d "${bin}/build" ]]; then + mkdir -p "${bin}/build" + fi + + if [ ! -f "${bin}/build/hugo" ]; then + wget -q -P "${bin}/build" ${HUGO_DOWNLOAD_URL} + tar -xzf "${bin}/build/${HUGO_PACKAGE_NAME}" -C "${bin}/build" + rm -rf "${bin}/build/${HUGO_PACKAGE_NAME}" + fi + + # Remove the old Hugo website + cd ${bin}/build + rm -rf ${bin}/build/web + + # Create a new Hugo website + ${bin}/build/hugo new site web + + # Setting the Hugo theme + cd ${bin}/build/web + git clone https://github.com/datastratolabs/gohugo-theme-ananke.git themes/ananke + + # Set the Hugo website configuration + echo "languageCode = 'en-us' + title = 'Gravitino Docs' + theme = 'ananke'" > hugo.toml + + copy_docs + + # Start the Hugo server + ${bin}/build/hugo server --buildDrafts +} + +if [[ "$1" == "update" ]]; then + copy_docs +elif [[ "$1" != "" ]]; then + echo ${USAGE} +else + launch_website +fi \ No newline at end of file diff --git a/docs/publish-docker-images.md b/docs/publish-docker-images.md index 1416b26441..9903040fdf 100644 --- a/docs/publish-docker-images.md +++ b/docs/publish-docker-images.md @@ -1,7 +1,9 @@ - +--- +title: "How to Publish Docker images" +date: 2023-10-01T09:03:20-08:00 +license: "Copyright 2023 Datastrato. +This software is licensed under the Apache License version 2." +--- # How to Publish Docker images ## Introduction @@ -27,7 +29,7 @@ We use Github Actions to publish the docker images to the DockerHub repository. 5. You must enter the correct `publish docker token` before you can execute run `Publish Docker Image` workflow. 6. Wait for the workflow to complete. You can see a new docker image shown in the [datastrato](https://hub.docker.com/u/datastrato) DockerHub repository. -[](assets/publish-docker-image.png) +![](assets/publish-docker-image.png) ## The version of the Data source in the Docker image - [gravitino-ci-hive](../dev/docker/hive/README.md) \ No newline at end of file