GH-41430: [Docs] Use sphinxcontrib-mermaid instead of generating images from .mmd #41455

kou · 2024-04-30T06:07:18Z

Rationale for this change

This is for easy to maintain.

What changes are included in this PR?

Install sphinxcontrib-mermaid
Install Chromium to generate SVG from .mmd
Use Debian instead of Ubuntu for building docs because Ubuntu provides Chromium only via snap
Use a normal user not root to build documents because Mermaid require additional --no-sandbox argument when we use root

Are these changes tested?

Yes.

Are there any user-facing changes?

No.

GitHub Issue: [Docs] Use sphinxcontrib-mermaid instead of generating images from .mmd #41430

github-actions · 2024-04-30T06:07:45Z

⚠️ GitHub issue #41430 has been automatically assigned in GitHub to PR creator.

kou · 2024-04-30T07:22:42Z

@github-actions crossbow submit preview-docs

raulcd · 2024-04-30T11:15:04Z

@github-actions crossbow submit preview-docs

zeroshade · 2024-04-30T21:30:04Z

ci/docker/linux-apt-docs.dockerfile

        wget && \
    apt-get clean && \
-    rm -rf /var/lib/apt/lists/*
+    rm -rf /var/lib/apt/lists/* && \
+    npm install -g yarn @mermaid-js/mermaid-cli


We should be setting

ENV CHROME_BIN="/usr/bin/chromium" \ PUPPETEER_SKIP_CHROMIUM_DOWNLOAD="true"

before we install mermaid-cli so that puppeteer doesn't download its own chromium binary, changing CHROME_BIN's value to wherever Debian installs it

Thanks for the info!
It seems that CHROME_BIN is needless. We need PUPPETEER_EXECUTABLE_PATH instead.

kou · 2024-05-01T07:16:24Z

@github-actions crossbow submit preview-docs

github-actions · 2024-05-04T21:21:13Z

Revision: 6cfc130

Submitted crossbow builds: ursacomputing/crossbow @ actions-d1fc246446

Task	Status
preview-docs

zeroshade

LGTM

kou · 2024-05-06T21:06:31Z

I'll merge this in a few days if nobody objects this.

conbench-apache-arrow · 2024-05-09T03:51:51Z

After merging your PR, Conbench analyzed the 0 benchmarking runs that have been run so far on merge-commit 46e7816.

None of the specified runs were found on the Conbench server.

The full Conbench report has more details.

### Rationale for this change Broken benchmarks after #41455 ### What changes are included in this PR? Use /tmp/arrow as build dir. ### Are these changes tested? ### Are there any user-facing changes? * GitHub Issue: #41630 Authored-by: Jacob Wujciak-Jens <[email protected]> Signed-off-by: Jacob Wujciak-Jens <[email protected]>

jorisvandenbossche · 2024-05-14T10:02:07Z

docs/source/conf.py

@@ -584,6 +587,9 @@
 #
 # texinfo_no_detailmenu = False

+# -- Options for mermaid output -------------------------------------------
+
+mermaid_output_format = 'svg'


Is there a reason we are setting this here, instead of letting it generate the svg image live through the js mermaid script? (the default setting)

That would avoid requiring mermaid-cli for building the docs

@zeroshade Why did you choose svg here? I just used your code here.

If I remember correctly, it would require us to actually pull down and statically add the mermaid js script to our resulting documentation output, which also ended up interfering with the anchor links on the right side based on headings. The "zoom" and other features that mermaid-js provides ended up causing issues so it was better for us to pre-generate the images during building the docs and include those images rather than rely on manually including and picking a version of mermaid-js to add to our static files.

from the sphixcontrib-mermaid docs:

The version of mermaid that will be used to parse raw output in HTML files. This should match a version available on https://unpkg.com/browse/mermaid/. The default is "10.2.0". If you need a newer version, you'll need to add the custom initialization. See below.

If it's set to "", the lib won't be automatically included from the CDN service and you'll need to add it as a local file in html_js_files.

…d change) (#513) The doc build was changed in apache/arrow#41455, we need a change here accordingly to ensure to keep updating the dev docs from that crossbow build.

…g images from .mmd (apache#41455) ### Rationale for this change This is for easy to maintain. ### What changes are included in this PR? * Install sphinxcontrib-mermaid * Install Chromium to generate SVG from .mmd * Use Debian instead of Ubuntu for building docs because Ubuntu provides Chromium only via snap * Use a normal user not root to build documents because Mermaid require additional `--no-sandbox` argument when we use root ### Are these changes tested? Yes. ### Are there any user-facing changes? No. * GitHub Issue: apache#41430 Authored-by: Sutou Kouhei <[email protected]> Signed-off-by: Sutou Kouhei <[email protected]>

…pache#41631) ### Rationale for this change Broken benchmarks after apache#41455 ### What changes are included in this PR? Use /tmp/arrow as build dir. ### Are these changes tested? ### Are there any user-facing changes? * GitHub Issue: apache#41630 Authored-by: Jacob Wujciak-Jens <[email protected]> Signed-off-by: Jacob Wujciak-Jens <[email protected]>

### Rationale for this change This is a follow-up of GH-41455. ### What changes are included in this PR? Add missing build directory argument. ### Are these changes tested? Yes. ### Are there any user-facing changes? No. * GitHub Issue: #41920 Authored-by: Sutou Kouhei <[email protected]> Signed-off-by: Sutou Kouhei <[email protected]>

kou requested review from assignUser and raulcd as code owners April 30, 2024 06:07

github-actions bot added Component: Documentation awaiting committer review Awaiting committer review labels Apr 30, 2024

kou force-pushed the docs-sphinx-mermaid branch from ac3dd7a to 64874ee Compare April 30, 2024 06:12

kou mentioned this pull request Apr 30, 2024

GH-41179: [Docs] Documentation for Dissociated IPC Protocol #41180

Merged

kou force-pushed the docs-sphinx-mermaid branch 3 times, most recently from e938b60 to 7378031 Compare April 30, 2024 07:22

This comment was marked as outdated.

Sign in to view

kou force-pushed the docs-sphinx-mermaid branch 2 times, most recently from f94eb24 to a9457bf Compare April 30, 2024 10:19

This comment was marked as outdated.

Sign in to view

kou force-pushed the docs-sphinx-mermaid branch 3 times, most recently from 1b40397 to f6081c6 Compare April 30, 2024 15:01

zeroshade reviewed Apr 30, 2024

View reviewed changes

github-actions bot added awaiting review Awaiting review awaiting changes Awaiting changes and removed awaiting committer review Awaiting committer review labels Apr 30, 2024

kou force-pushed the docs-sphinx-mermaid branch from f6081c6 to e686443 Compare May 1, 2024 07:11

github-actions bot added awaiting review Awaiting review and removed awaiting review Awaiting review awaiting changes Awaiting changes labels May 1, 2024

kou force-pushed the docs-sphinx-mermaid branch from e686443 to 2905424 Compare May 1, 2024 07:15

github-actions bot removed the awaiting review Awaiting review label May 1, 2024

zeroshade reviewed May 6, 2024

View reviewed changes

zeroshade approved these changes May 6, 2024

View reviewed changes

github-actions bot added awaiting changes Awaiting changes awaiting merge Awaiting merge and removed awaiting change review Awaiting change review awaiting changes Awaiting changes labels May 6, 2024

bkietz approved these changes May 6, 2024

View reviewed changes

kou merged commit 46e7816 into apache:main May 8, 2024
64 of 66 checks passed

kou deleted the docs-sphinx-mermaid branch May 8, 2024 21:29

kou removed the awaiting merge Awaiting merge label May 8, 2024

kou mentioned this pull request May 8, 2024

[Docs] Use sphinxcontrib-mermaid instead of generating images from .mmd #41430

Closed

This was referenced May 13, 2024

[Benchmarking] Out-of-source python build breaks benchmarks #41630

Closed

GH-41630: [Benchmarking] Fix out-of-source build in benchmarks #41631

Merged

jorisvandenbossche mentioned this pull request May 14, 2024

Fix update of dev docs (test-ubuntu-*-docs -> test-debian-*-docs build change) apache/arrow-site#513

Merged

jorisvandenbossche reviewed May 14, 2024

View reviewed changes

github-actions bot added the awaiting changes Awaiting changes label May 14, 2024

jorisvandenbossche mentioned this pull request May 18, 2024

GH-37929: [Python] begin moving static settings to pyproject.toml #41041

Merged

This was referenced Jun 1, 2024

[CI][JS] Missing build directory #41920

Closed

GH-41920: [CI][JS] Add missing build directory argument #41921

Merged

laurentgo mentioned this pull request Jun 3, 2024

GH-40819: [Java] Adding Spotless to Algorithm module #41825

Merged

llama90 mentioned this pull request Jun 6, 2024

GH-42005: [Java][Integration][CI] Fix ARROW_BUILD_ROOT Path to find pom.xml #42008

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GH-41430: [Docs] Use sphinxcontrib-mermaid instead of generating images from .mmd #41455

GH-41430: [Docs] Use sphinxcontrib-mermaid instead of generating images from .mmd #41455

kou commented Apr 30, 2024 •

edited by github-actions bot

Loading

github-actions bot commented Apr 30, 2024

kou commented Apr 30, 2024

This comment was marked as outdated.

raulcd commented Apr 30, 2024

This comment was marked as outdated.

zeroshade Apr 30, 2024

kou May 1, 2024

kou commented May 1, 2024

github-actions bot commented May 4, 2024

zeroshade left a comment

kou commented May 6, 2024

conbench-apache-arrow bot commented May 9, 2024

jorisvandenbossche May 14, 2024

kou May 14, 2024 •

edited

Loading

zeroshade May 15, 2024

GH-41430: [Docs] Use sphinxcontrib-mermaid instead of generating images from .mmd #41455

GH-41430: [Docs] Use sphinxcontrib-mermaid instead of generating images from .mmd #41455

Conversation

kou commented Apr 30, 2024 • edited by github-actions bot Loading

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

github-actions bot commented Apr 30, 2024

kou commented Apr 30, 2024

This comment was marked as outdated.

raulcd commented Apr 30, 2024

This comment was marked as outdated.

zeroshade Apr 30, 2024

Choose a reason for hiding this comment

kou May 1, 2024

Choose a reason for hiding this comment

kou commented May 1, 2024

github-actions bot commented May 4, 2024

zeroshade left a comment

Choose a reason for hiding this comment

kou commented May 6, 2024

conbench-apache-arrow bot commented May 9, 2024

jorisvandenbossche May 14, 2024

Choose a reason for hiding this comment

kou May 14, 2024 • edited Loading

Choose a reason for hiding this comment

zeroshade May 15, 2024

Choose a reason for hiding this comment

kou commented Apr 30, 2024 •

edited by github-actions bot

Loading

kou May 14, 2024 •

edited

Loading