chore: add a tool to check links [POC]

[tbouffard]

WIP
Covers #430 and #594.

Use htmltest: https://github.com/wjdp/htmltest

Configuration of the preview script use to build the site that is checked. This is why we are not using the production build for the checks:

• only build the supported version: out-of-support content is not updated
• no "edit this page": generate a lot of links to GitHub, this is an Antora feature, we don't want to test that
• force usage of the production navbar to have the content as close as possible of the production site
• use a dedicated "type" option to produce convenient HTML files for URL checks (HTML extension for local file browsing, no redirect pages)

The GH workflow enables the real use of Antora Atlas to resolve links to pages not built in the preview (links to pages related to unsupported versions). This may consume extra bandwidth in our Netlify plan.

Remaining tasks

• explain how to install locally in the readme or in a dedicated doc file. Including links resolutions to pages of unsupported version (with Antora Atlas)
• upload the artifact of the build site
• upload the artifact of htmltest logs
• configure user agent to better simulate a user. LinkedIn reject request from htmltest or curl without specific user agent. Work with curl by setting "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:131.0) Gecko/20100101 Firefox/131.0"
• review the htmltest configuration to ignore dependencies pages as they are generated and we cannot changes wrong external links. Try to use the IgnoreURLs option - IgnoreDirs cannot be used as the dependencies pages are located in the ROOT module at the root of the directory.
• how do we handle refcache update? commit the cache periodically by creating a PR after the execution ?
• build on schedule: do we send a slack notification on error?
• remove extra changes used during development
• plan fixes in the documentation content repository

Example of preview commands to generate the site to check locally

Single component version

./build-preview-dev.bash --hide-edit-page-links true --force-production-navbar true --type links-check --local-sources --component bonita --branch 2022.2
./build-preview-dev.bash --hide-edit-page-links true --force-production-navbar true --type links-check --local-sources --component cloud --branch master

Note:

• it is possible to activate Atlas reference resolution by installing Antora 3.2 alpha
• in this case, the link resolution is done with the production site if the page is not available locally
• this could help to test smaller part of the site by also checking some part of the production site (consume more bandwith for the Netlify site)

Build whole site (include bonita out of support version)

./build-preview-dev.bash --hide-edit-page-links true --force-production-navbar true --type links-check --local-sources --use-all-repositories true

Command to only build supported versions
The real check requires to enable Antora Atlas.
Otherwise, some cross references are not resolved like WARN: missing ref for bonita 2021.1 - some content have xref to this version

./build-preview-dev.bash \
  --hide-edit-page-links true \
  --force-production-navbar \
  --type links-check \
  --use-multi-repositories \
  --component-with-branches bcd:3.6,4.0 \
  --component-with-branches bonita:archives,2021.2,2022.1,2022.2,2023.1,2023.2 \
  --component-with-branches central:1.0 \
  --component-with-branches cloud:master \
  --component-with-branches labs:master \
  --component-with-branches test-toolkit:1.0,2.0 \
  --local-sources

Output

analysis of supported versions (without bonita 2021.1) 2023-09-01
✘✘✘ failed in 11m27.1693513s
987 errors in 3135 documents

Installation options

$ curl https://htmltest.wjdp.uk | bash -s -- -h
bash: download go binaries for wjdp/htmltest

Usage: bash [-b] bindir [-d] [tag]
  -b sets bindir or installation directory, Defaults to ./bin
  -d turns on debug logging
   [tag] is a tag from
   https://github.com/wjdp/htmltest/releases
   If tag is missing, then the latest will be used.

[github-actions]

🎊 PR Preview c7f6a8f has been successfully built and deployed to https://bonitasoft-bonita-documentation-site-test-pr-618.surge.sh

🕐 Build time: 0.013s

_{🤖 By surge-preview}

[github-actions]

🎊 PR Preview 944a9bb has been successfully built and deployed to https://bonitasoft-bonita-documentation-site-site-pr-618.surge.sh

🕐 Build time: 0.013s

_{🤖 By surge-preview}

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

[tbouffard]

⚠️ Currently, we only check a single bonita branch, we should check the whole site

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud