Tool: detect pages that are not in the navigation/taxonomy files

[tbouffard]

IMPORTANT: Be aware of #422

All pages of a component version are supposed to be accessed publicly and should be part of the navigation/taxonomy.

The verification could be done as part of the repository content checks.
Generally, all pages are expected to be in the navigation/taxonomy. Such tool would warn contributors when they miss a page.
This can also occurred when a page is renamed and the navigation/taxonomy is not updated: bonitasoft/bonita-doc#1749. Such a case should now be detected by the workflow validating the references.
See also https://documentation.bonitasoft.com/bonita/7.11/bonita-bpm-overview

The Antora extension tutorial described in details how to detect such pages: https://docs.antora.org/antora/3.1/extend/extension-tutorial/ and https://docs.antora.org/antora/3.1/extend/extension-use-cases/#report-unlisted-pages
So, we could create an extension for such purpose.

Exceptions

In special situation, some pages are not put in the navigation/taxonomy on purpose. For instance, the bonita-doc dependencies pages.
Such pages could be listed in a configuration file to ignore them during the detection.
Notice that such pages:

• shouldn't be indexed as well as part of SEO (see Only index the latest versions of a component in Google, Bing, ... #566)
• are listed in the sitemap, and this should not be a problem.
• are also currently available via the DocSearch page. If they shouldn't, create a specific issue.

Proposal for configuring pages to ignore during navigation/taxonomy check

Ideally, this should be configured in the documentation content repository. This is where writer maintain such pages and they should be able to decide when a page legitimately should not be in the navigation/taxonomy.

This may be done using the antora.yml file, as it already centralized all tuning for the version of a component.
If so, we could use a page AsciiDoc attribute as we already do for other settings.
The list of pages to ignore could be passed as a single string, separated by a comma.

Remember that in yml, it is possible to declare such a string with multiple lines. For example, this is what we do in

bonita-documentation-site/.github/workflows/_reusable_pr-validate-references.yml

Lines 48 to 57 in 6e61d79

	# '>' Replace newlines with spaces (folded)
	# '-' No newline at end (strip)
	run: >-
	./build-preview.bash
	--branch "${{ github.head_ref }}"
	--component "${{ inputs.component-name }}"
	--component-repo-url "${{ github.event.pull_request.head.repo.clone_url }}"
	--fail-on-warning "${{ inputs.fail-on-warning }}"
	--fetch-sources true
	--pr "${{ github.event.pull_request.number }}"

If declaring all pages is painful, we could also consider using glob/pattern, but only if we see that a component requires it

Existing resources

• https://github.com/redpanda-data/docs-extensions-and-macros/blob/8a7269bfc55e29a43f4193015ff51578041499e2/README.adoc#unlisted-pages

This extension identifies and logs any pages that aren’t listed in the navigation (nav) file of each version of each component. It then optionally adds these unlisted pages to the end of the navigation tree under a configurable heading