Add auto-generated connected components in documentation

[thampiotr]

Description

This PR adds:

• An auto-generated section "Compatible Components" on each component's reference page that lists the most common ways the components can be connected. The "most common ways" is determined by type of data being used: Targets, Prom metrics, Loki logs, Pyroscope profiles, OTEL.
• A new "Compatible Components" page which lists some of the most common data types used in telemetry pipelines and which components can be used to either export or consume these data types.
• Code required to generate these pages.
• Tests that verify that the checked-in docs match with in-memory generated docs, so we can verify that they are being updated correctly. This test also allows to overwrite local files when developer wants to update the docs locally with the generated versions.

Generation workflow

The docs can be re-generated using make generate-docs.
There is a CI test that compares the current docs vs. the in-memory generated docs. If differences are found, the test fails and adequate error message is printed. This can be fixed by running make generate-docs and the changes should be reviewed as part of PR.

Examples

Loki section on "Compatible Components" page. The list of loki namespace components is collapsed, cause it's long.

And an example of "Compatible Components" section on "loki.process" reference page:

Fixes: #4754

[ptodev]

Thank you for working on this! I really like the change. I put in some comments with suggestions on how to make it more polished.

[ptodev]

generate-versioned-files also generates docs-related content. It might be a bit confusing to have a generate-docs alongside it.

[thampiotr]

Hmmm, I am not a fan of "versioned files", cause I'm not sure what it is by reading the name. It could be a lot of things. Do you have suggestions? I'm trying not to refactor existing code here though, as it's already a big PR.

[ptodev]

TBH I cannot think of more appropriate names... generate-docs literally runs go generate on the docs, and generate-versioned-files is not docs specific. It updates a template file which is in pkg/operator

└─▪ find . -type f -name "*.t"
./docs/sources/_index.md.t
./pkg/operator/defaults.go.t

I'm ok with leaving the names as they are, but I think it might be a bit confusing since people will expect a step called "generate docs" to also update the Agent version in the docs.

[ptodev]

IMO saying " can accept [...] from the following components" makes it sound like the component is data telemetry from those components. However, in this case faro.erceiver is sending data to them.

It'd help to have a more neutral language like:

faro.receiver can be combined with any of the following components:

Or:

faro.receiver is compatible with any of the following components:

[ptodev]

It would also help to clarify that otelcol.Consumer is mentioned due to output -> traces, and that LogsReceiver is mentioned due to output -> logs . Not sure if it's easy to do something like this:

Compatible components

output -> logs can be configured using a component which exports [Loki LogsReceiver]({{< relref "../compatibility/#loki-logsreceiver-exporters" >}})

output -> traces can be configured using a component which exports [OpenTelemetry otelcol.Consumer]({{< relref "../compatibility/#opentelemetry-otelcolconsumer-exporters" >}})

[thampiotr]

I know that "can accept arguments" can sound misleading if one forgets that we (sometimes...) use receivers to pass data around. But without clarification that the components can be combined via either argument or export, I find it frustrating to have to figure it out yourself. It requires extra clicks and jumps around the documentation, making the reading experience a frustrating one. So I'd prefer to be specific and not more neutral.

I like the idea of providing by name the arguments/exports that can be used to combine components, but I'd like this to keep as a future improvement, not for this PR.

[ptodev]

It bothers me a bit that the title in {{< collapse title="" >}} is bigger than a ### title. So "Targets Exporters" appears smaller than "discovery".

Is there a way to make the expandable section a bit smaller? OR at least to make its title smaller?

[thampiotr]

Not something I can do here, but we can probably request the UI team to help with this in a separate PR? cc @jdbaldry perhaps? Here's an illustration of the problem:

[clayton-cornell]

Yeah it's currently locked in at a single size. It def could use a revisit to improve the way it is styled.

[ptodev]

Suggested change

	### Prometheus `MetricsReceiver` Exporters
	### Exporters

This will make the ToC on the right of the web page look much cleaner. I think we can just have subsections called "Exporters" and "Receivers" for each type.

[thampiotr]

This would break the anchors that we link to directly. I'm aware of the TOC, but I thought that it's more important to have good anchors 🤔 We could link to the parent heading though as an alternative...

[ptodev]

It seems there is a workaround to have links to duplicate titles, but not a very reliable one.... I'm happy if you just leave it as it is. Having this as a working hyperlink does feel more important.

[thampiotr]

Thanks @ptodev and @clayton-cornell for the feedback. I've now addressed&resolved or commented on all the threads. PTAL when you can, thanks.

[ptodev]

Thank you! There are a few things we could polish over. time, but the change looks good and I'm very happy for it to be merged.