[DOC] Updating Zero-ETL data source

[RyanL1997]

Description

In 2.16 we have migrated the original S3 and Prometheus data source in dashboards observability into Dashboards management section of OSD core. This created an unified experience with opensearch connection data source. In addition, it also start supporting MDS features and allow users to view + edit an datasource from a remote cluster when the MDS feature flag is enabled (data_source.enabled: true).

UI flow changes

Before 2.16.0 - Current experience in dashboards observability

At the left side menu, user can see the option for access into the Zero-ETL data source:

For the data connection page it only supports the management of connections that is from local cluster

For the creation flow, we have a tab for creating S3/Prometheus data sources

After 2.16.0 - Unified experience with Multiple data sources in Dashboards Management

After the migration in version 2.16.0, we have unregistered the old data source option from the left side menu, and right now user should be able to access their data source management page from Dashboards managment --> Data sources

MDS disabled - data_source.enabled: false

when user has their MDS flag disabled, from the data sources management page, it will only show the Zero-ETL data sources from the local cluster

For the creation flow, by clicking the button Create data source connection at the top right, it will pop up the card view as creation options

MDS enabled - data_source.enabled: true

After the enabling of the MDS, at the data source management page, user should be able to see both "OpenSearch connections" tab for letting user to manage their remote cluster connections and the "Direct query connections" tab for manage their Zero-ETL data sources.

For the "Direct query connections" tab, it adds a cluster picker for the remote cluster selection, so that user should be able to view and manage their Zero ETL data connections for their remote cluster

Screen.Recording.2024-08-08.at.1.09.57.PM.mov

For the creation flow's card view, user should be able to see the creating option for OpenSearch connections

• Request a change to existing documentation
• Add new documentation
• Report a technical problem with the documentation
• Other

References

• [MDS] Observability Datasource Plugin migration with MDS support OpenSearch-Dashboards#7143
• [MDS] Data Connection details page with MDS support OpenSearch-Dashboards#7323
• [forwardport main] Fix minor issues for Direct Query Datasource (#7419) OpenSearch-Dashboards#7461

[Naarcha-AWS]

@RyanL1997: would it make sense to add these new flow additions to PR #7800?

[RyanL1997]

@Naarcha-AWS Sure and that will be great.

[AntonEliatra]

@RyanL1997 Am I correct in assuming all MDS connections are listed under "OpenSearch connections" and all Zero-ETL are under "Direct Query connections"? If not can you elaborate on the difference please

[Naarcha-AWS]

@RyanL1997: Also, is the Zero-ETL view behind a feature flag or just MDS?

[vagimeli]

@RyanL1997 Have you created a doc PR for this feature? Do you want to update the existing documentation (https://opensearch.org/docs/latest/dashboards/management/data-sources/)?