Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Discover content for 2.0 features #8177

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 73 additions & 0 deletions _dashboards/discover/defining-analyzing-searches.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
---
layout: default
title: Defining and analyzing searches
parent: Analyzing data
nav_order: 10
---

# Defining and analyzing searches

Discovering and exploring data in OpenSearch Dashboards is a powerful way to gain insights from your data. The Discover application within OpenSearch Dashboards provides a flexible interface for defining and analyzing searches across your data.

## Defining a search

To define a search, follow these steps:

1. On the OpenSearch Dashboards navigation menu, select **Discover**.
2. Choose the data you want to work with. In this case, choose `opensearch_dashboards_sample_data_flights` from the upper-left dropdown menu.
3. Select the {::nomarkdown}<img src="{{site.url}}{{site.baseurl}}/images/icons/calendar-oui.png" class="inline-icon" alt="calendar icon"/>{:/} icon to change the time range of your search and then select **Refresh**.

## Analyzing document tables

In OpenSearch, a document table stores unstructured data. In a document table, each row represents a single document, and each column contains document attributes.

### View document attributes

To review document attributes, follow these steps:

1. From the data table's left column, choose the {::nomarkdown}<img src="{{site.url}}{{site.baseurl}}/images/icons/inspect-icon.png" class="inline-icon" alt="inspect icon"/>{:/} icon to open the **Document Details** window. Select the {::nomarkdown}<img src="{{site.url}}{{site.baseurl}}/images/icons/minimize-icon.png" class="inline-icon" alt="minimize icon"/>{:/} icon to close the **Document Details** window.
2. Examine the metadata. You can switch between the **Table** and **JSON** tabs to view the data in your preferred format.
3. Select **View surrounding documents** to view data for other log entries either preceding or following your current document or select **View single document** to view a particular log entry.

### Add or delete fields in document tables

To add or delete fields in a document table, follow these steps:

1. View the data fields listed under **Available fields** and select the {::nomarkdown}<img src="{{site.url}}{{site.baseurl}}/images/icons/plus-icon.png" class="inline-icon" alt="plus icon"/>{:/} icon to add the desired fields to the document table. The field will be automatically added to both **Selected fields** and the document table. For this example, choose the fields `Carrier`, `AvgTicketPrice`, and `Dest`.
2. Select **Sort fields** > **Pick fields to sort by**. Drag and drop the chosen fields in the desired sort order.

## Searching data

You can use the search toolbar to enter a [DQL]({{site.url}}{{site.baseurl}}/dashboards/discover/dql/) or [query string]({{site.url}}{{site.baseurl}}/query-dsl/full-text/query-string/) query. The search toolbar is best for basic queries; for full query and filter capability, use [query domain-specific language (DSL)]({{site.url}}{{site.baseurl}}/query-dsl/index/) in the [Dev Tools console]({{site.url}}{{site.baseurl}}/dashboards/dev-tools/index-dev/).

For more information, see [Discover and Dashboard search toolbar]({{site.url}}{{site.baseurl}}/dashboards/index/#discover-and-dashboard-search-bar).

## Filtering data

Filters allow you to narrow the results of a query by specifying certain criteria. You can filter by field, value, or range. The **Add filter** pop-up suggests the available fields and operators.

To filter your data, follow these steps:

1. Under the DQL search bar, choose **Add filter**.
2. Select the desired options from the **Field**, **Operator**, and **Value** dropdown lists. For example, select `Cancelled`, `is`, and `true`.
3. Choose **Save**.
4. To remove a filter, choose the {::nomarkdown}<img src="{{site.url}}{{site.baseurl}}/images/icons/cross-icon.png" class="inline-icon" alt="cross icon"/>{:/} icon to the right of the filter name.

## Saving a search

To save your search, including the query text, filters, and current data view, follow these steps:

1. Select **Save** on the upper-right toolbar.
2. Add a title, and then choose **Save**.
3. Select **Open** on the upper-right toolbar to access your saved searches.

## Visualizing data findings

To visualize your data findings, follow these steps:

1. Select the {::nomarkdown}<img src="{{site.url}}{{site.baseurl}}/images/icons/inspect-icon.png" class="inline-icon" alt="inspect icon"/>{:/} icon to the right of the field you want to visualize.
2. Select the **Visualize** button. When the **Visualize** application is launched, a visualization appears.

## Setting alerts

Set alerts to notify you when your data exceeds your specified thresholds. Go to [Alerting dashboards and visualizations]({{site.url}}{{site.baseurl}}/observing-your-data/alerting/dashboards-alerting/) to learn about creating and managing alerts.
73 changes: 73 additions & 0 deletions _dashboards/discover/exploring-query-enhancements.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
---
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sejli Please tech review this documentation. It's the new content. All other files are changes to navigation and don't require tech approval. Thank you.

layout: default
title: Exploring query enhancements
parent: Analyzing data
nav_order: 20
---

# Exploring query enhancements
Introduced 2.17
{: .label .label-purple }

Starting with OpenSearch 2.17, query enhancements have been made. These enhancements are experimental and may be subject to change or instability. Enhancements include the following:

- Query languages PPL and SQL, with **Query Assist** for PPL
- Multiline query editor for PPL and SQL and autocomplete for PPL and DQL
- Data selector with **Data Explorer** that supports index patterns, indexes, and Amazon S3 connections and data configuration that supports selecting the appropriate query language based on the data type
- Link sharing through URLs without needing write permission to create an index pattern

All existing functionality remains in OpenSearch 2.17, and the new features are designed to improve your data exploration experience. The **Enable query enhancements** is currently not on a minimum distribution. <SME: Do we need to add text about plugins?>. You can leave your feedback at <insert-link> to help the OpenSearch Project improve.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sejli Will you provide me with the link for users to leave feedback about these features? Also, please review this paragraph. Do we need to add text about plugins?

{: .note}

## Enabling query enhancements

To enable the query enhancements through OpenSearch Dashboards, follow these steps:

1. Go to **Dashboards Management** > **Advanced settings** > **Search** and toggle on **Enable query enhancements**. Tip: You can select the **Search** pane from the **Category** dropdown menu in the upper-right search bar.
2. Select the **Save** button to save your changes.
3. Reload the page as prompted in the pop-up message.

Alternatively, you can override the setting on startup by running the following command:

```
./bin/opensearch-dashboards --uiSettings.overrides['query:enhancements:enabled']=true
```
{% include copy-curl.html %}

## Using the experimental features

The following tutorials guide you through some of the experimental features and capabilities.

### Query language enhancements

You can now use PPL in **Discover**. Follow these steps to try out the feature:

1. Go to **Discover** and select **PPL** from the query language dropdown menu in the upper-right search bar. You should see a dashboard containing the query editor, histogram, and data table panes.
2. Select a sample dataset. For this example, select `opensearch_dashboards_sample_data_ecommerce` from the data source dropdown menu above the query editor and adjust the time filter to **Last 1 year**.
3. Enter the following example PPL query:

```json
source = opensearch_dashboards_sample_data_logs
| where tags = "success"
| where geo.dest = "US"
```
{% include copy-curl.html %}

4. View the resulting output that shows the number of successful log entries originating from the United States. You should see an updated histogram and data table following the query editor.
5. Select the **Recent queries** option within the query editor toolbar to display your recent queries.

PPL and DQL provide an autocomplete option that suggests field names, functions, and syntax.


## Data sources and data types enhancements

You can now select your data sources and types from within the **Discover** dashboard. Follow these steps to try out the feature.

1. From the **Discover** page, select a data source from the dropdown menu in the upper toolbar.
2. Select the **View all available data** button to display a list of your available data sources. You may need to refresh your page to display any newly added data sources.
3. Select the desired data source and follow steps displayed in the data sources window to manage your data source.

You can now use **Query Assist** with PPL queries. With **Query Assist**, you can ask questions like _Are there any errors in my logs?_. The assistant includes predefined prompts. Follow these steps to try out the feature:

1. Select **PPL** from the dropdown menu in the query toolbar.
2. Select the {::nomarkdown}<img src="{{site.url}}{{site.baseurl}}/images/dashboards/query-assist.png" class="inline-icon" alt="query assist icon"/>{:/} icon and choose a predefined question. The resulting output is displaying in the query editor pane.
99 changes: 3 additions & 96 deletions _dashboards/discover/index-discover.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,105 +9,12 @@ redirect_from:

# Analyzing data

To analyze your data in OpenSearch and visualize key metrics, you can use the **Discover** application in OpenSearch Dashboards. An example of data analysis in **Discover** is shown in the following image.

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/discover.png" alt="A Discover default page" width="700">
You can analyze your data in OpenSearch and visualize key metrics using the **Discover** application in OpenSearch Dashboards. Using **Discover**, you can explore and visualize data from various data sources, data types, and query languages.

## Getting started

In this tutorial, you'll learn about using **Discover** to:

- Add data.
- Interpret and visualize data.
- Share data findings.
- Set alerts.

Before getting started, make sure you:
Before getting started with exploring and visualizing your data using **Discover**, make sure you:

- Install [OpenSearch Dashboards](https://opensearch.org/downloads.html).
- Add sample data or import your own data into OpenSearch. Go to the [OpenSearch Dashboards quickstart guide]({{site.url}}{{site.baseurl}}/dashboards/quickstart/) to learn about adding sample datasets. Go to [Managing indexes]({{site.url}}{{site.baseurl}}/im-plugin/index/) to learn about importing your own data.
- Add sample data or import your own data into OpenSearch. See [OpenSearch Dashboards quickstart guide]({{site.url}}{{site.baseurl}}/dashboards/quickstart/) to learn about adding sample datasets or [Managing indexes]({{site.url}}{{site.baseurl}}/im-plugin/index/) to learn about importing your own data.
- Have a foundational understanding of [OpenSearch documents and indexes]({{site.url}}{{site.baseurl}}/im-plugin/index/).

## Defining the search

To define a search, follow these steps:

1. On the OpenSearch Dashboards navigation menu, select **Discover**.
2. Choose the data you want to work with. In this case, choose `opensearch_dashboards_sample_data_flights` from the upper-left dropdown menu.
3. Select the {::nomarkdown}<img src="{{site.url}}{{site.baseurl}}/images/icons/calendar-oui.png" class="inline-icon" alt="calendar icon"/>{:/} icon to change the time range of your search and then select **Refresh**.

The resulting view is shown in the following image.

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/define-search.png" alt="Discover interface showing search of flight sample data for Last 7 days" width="700">

## Analyzing document tables

In OpenSearch, a document table stores unstructured data. In a document table, each row represents a single document, and each column contains document attributes.

To examine document attributes, follow these steps:

1. From the data table's left column, choose the {::nomarkdown}<img src="{{site.url}}{{site.baseurl}}/images/icons/inspect-icon.png" class="inline-icon" alt="inspect icon"/>{:/} icon to open the **Document Details** window. Select the {::nomarkdown}<img src="{{site.url}}{{site.baseurl}}/images/icons/minimize-icon.png" class="inline-icon" alt="minimize icon"/>{:/} icon to close the **Document Details** window.
2. Examine the metadata. You can switch between the **Table** and **JSON** tabs to view the data in your preferred format.
3. Select **View surrounding documents** to view data for other log entries either preceding or following your current document or select **View single document** to view a particular log entry.

The resulting view is shown in the following image.

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/doc-details.png" alt="Document attributes" width="700">

To add or delete fields in a document table, follow these steps:

1. View the data fields listed under **Available fields** and select the {::nomarkdown}<img src="{{site.url}}{{site.baseurl}}/images/icons/plus-icon.png" class="inline-icon" alt="plus icon"/>{:/} icon to add the desired fields to the document table. The field will be automatically added to both **Selected fields** and the document table. For this example, choose the fields `Carrier`, `AvgTicketPrice`, and `Dest`.
2. Select **Sort fields** > **Pick fields to sort by**. Drag and drop the chosen fields in the desired sort order.

The resulting view is shown in the following image.

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/add-data-fields.png" alt="Adding and deleting data fields" width="700">

## Searching data

You can use the search toolbar to enter a [DQL]({{site.url}}{{site.baseurl}}/dashboards/discover/dql/) or [query string]({{site.url}}{{site.baseurl}}/query-dsl/full-text/query-string/) query. The search toolbar is best for basic queries; for full query and filter capability, use [query domain-specific language (DSL)]({{site.url}}{{site.baseurl}}/query-dsl/index/) in the [Dev Tools console]({{site.url}}{{site.baseurl}}/dashboards/dev-tools/index-dev/).

For more information, see [Discover and Dashboard search toolbar]({{site.url}}{{site.baseurl}}/dashboards/index/#discover-and-dashboard-search-bar).

## Filtering data

Filters allow you to narrow the results of a query by specifying certain criteria. You can filter by field, value, or range. The **Add filter** pop-up suggests the available fields and operators.

To filter your data, follow these steps:

1. Under the DQL search bar, choose **Add filter**.
2. Select the desired options from the **Field**, **Operator**, and **Value** dropdown lists. For example, select `Cancelled`, `is`, and `true`.
3. Choose **Save**.
4. To remove a filter, choose the {::nomarkdown}<img src="{{site.url}}{{site.baseurl}}/images/icons/cross-icon.png" class="inline-icon" alt="cross icon"/>{:/} icon to the right of the filter name.

The resulting view is shown in the following image.

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/discover-filter.png" alt="Visualize data findings interface" width="700"/>

## Saving a search

To save your search, including the query text, filters, and current data view, follow these steps:

1. Select **Save** on the upper-right toolbar.
2. Add a title, and then choose **Save**.
3. Select **Open** on the upper-right toolbar to access your saved searches.

## Visualizing data findings

To visualize your data findings, follow these steps:

1. Select the {::nomarkdown}<img src="{{site.url}}{{site.baseurl}}/images/icons/inspect-icon.png" class="inline-icon" alt="inspect icon"/>{:/} icon to the right of the field you want to visualize.

The resulting view is shown in the following image.

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/visualize-discover.png" alt="Visualize data findings interface" width="700"/>

2. Select the **Visualize** button. When the **Visualize** application is launched, a visualization appears.

The resulting view is shown in the following image.

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/visualization-flight.png" alt="Data visualization of flight sample data field destination" width="700"/>

## Setting alerts

Set alerts to notify you when your data exceeds your specified thresholds. Go to [Alerting dashboards and visualizations]({{site.url}}{{site.baseurl}}/observing-your-data/alerting/dashboards-alerting/) to learn about creating and managing alerts.
2 changes: 1 addition & 1 deletion _dashboards/discover/time-filter.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: default
title: Time filter
parent: Analyzing data
nav_order: 20
nav_order: 50
redirect_from:
- /dashboards/get-started/time-filter/
- /dashboards/discover/time-filter/
Expand Down
Binary file added images/dashboards/query-assist.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading