[Discover Next] Dataset Dropdown #7267

sejli · 2024-07-16T20:03:50Z

Description

Adds a new UI component to support datasets in Discover.

Issues Resolved

Screenshot

Testing the changes

Changelog

Check List

All tests pass
- yarn test:jest
- yarn test:jest_integration
New functionality includes testing.
New functionality has been documented.
Update CHANGELOG.md
Commits are signed per the DCO using --signoff

Signed-off-by: Kawika Avilla <[email protected]>

Signed-off-by: Sean Li <[email protected]>

* Update plugin.ts to comment out SQL Async and disable filters to SQL * Remove target directory Signed-off-by: Sean Li <[email protected]>

Signed-off-by: Sean Li <[email protected]>

Add target and build directories to .gitignore Signed-off-by: Sean Li <[email protected]>

Going for the light weight solution for 2.15. So took out what wasn't required deliverable. However, I created a branch `feature/discover-next` which has the state where all the features where added so we can port it over post 2.15. Signed-off-by: Kawika Avilla <[email protected]>

Signed-off-by: Sean Li <[email protected]>

PPL Query fix

Signed-off-by: Kawika Avilla <[email protected]>

Some clean up to reduce file count

…arch-project#6895) it adds query assist specific logic in query enhancements plugin to show a UI above the PPL search bar if user has configured PPL agent. Issues Resolved: opensearch-project#6820 * add query assist to query enhancements Signed-off-by: Joshua Li <[email protected]> * align language to uppercase Signed-off-by: Joshua Li <[email protected]> * pick PR 6167 Signed-off-by: Joshua Li <[email protected]> * use useState hooks for query assist There is a bug in data explorer `AppContainer` where memorized `DiscoverCanvas` gets unmounted after `setQuery`. PR 6167 works around it by memorizing `AppContainer`. As query assist is no longer being unmounted, we can use proper hooks to persist state now. Signed-off-by: Joshua Li <[email protected]> * Revert "pick PR 6167" This reverts commit acb0d41. Wait for official 6167 to merge to avoid conflict Signed-off-by: Joshua Li <[email protected]> * address comments for PR 6894 Signed-off-by: Joshua Li <[email protected]> --------- Signed-off-by: Joshua Li <[email protected]> (cherry picked from commit 016e0f2)

…assist (opensearch-project#6933) * pass dependencies to isEnabled func Signed-off-by: Joshua Li <[email protected]> * add lazy and memo to search bar extensions Signed-off-by: Joshua Li <[email protected]> * move ppl specific string out from query assist Signed-off-by: Joshua Li <[email protected]> * prevent setstate after hook unmounts Signed-off-by: Joshua Li <[email protected]> * add max-height to search bar extensions Signed-off-by: Joshua Li <[email protected]> * prevent setstate after component unmounts Signed-off-by: Joshua Li <[email protected]> * move ml-commons API to common/index.ts Signed-off-by: Joshua Li <[email protected]> * improve i18n and accessibility usages Signed-off-by: Joshua Li <[email protected]> * add hard-coded suggestions for sample data indices Signed-off-by: Joshua Li <[email protected]> --------- Signed-off-by: Joshua Li <[email protected]> (cherry picked from commit 4aade0f)

…ject#6972) * disable query assist for non-default datasource Signed-off-by: Joshua Li <[email protected]> * disable query assist input when loading Signed-off-by: Joshua Li <[email protected]> * support MDS for query assist Signed-off-by: Joshua Li <[email protected]> * add unit tests for agents Signed-off-by: Joshua Li <[email protected]> * Revert "add unit tests for agents" This reverts commit 983514e. The test configs are not yet setup in query_enhancements plugins. Signed-off-by: Joshua Li <[email protected]> --------- Signed-off-by: Joshua Li <[email protected]> (cherry picked from commit 328e08e)

Signed-off-by: Joshua Li <[email protected]>

Signed-off-by: Kawika Avilla <[email protected]>

Signed-off-by: Joshua Li <[email protected]>

This is a temporary solution given that in discover the index pattern selector will be removed. Before datasource and dataset selectors are added, query assist will rely on this index pattern selector to determine which index user wants to query. Signed-off-by: Joshua Li <[email protected]>

Related to: opensearch-project#7059

Utilizing the work from MDS to make PPL and SQL calls to a remote cluster. Signed-off-by: Kawika Avilla <[email protected]>

Signed-off-by: Joshua Li <[email protected]>

This PR picks [Discover-next] add query assist to query enhancements plugin opensearch-project#6895 [Discover-next] Address comments for search bar extensions and query assist opensearch-project#6933 [Discover-next] Support data sources for query assist opensearch-project#6972 adds query assist banner a callout that advertises query assist when user has at least 1 language with query assist configured and is not on a configured language adds index selector for local cluster only This is a temporary solution given that in discover the index pattern selector will be removed. Before datasource and dataset selectors are added, query assist will rely on this index pattern selector to determine which index user wants to query. Signed-off-by: Joshua Li <[email protected]>

Utilizing the work from MDS to make PPL and SQL calls to a remote cluster. Uses: https://github.com/opensearch-project/OpenSearch-Dashboards/pull/7092/files

Signed-off-by: Kawika Avilla <[email protected]>

Add missing license headers

add more tests for query assist update eslint config to make it work increase index selector width Signed-off-by: Joshua Li <[email protected]>

The `Connection` type does not have the extra `dataSource` object around response, and most of the time code references connection over dataSource. remove the redundent object to make response consistent with type. Signed-off-by: Joshua Li <[email protected]>

Signed-off-by: Joshua Li <[email protected]>

As part of some work on opensearch-project#24, this PR introduces some mock-heavy tests for the SQL search strategy. It's also helpful to write down the loose interfaces expected from different methods -- not entirely sure if I got them all right. Signed-off-by: Simeon Widdis <[email protected]>

…nnection previously query assist reads the datasource id from the selected index pattern. this PR updates it to read from the new `ConnectionsService` - fix!: remove dataSource key in get connection response - this is a breaking change on the return response of `GET ${API.DATA_SOURCE.CONNECTIONS}/{dataSourceId}` to be consistent with type - change isEnabled to an observable based on selected connection - this depends on opensearch-project#7183, should go in with that PR Signed-off-by: Joshua Li <[email protected]>

Signed-off-by: Kawika Avilla <[email protected]>

…4901eaf67bb619c6b01f68' git-subtree-dir: src/plugins/query_enhancements git-subtree-mainline: 87627af git-subtree-split: 9abc883

Signed-off-by: Kawika Avilla <[email protected]>

…shboards-1 into query_enhancements

Signed-off-by: Kawika Avilla <[email protected]>

Signed-off-by: Sean Li <[email protected]>

github-actions · 2024-07-16T20:04:21Z

❌ Empty Changelog Section

The Changelog section in your PR description is empty. Please add a valid changelog entry or entries. If you did add a changelog entry, check to make sure that it was not accidentally included inside the comment block in the Changelog section.

codecov · 2024-07-16T20:18:42Z

Codecov Report

Attention: Patch coverage is 42.53731% with 231 lines in your changes missing coverage. Please review.

Project coverage is 67.37%. Comparing base (87627af) to head (0e026d4).
Report is 15 commits behind head on main.

Files	Patch %	Lines
src/plugins/query_enhancements/common/utils.ts	8.77%	52 Missing ⚠️
.../public/ui/dataset_navigator/dataset_navigator.tsx	14.28%	30 Missing ⚠️
...a_source_connection/components/connections_bar.tsx	3.33%	29 Missing ⚠️
...plugins/query_enhancements/server/utils/plugins.ts	10.00%	27 Missing ⚠️
...c/plugins/query_enhancements/server/utils/facet.ts	25.71%	26 Missing ⚠️
.../data_explorer/public/components/sidebar/index.tsx	41.17%	10 Missing ⚠️
...lic/query_assist/components/query_assist_input.tsx	65.51%	6 Missing and 4 partials ⚠️
...query_enhancements/server/utils/shim_schema_row.ts	0.00%	10 Missing ⚠️
...gins/query_enhancements/server/utils/shim_stats.ts	0.00%	10 Missing ⚠️
..._source_connection/services/connections_service.ts	75.75%	8 Missing ⚠️
... and 6 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7267      +/-   ##
==========================================
- Coverage   67.55%   67.37%   -0.19%     
==========================================
  Files        3469     3495      +26     
  Lines       68479    69157     +678     
  Branches    11130    11262     +132     
==========================================
+ Hits        46264    46592     +328     
- Misses      19512    19853     +341     
- Partials     2703     2712       +9

Flag	Coverage Δ
Linux_1	`?`
Linux_2	`55.30% <ø> (+0.03%)`	⬆️
Linux_3	`?`
Linux_4	`?`
Windows_1	`33.27% <13.33%> (+0.11%)`	⬆️
Windows_2	`55.25% <ø> (+0.03%)`	⬆️
Windows_3	`45.17% <27.69%> (-0.15%)`	⬇️
Windows_4	`34.79% <41.77%> (+0.07%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Sean Li <[email protected]>

sejli · 2024-07-17T18:43:34Z

Closing for #7289

kavilla and others added 30 commits June 11, 2024 18:32

init

40d6ed1

Signed-off-by: Kawika Avilla <[email protected]>

fixing bugs and removing target

b7063c5

Signed-off-by: Sean Li <[email protected]>

Updating plugin.ts and removing target directory

a34f895

* Update plugin.ts to comment out SQL Async and disable filters to SQL * Remove target directory Signed-off-by: Sean Li <[email protected]>

add target and build to .gitignore

c767590

Signed-off-by: Sean Li <[email protected]>

Add target and build to .gitignore

50ce9c6

Add target and build directories to .gitignore Signed-off-by: Sean Li <[email protected]>

adding fix for PPL queries

c0304d1

Signed-off-by: Sean Li <[email protected]>

Merge pull request #3 from sejli/ppl-fix

c617aa5

PPL Query fix

Clean up files and add helper functions

67e6b30

Signed-off-by: Kawika Avilla <[email protected]>

final touches

e11a384

Signed-off-by: Kawika Avilla <[email protected]>

Clean up files and use helper functions

8867564

Some clean up to reduce file count

add query assist banner

56ab13b

Signed-off-by: Joshua Li <[email protected]>

implement banner rendering logic

43d1d43

Signed-off-by: Joshua Li <[email protected]>

pick opensearch-project/dashboards-observability/pull/1896

c97cd2d

Signed-off-by: Joshua Li <[email protected]>

Conditionally render the data source select config

48f9c7a

Signed-off-by: Kawika Avilla <[email protected]>

restore no space

ec37345

Signed-off-by: Kawika Avilla <[email protected]>

add basic tests

780fbd5

Signed-off-by: Joshua Li <[email protected]>

Conditionally render the data source select config

c636dee

Related to: opensearch-project#7059

MDS working with PPL and SQL

8572c23

Utilizing the work from MDS to make PPL and SQL calls to a remote cluster. Signed-off-by: Kawika Avilla <[email protected]>

rename logo to mark

fd651e7

Signed-off-by: Joshua Li <[email protected]>

extract supported languages to config

8ad8600

Signed-off-by: Joshua Li <[email protected]>

Merge branch 'main' into kavilla/mds_working

97b57ba

MDS working + more cleanups

3d2e294

Utilizing the work from MDS to make PPL and SQL calls to a remote cluster. Uses: https://github.com/opensearch-project/OpenSearch-Dashboards/pull/7092/files

Add missing license headers

2e3304e

Signed-off-by: Kawika Avilla <[email protected]>

Adds license headers

6192877

Add missing license headers

joshuali925 and others added 22 commits July 3, 2024 10:45

add more tests for query assist (opensearch-project#31)

3543d51

add more tests for query assist update eslint config to make it work increase index selector width Signed-off-by: Joshua Li <[email protected]>

change isEnabled to an observable based on selected connection

9f43eba

Signed-off-by: Joshua Li <[email protected]>

protect duplicated calls

a43fcdc

Signed-off-by: Joshua Li <[email protected]>

[build] import via start services

9abc883

Signed-off-by: Kawika Avilla <[email protected]>

Add 'src/plugins/query_enhancements/' from commit '9abc883f56e1d032fd…

e2a1636

…4901eaf67bb619c6b01f68' git-subtree-dir: src/plugins/query_enhancements git-subtree-mainline: 87627af git-subtree-split: 9abc883

Changeset file for PR opensearch-project#7212 created/updated

fdab7a2

foobar

dbf05b5

Signed-off-by: Kawika Avilla <[email protected]>

Merge branch 'query_enhancements' of github.com:kavilla/OpenSearch-Da…

b398640

…shboards-1 into query_enhancements

updating imports still in progress

f879c1b

Signed-off-by: Kawika Avilla <[email protected]>

Update more imports

ece47d0

Signed-off-by: Kawika Avilla <[email protected]>

fix setting of state

f58c4ba

Signed-off-by: Kawika Avilla <[email protected]>

fixes bugs

20411f1

Signed-off-by: Kawika Avilla <[email protected]>

initial implementation for dataset dropdown

70ed2c7

Signed-off-by: Sean Li <[email protected]>

initial metadata commit

26385ef

Signed-off-by: Sean Li <[email protected]>

bug fixes

6099185

Signed-off-by: Sean Li <[email protected]>

hiding datasource cluster selctor and other bug fixes

c45be4a

Signed-off-by: Sean Li <[email protected]>

refactoring and fixing datasource selector button

fc025aa

Signed-off-by: Sean Li <[email protected]>

fix breaking loop

a5144f4

Signed-off-by: Sean Li <[email protected]>

last working commit

0e026d4

Signed-off-by: Sean Li <[email protected]>

github-actions bot added the repeat-contributor label Jul 16, 2024

github-actions bot added the failed changeset label Jul 16, 2024

WIP indices

9235350

Signed-off-by: Sean Li <[email protected]>

sejli mentioned this pull request Jul 17, 2024

[Discover Next] Dataset Dropdown #7289

Closed

7 tasks

sejli closed this Jul 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Discover Next] Dataset Dropdown #7267

[Discover Next] Dataset Dropdown #7267

sejli commented Jul 16, 2024

github-actions bot commented Jul 16, 2024

codecov bot commented Jul 16, 2024 •

edited

Loading

sejli commented Jul 17, 2024

[Discover Next] Dataset Dropdown #7267

[Discover Next] Dataset Dropdown #7267

Conversation

sejli commented Jul 16, 2024

Description

Issues Resolved

Screenshot

Testing the changes

Changelog

Check List

github-actions bot commented Jul 16, 2024

❌ Empty Changelog Section

codecov bot commented Jul 16, 2024 • edited Loading

Codecov Report

sejli commented Jul 17, 2024

codecov bot commented Jul 16, 2024 •

edited

Loading