[Workspace] Integrate workspace front end with data connection type saved object #8013

raintygao · 2024-09-05T09:10:08Z

Description

Integrate workspace with data connection type saved object in frontend

Screenshot

Testing the changes

Use APIs in #7925 to create data connection object, then goes to workspace create page to assign/unassign data connection objects.

Changelog

feat: Integrate workspace with data connections in front end

Check List

All tests pass
- yarn test:jest
- yarn test:jest_integration
New functionality includes testing.
New functionality has been documented.
Update CHANGELOG.md
Commits are signed per the DCO using --signoff

codecov · 2024-09-13T11:15:14Z

Codecov Report

Attention: Patch coverage is 91.66667% with 3 lines in your changes missing coverage. Please review.

Project coverage is 64.11%. Comparing base (2213b57) to head (c87b2d8).
Report is 25 commits behind head on main.

Files with missing lines	Patch %	Lines
src/plugins/workspace/public/utils.ts	87.50%	2 Missing ⚠️
...components/workspace_form/connection_type_icon.tsx	85.71%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8013      +/-   ##
==========================================
+ Coverage   64.09%   64.11%   +0.01%     
==========================================
  Files        3744     3744              
  Lines       88822    88842      +20     
  Branches    13841    13847       +6     
==========================================
+ Hits        56931    56958      +27     
+ Misses      31277    31270       -7     
  Partials      614      614

Flag	Coverage Δ
Linux_1	`30.07% <91.66%> (+0.03%)`	⬆️
Linux_2	`58.85% <ø> (ø)`
Linux_3	`40.33% <ø> (ø)`
Linux_4	`31.57% <ø> (ø)`
Windows_1	`30.09% <91.66%> (+0.03%)`	⬆️
Windows_2	`58.80% <ø> (ø)`
Windows_3	`40.33% <ø> (ø)`
Windows_4	`31.57% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

src/plugins/workspace/common/constants.ts

kavilla · 2024-09-13T20:49:46Z

src/plugins/workspace/common/constants.ts

@@ -158,3 +160,9 @@ export enum AssociationDataSourceModalMode {
  DirectQueryConnections = 'direction-query-connections',
 }
 export const USE_CASE_PREFIX = 'use-case-';
+
+// Workspace will handle both data source and data connection type saved object.
+export const WORKSPACE_DATA_SOURCE_AND_CONNECTION_OBJECT_TYPES = [


any thought about using the data structure type?

https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/src/plugins/data/common/datasets/types.ts#L111

example:

https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/src/plugins/query_enhancements/public/datasets/s3_type.ts#L181

Looks good, will take a look on whether we could use this.

Why isn't type in https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/src/plugins/data/common/datasets/types.ts#L41 data-source which is a type for data source object?

Is there an example for data-connection type object?

Im thinking whether data structure type would be a little complex for workspace, because workspace may don't need dataset, column, table related info.

src/plugins/workspace/public/components/workspace_creator/workspace_creator.tsx

kavilla · 2024-09-13T20:52:07Z

src/plugins/workspace/public/components/workspace_detail/association_data_source_modal.tsx

@@ -31,6 +31,9 @@ import { HttpStart, NotificationsStart, SavedObjectsStart } from '../../../../..
 import { AssociationDataSourceModalMode } from '../../../common/constants';
 import { Logos } from '../../../../../core/common';
 import { DirectQueryConnectionIcon } from '../workspace_form';
+import { DataConnectionIcon } from '../workspace_form';


import this from line 33. also do we have multiple cases of defining icon throughout the app?

Yes, this is still pending as I'm finding a better way to maintain different state between data-source, query connection, data-connection.

kavilla · 2024-09-13T20:54:17Z

src/plugins/workspace/public/components/workspace_detail/association_data_source_modal.tsx

@@ -31,6 +31,9 @@ import { HttpStart, NotificationsStart, SavedObjectsStart } from '../../../../..
 import { AssociationDataSourceModalMode } from '../../../common/constants';
 import { Logos } from '../../../../../core/common';
 import { DirectQueryConnectionIcon } from '../workspace_form';
+import { DataConnectionIcon } from '../workspace_form';
+
+import { DATA_CONNECTION_SAVED_OBJECT_TYPE } from '../../../../data_source/common/data_connections';

 const ConnectionIcon = ({


imo this could be cleaned up with using the data structures https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/src/plugins/data/common/datasets/types.ts#L75.

where we have the information about the icon and type

Thanks, will take a look.

I checked the interface of this data structure and haven't seen any related integration with data-connection type object. I suppose this is still in progress?

raintygao · 2024-09-14T06:18:05Z

Server side changes are extracted in (#8200) in order to start testing as soon as possible.

wanglam · 2024-09-14T06:40:30Z

src/plugins/workspace/public/utils.ts

-  return [
+  const dataConnections = dataSources
+    .filter((ds) => ds.type === DATA_CONNECTION_SAVED_OBJECT_TYPE)
+    .map((ds) => ({ ...ds, name: ds.id }));


The type of data source connection will be displayed in the data source connection table. There will be a type filter at the top right of the data source connection table. There is a type property in the definition of data connection saved object. Will it been displayed in the data source connection table? If yes, i'm prefer using DataSourceConnectionType.DataConnection to indicate the new data-connection;

ruanyl · 2024-09-20T05:58:25Z

src/plugins/workspace/public/components/workspace_detail/workspace_detail_connection_table.tsx

@@ -9,7 +9,6 @@ import { i18n } from '@osd/i18n';
 import { DataSourceConnection, DataSourceConnectionType } from '../../../common/types';
 import { AssociationDataSourceModalMode } from '../../../common/constants';
 import { DataSourceConnectionTable } from '../workspace_form';
-


Nit: this empty line should be kept

ruanyl · 2024-09-20T06:13:39Z

src/plugins/workspace/public/utils.ts

          const workspaces = source.workspaces ?? [];
          const auth = source.get('auth');
          const description = source.get('description');
          const dataSourceEngineType = source.get('dataSourceEngineType');
+          const type = source.type;
+          // This is a field only for detail type of data connection in order not to mix saved object type.


Nit:

Suggested change

// This is a field only for detail type of data connection in order not to mix saved object type.

// This is a field only for detail type of data connection in order not to mix with saved object type.

ruanyl · 2024-09-20T06:15:32Z

src/plugins/workspace/public/utils.ts

+      };
+    });
+
+export const convertDataSourcesToDataConnections = (


Nit: maybe can consider to combine convertDataSourcesToOpenSearchConnections and convertDataSourcesToDataConnections

ruanyl · 2024-09-20T06:23:22Z

src/plugins/workspace/public/utils.ts

    ...fulfillRelatedConnections(openSearchConnections, directQueryConnections),
    ...directQueryConnections,
-  ].sort((a, b) => a.name.localeCompare(b.name));
+    ...dataConnections,
+  ].sort((a, b) => a?.name?.localeCompare(b?.name));


It doesn't seem optional chain is needed here since the type definition shows name is mandatory.

Signed-off-by: tygao <[email protected]>

Co-authored-by: SuZhou-Joe <[email protected]> Signed-off-by: Tianyu Gao <[email protected]>

Signed-off-by: tygao <[email protected]>

…aved object (#8013) * feat: integrate workspace with data connections Signed-off-by: tygao <[email protected]> * update workspace pages and hooks to integrate with data connection Signed-off-by: tygao <[email protected]> * Changeset file for PR #8013 created/updated * remove extra comments Signed-off-by: tygao <[email protected]> * update data source import Signed-off-by: tygao <[email protected]> * test: update tests Signed-off-by: tygao <[email protected]> * Changeset file for PR #8013 created/updated * unify connection type icon and connectionType Signed-off-by: tygao <[email protected]> * test: add tests Signed-off-by: tygao <[email protected]> * display text directly instead of link for data connection in table Signed-off-by: tygao <[email protected]> * Apply suggestions from code review Co-authored-by: SuZhou-Joe <[email protected]> Signed-off-by: Tianyu Gao <[email protected]> * update Signed-off-by: tygao <[email protected]> --------- Signed-off-by: tygao <[email protected]> Signed-off-by: Tianyu Gao <[email protected]> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> Co-authored-by: SuZhou-Joe <[email protected]> (cherry picked from commit c894584) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…aved object (#8013) (#8269) * feat: integrate workspace with data connections * update workspace pages and hooks to integrate with data connection * Changeset file for PR #8013 created/updated * remove extra comments * update data source import * test: update tests * Changeset file for PR #8013 created/updated * unify connection type icon and connectionType * test: add tests * display text directly instead of link for data connection in table * Apply suggestions from code review * update --------- (cherry picked from commit c894584) Signed-off-by: tygao <[email protected]> Signed-off-by: Tianyu Gao <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> Co-authored-by: SuZhou-Joe <[email protected]>

…aved object (opensearch-project#8013) * feat: integrate workspace with data connections Signed-off-by: tygao <[email protected]> * update workspace pages and hooks to integrate with data connection Signed-off-by: tygao <[email protected]> * Changeset file for PR opensearch-project#8013 created/updated * remove extra comments Signed-off-by: tygao <[email protected]> * update data source import Signed-off-by: tygao <[email protected]> * test: update tests Signed-off-by: tygao <[email protected]> * Changeset file for PR opensearch-project#8013 created/updated * unify connection type icon and connectionType Signed-off-by: tygao <[email protected]> * test: add tests Signed-off-by: tygao <[email protected]> * display text directly instead of link for data connection in table Signed-off-by: tygao <[email protected]> * Apply suggestions from code review Co-authored-by: SuZhou-Joe <[email protected]> Signed-off-by: Tianyu Gao <[email protected]> * update Signed-off-by: tygao <[email protected]> --------- Signed-off-by: tygao <[email protected]> Signed-off-by: Tianyu Gao <[email protected]> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> Co-authored-by: SuZhou-Joe <[email protected]>

…aved object (opensearch-project#8013) (opensearch-project#8269) * feat: integrate workspace with data connections * update workspace pages and hooks to integrate with data connection * Changeset file for PR opensearch-project#8013 created/updated * remove extra comments * update data source import * test: update tests * Changeset file for PR opensearch-project#8013 created/updated * unify connection type icon and connectionType * test: add tests * display text directly instead of link for data connection in table * Apply suggestions from code review * update --------- (cherry picked from commit c894584) Signed-off-by: tygao <[email protected]> Signed-off-by: Tianyu Gao <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> Co-authored-by: SuZhou-Joe <[email protected]>

github-actions bot added the all-star-contributor label Sep 5, 2024

opensearch-changeset-bot bot added a commit to raintygao/OpenSearch-Dashboards that referenced this pull request Sep 5, 2024

Changeset file for PR opensearch-project#8013 created/updated

ef45821

raintygao force-pushed the connection branch from ef45821 to 64d6c94 Compare September 13, 2024 10:31

opensearch-changeset-bot bot added a commit to raintygao/OpenSearch-Dashboards that referenced this pull request Sep 13, 2024

Changeset file for PR opensearch-project#8013 created/updated

bb9bc5b

kavilla reviewed Sep 13, 2024

View reviewed changes

src/plugins/workspace/common/constants.ts Outdated Show resolved Hide resolved

kavilla reviewed Sep 13, 2024

View reviewed changes

src/plugins/workspace/public/components/workspace_creator/workspace_creator.tsx Outdated Show resolved Hide resolved

kavilla reviewed Sep 13, 2024

View reviewed changes

raintygao mentioned this pull request Sep 14, 2024

[Workspace] feat: update workspace server to support data connection type #8200

Merged

7 tasks

wanglam reviewed Sep 14, 2024

View reviewed changes

raintygao pushed a commit to raintygao/OpenSearch-Dashboards that referenced this pull request Sep 18, 2024

Changeset file for PR opensearch-project#8013 created/updated

0a26d80

raintygao force-pushed the connection branch from 4af08ef to 7c1ca5d Compare September 18, 2024 02:43

opensearch-changeset-bot bot added a commit to raintygao/OpenSearch-Dashboards that referenced this pull request Sep 18, 2024

Changeset file for PR opensearch-project#8013 created/updated

6301de0

raintygao pushed a commit to raintygao/OpenSearch-Dashboards that referenced this pull request Sep 18, 2024

Changeset file for PR opensearch-project#8013 created/updated

7301b3d

raintygao pushed a commit to raintygao/OpenSearch-Dashboards that referenced this pull request Sep 18, 2024

Changeset file for PR opensearch-project#8013 created/updated

664db8c

raintygao force-pushed the connection branch from 6301de0 to 0587aea Compare September 18, 2024 08:06

raintygao changed the title ~~[Workspace] Integrate workspace with data connection type saved object~~ [Workspace] Integrate workspace front end with data connection type saved object Sep 18, 2024

raintygao marked this pull request as ready for review September 18, 2024 08:07

raintygao requested review from ananzh, AMoo-Miki, ashwin-pc, joshuarrrr, abbyhu2000, zengyan-amazon, zhongnansu, manasvinibs and ZilongX as code owners September 18, 2024 08:07

SuZhou-Joe added backport 2.x workspace 2.17.1 labels Sep 20, 2024

raintygao dismissed SuZhou-Joe’s stale review via fe658c5 September 20, 2024 05:10

SuZhou-Joe previously approved these changes Sep 20, 2024

View reviewed changes

ruanyl previously approved these changes Sep 20, 2024

View reviewed changes

raintygao and others added 12 commits September 20, 2024 16:23

feat: integrate workspace with data connections

a137624

Signed-off-by: tygao <[email protected]>

update workspace pages and hooks to integrate with data connection

4771d37

Signed-off-by: tygao <[email protected]>

Changeset file for PR opensearch-project#8013 created/updated

5136db1

remove extra comments

7af04ca

Signed-off-by: tygao <[email protected]>

update data source import

2e303f1

Signed-off-by: tygao <[email protected]>

test: update tests

c628a36

Signed-off-by: tygao <[email protected]>

Changeset file for PR opensearch-project#8013 created/updated

78629d7

unify connection type icon and connectionType

e4d1916

Signed-off-by: tygao <[email protected]>

test: add tests

c4ae001

Signed-off-by: tygao <[email protected]>

display text directly instead of link for data connection in table

a78d01a

Signed-off-by: tygao <[email protected]>

Apply suggestions from code review

db41ee2

Co-authored-by: SuZhou-Joe <[email protected]> Signed-off-by: Tianyu Gao <[email protected]>

update

c87b2d8

Signed-off-by: tygao <[email protected]>

raintygao dismissed stale reviews from ruanyl and SuZhou-Joe via c87b2d8 September 20, 2024 08:40

raintygao force-pushed the connection branch from fe658c5 to c87b2d8 Compare September 20, 2024 08:40

ruanyl approved these changes Sep 20, 2024

View reviewed changes

SuZhou-Joe approved these changes Sep 20, 2024

View reviewed changes

SuZhou-Joe merged commit c894584 into opensearch-project:main Sep 20, 2024
67 checks passed

opensearch-trigger-bot bot mentioned this pull request Sep 20, 2024

[Backport 2.x] [Workspace] Integrate workspace front end with data connection type saved object #8269

Merged

AMoo-Miki added the in-next label Sep 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Workspace] Integrate workspace front end with data connection type saved object #8013

[Workspace] Integrate workspace front end with data connection type saved object #8013

raintygao commented Sep 5, 2024 •

edited

Loading

codecov bot commented Sep 13, 2024 •

edited

Loading

kavilla Sep 13, 2024

raintygao Sep 14, 2024

raintygao Sep 18, 2024 •

edited

Loading

kavilla Sep 13, 2024

raintygao Sep 14, 2024

kavilla Sep 13, 2024

raintygao Sep 14, 2024

raintygao Sep 18, 2024

raintygao commented Sep 14, 2024 •

edited

Loading

wanglam Sep 14, 2024 •

edited

Loading

raintygao Sep 18, 2024

ruanyl Sep 20, 2024

raintygao Sep 20, 2024

ruanyl Sep 20, 2024

raintygao Sep 20, 2024

ruanyl Sep 20, 2024

raintygao Sep 20, 2024

ruanyl Sep 20, 2024

raintygao Sep 20, 2024

	// This is a field only for detail type of data connection in order not to mix saved object type.
	// This is a field only for detail type of data connection in order not to mix with saved object type.

[Workspace] Integrate workspace front end with data connection type saved object #8013

[Workspace] Integrate workspace front end with data connection type saved object #8013

Conversation

raintygao commented Sep 5, 2024 • edited Loading

Description

Screenshot

Testing the changes

Changelog

Check List

codecov bot commented Sep 13, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

raintygao Sep 18, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

raintygao commented Sep 14, 2024 • edited Loading

wanglam Sep 14, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

raintygao commented Sep 5, 2024 •

edited

Loading

codecov bot commented Sep 13, 2024 •

edited

Loading

raintygao Sep 18, 2024 •

edited

Loading

raintygao commented Sep 14, 2024 •

edited

Loading

wanglam Sep 14, 2024 •

edited

Loading