Merge 2.x to model registry (#289)

* Increment version to 2.8.0.0 (#169) Signed-off-by: opensearch-ci-bot <[email protected]> Co-authored-by: opensearch-ci-bot <[email protected]> * chore: update husky to 8.0.3 to avoid use execa (#160) (#189) Signed-off-by: Lin Wang <[email protected]> (cherry picked from commit 6a3faee) Co-authored-by: Lin Wang <[email protected]> * doc: add release notes for 2.8.0 (#190) (#191) Signed-off-by: Lin Wang <[email protected]> (cherry picked from commit f2b701b) Co-authored-by: Lin Wang <[email protected]> * feat: remove experiment warning banner (#194) (#195) * feat: remove experiment warning banner Signed-off-by: Lin Wang <[email protected]> * doc: update release notes for remove experiment banner Signed-off-by: Lin Wang <[email protected]> --------- Signed-off-by: Lin Wang <[email protected]> (cherry picked from commit 0ef4753) Co-authored-by: Lin Wang <[email protected]> * feat: enable ml-commons-dashboards plugin by default (#200) (#201) * feat: enable ml-commons-dashboards plugin by default Signed-off-by: Lin Wang <[email protected]> * feat: update release notes for enable plugin by default Signed-off-by: Lin Wang <[email protected]> --------- Signed-off-by: Lin Wang <[email protected]> (cherry picked from commit a1cb7de) Co-authored-by: Lin Wang <[email protected]> * Revert remove experiment banner and enable by default (#203) (#204) * Revert "feat: enable ml-commons-dashboards plugin by default (#200)" This reverts commit a1cb7de. Signed-off-by: Lin Wang <[email protected]> * Revert "feat: remove experiment warning banner (#194)" This reverts commit 0ef4753. Signed-off-by: Lin Wang <[email protected]> --------- Signed-off-by: Lin Wang <[email protected]> (cherry picked from commit 31df7d4) Co-authored-by: Lin Wang <[email protected]> * remove background for dark mode to fix #206 (#209) (#212) Signed-off-by: Lin Wang <[email protected]> (cherry picked from commit 2bc3f38) Co-authored-by: Lin Wang <[email protected]> * Fix ml document link address (#210) (#213) * update to correct machine learning document address to fix #208 Signed-off-by: Lin Wang <[email protected]> * fix: update link in deployment table empty screen Signed-off-by: Lin Wang <[email protected]> --------- Signed-off-by: Lin Wang <[email protected]> (cherry picked from commit 7a33d01) Co-authored-by: Lin Wang <[email protected]> * feat: open external link in blank page (#211) (#214) Signed-off-by: Lin Wang <[email protected]> (cherry picked from commit e2825c4) Co-authored-by: Lin Wang <[email protected]> * Increment version to 2.9.0.0 (#222) Signed-off-by: opensearch-ci-bot <[email protected]> Co-authored-by: opensearch-ci-bot <[email protected]> * feat: exclude remote model for admin UI (#225) (#228) Signed-off-by: Lin Wang <[email protected]> (cherry picked from commit ba01d34) Co-authored-by: Lin Wang <[email protected]> * Revert "Revert remove experiment banner and enable by default (#203)" (#229) (#231) * Revert "Revert remove experiment banner and enable by default (#203)" This reverts commit 31df7d4. Signed-off-by: Lin Wang <[email protected]> * doc: remove release update for 2.8.0 Signed-off-by: Lin Wang <[email protected]> --------- Signed-off-by: Lin Wang <[email protected]> (cherry picked from commit 2caf020) Co-authored-by: Lin Wang <[email protected]> * doc: add release notes for 2.9.0 (#232) (#233) Signed-off-by: Lin Wang <[email protected]> (cherry picked from commit 02e80a9) Co-authored-by: Lin Wang <[email protected]> * Add source field to distinguish local and external model (#239) (#240) * Revert "feat: exclude remote model for admin UI (#225)" This reverts commit ba01d34. Signed-off-by: Lin Wang <[email protected]> * feat: add source field to distinguish local and external model Signed-off-by: Lin Wang <[email protected]> * feat: add miss display words Signed-off-by: Lin Wang <[email protected]> --------- Signed-off-by: Lin Wang <[email protected]> (cherry picked from commit 63c7a5a) Co-authored-by: Lin Wang <[email protected]> * Fix no model show up when search a model (#238) (#242) * fix: reset to page 1 after name or state search Signed-off-by: Lin Wang <[email protected]> * refactor: remove unnecessary act call Signed-off-by: Lin Wang <[email protected]> --------- Signed-off-by: Lin Wang <[email protected]> (cherry picked from commit 24aa9df) Co-authored-by: Lin Wang <[email protected]> * Feat mitigate to oui style variables (#227) (#246) * feat: remove customized font size and font weight Signed-off-by: Lin Wang <[email protected]> * feat: update color to $ouiLinkColor Signed-off-by: Lin Wang <[email protected]> * feat: remove white background for refresh interval Signed-off-by: Lin Wang <[email protected]> * remove ace theme Signed-off-by: Lin Wang <[email protected]> * refactor: update <b> to <strong> Signed-off-by: Lin Wang <[email protected]> --------- Signed-off-by: Lin Wang <[email protected]> (cherry picked from commit 3aa8eaf) Co-authored-by: Lin Wang <[email protected]> * feat: support external models in deployed model list (#248) (#251) * feat: support external models in deployed model list Signed-off-by: Lin Wang <[email protected]> * refactor: update name to required Signed-off-by: Lin Wang <[email protected]> * fix: connector index not found Signed-off-by: Lin Wang <[email protected]> * refactor: update options filter with normal string[] value Signed-off-by: Lin Wang <[email protected]> * fix: hits not defined Signed-off-by: Lin Wang <[email protected]> * fix: update wording Signed-off-by: Lin Wang <[email protected]> * fix: connector id not exists in all connectors Signed-off-by: Lin Wang <[email protected]> * fix: show models when failed to load all external connectors Signed-off-by: Lin Wang <[email protected]> * feat: update deployed models title to models Signed-off-by: Lin Wang <[email protected]> * refactor: remove unused code in model connector filter Signed-off-by: Lin Wang <[email protected]> * feat: address PR comments Signed-off-by: Lin Wang <[email protected]> --------- Signed-off-by: Lin Wang <[email protected]> (cherry picked from commit cc3810b) Co-authored-by: Lin Wang <[email protected]> * update preview panel for external models (#252) (#253) * feat: update preview panel for external models Signed-off-by: tygao <[email protected]> * feat: update preview panel for external models Signed-off-by: tygao <[email protected]> --------- Signed-off-by: tygao <[email protected]> (cherry picked from commit 9bbd25f) Co-authored-by: raintygao <[email protected]> * Increment version to 2.10.0.0 (#237) Signed-off-by: opensearch-ci-bot <[email protected]> Co-authored-by: opensearch-ci-bot <[email protected]> * feat: update ui component and use fallback (#254) (#256) Signed-off-by: tygao <[email protected]> * feat: replace dash with em dash (#255) (#257) Signed-off-by: Lin Wang <[email protected]> (cherry picked from commit b817d98) Co-authored-by: Lin Wang <[email protected]> * doc: add release notes for 2.10 (#258) (#259) * doc: add release notes for 2.10 Signed-off-by: tygao <[email protected]> * doc: append prs Signed-off-by: tygao <[email protected]> * doc: separate prs Signed-off-by: tygao <[email protected]> --------- Signed-off-by: tygao <[email protected]> (cherry picked from commit db0003d) Co-authored-by: tygao <[email protected]> * Increment version to 2.11.0.0 (#265) Signed-off-by: opensearch-ci-bot <[email protected]> Co-authored-by: opensearch-ci-bot <[email protected]> * doc: add release notes for 2.11.0.0 (#276) (#277) Signed-off-by: Lin Wang <[email protected]> (cherry picked from commit cc2d58e) Co-authored-by: Lin Wang <[email protected]> * Onboard jenkins build docker image to ml-commons-dashboards github ci checks 2.x (#281) Signed-off-by: Peter Zhu <[email protected]> * Increment version to 2.12.0.0 (#279) Signed-off-by: opensearch-ci-bot <[email protected]> Co-authored-by: opensearch-ci-bot <[email protected]> --------- Signed-off-by: opensearch-ci-bot <[email protected]> Signed-off-by: tygao <[email protected]> Signed-off-by: Peter Zhu <[email protected]> Signed-off-by: Lin Wang <[email protected]> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <[email protected]> Co-authored-by: raintygao <[email protected]> Co-authored-by: Peter Zhu <[email protected]>
opensearch-project · Nov 16, 2023 · 54f8603 · 54f8603
1 parent 34c7157
commit 54f8603
Show file tree

Hide file tree

Showing 43 changed files with 1,402 additions and 320 deletions.
diff --git a/.github/workflows/unit-tests-workflow.yml b/.github/workflows/unit-tests-workflow.yml
@@ -9,41 +9,39 @@ on:
 env:
   OPENSEARCH_DASHBOARDS_VERSION: '2.x'
 jobs:
+  Get-CI-Image-Tag:
+    uses: opensearch-project/opensearch-build/.github/workflows/get-ci-image-tag.yml@main
+    with:
+      product: opensearch-dashboards
+
   tests:
     name: Run unit tests
     runs-on: ubuntu-latest
+    needs: Get-CI-Image-Tag
+    container:
+      # using the same image which is used by opensearch-build team to build the OpenSearch Distribution
+      # this image tag is subject to change as more dependencies and updates will arrive over time
+      image: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-version-linux }}
+      # need to switch to root so that github actions can install runner binary on container without permission issues.
+      options: --user root
+
     steps:
       - name: Checkout OpenSearch Dashboards
         uses: actions/checkout@v2
         with:
           repository: opensearch-project/OpenSearch-Dashboards
           ref: ${{ env.OPENSEARCH_DASHBOARDS_VERSION }}
           path: OpenSearch-Dashboards
-      - name: Setup Node
-        uses: actions/setup-node@v3
-        with:
-          node-version-file: './OpenSearch-Dashboards/.nvmrc'
-          registry-url: 'https://registry.npmjs.org'
-      - name: Install Yarn
-        # Need to use bash to avoid having a windows/linux specific step
-        shell: bash
-        run: |
-          YARN_VERSION=$(node -p "require('./OpenSearch-Dashboards/package.json').engines.yarn")
-          echo "Installing yarn@$YARN_VERSION"
-          npm i -g yarn@$YARN_VERSION
-      - run: node -v
-      - run: yarn -v
       - name: Checkout ML Commons OpenSearch Dashboards plugin
         uses: actions/checkout@v2
         with:
           path: OpenSearch-Dashboards/plugins/ml-commons-dashboards
-      - name: Bootstrap plugin/opensearch-dashboards
-        run: |
-          cd OpenSearch-Dashboards/plugins/ml-commons-dashboards
-          yarn osd bootstrap
-      - name: Run tests with coverage
+      - name: Bootstrap / build / unittest
         run: |
-          cd OpenSearch-Dashboards/plugins/ml-commons-dashboards
-          yarn run test:jest --silent --coverage
+          chown -R 1000:1000 `pwd`
+          cd ./OpenSearch-Dashboards/
+          su `id -un 1000` -c "source $NVM_DIR/nvm.sh && nvm use && node -v && yarn -v &&
+                               cd ./plugins/ml-commons-dashboards &&
+                               whoami && yarn osd bootstrap && yarn run test:jest --silent --coverage"
       - name: Uploads coverage
         uses: codecov/codecov-action@v1
diff --git a/opensearch_dashboards.json b/opensearch_dashboards.json
@@ -1,7 +1,7 @@
 {
   "id": "mlCommonsDashboards",
-  "version": "2.8.0.0",
-  "opensearchDashboardsVersion": "2.8.0",
+  "version": "2.12.0.0",
+  "opensearchDashboardsVersion": "2.12.0",
   "server": true,
   "ui": true,
   "requiredPlugins": [

diff --git a/package.json b/package.json
@@ -4,7 +4,7 @@
   "config": {
     "id": "mlCommonsDashboards"
   },
-  "version": "2.8.0.0",
+  "version": "2.12.0.0",
   "scripts": {
     "build": "yarn plugin-helpers build",
     "plugin-helpers": "node ../../scripts/plugin_helpers",

diff --git a/public/apis/__mocks__/connector.ts b/public/apis/__mocks__/connector.ts
@@ -0,0 +1,24 @@
+/*
+ * Copyright OpenSearch Contributors
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+export class Connector {
+  public async getAll() {
+    return {
+      data: [
+        {
+          id: 'external-connector-1-id',
+          name: 'External Connector 1',
+        },
+      ],
+      total_connectors: 1,
+    };
+  }
+
+  public async getAllInternal() {
+    return {
+      data: ['Internal Connector 1', 'Common Connector'],
+    };
+  }
+}
diff --git a/public/apis/api_provider.ts b/public/apis/api_provider.ts
@@ -5,6 +5,7 @@
 
 import { ModelVersion } from './model_version';
 import { ModelAggregate } from './model_aggregate';
+import { Connector } from './connector';
 import { Model } from './model';
 import { ModelRepository } from './model_repository';
 import { Profile } from './profile';
@@ -19,6 +20,7 @@ const apiInstanceStore: {
   task: Task | undefined;
   modelRepository: ModelRepository | undefined;
   model: Model | undefined;
+  connector: Connector | undefined;
 } = {
   modelVersion: undefined,
   modelAggregate: undefined,
@@ -27,6 +29,7 @@ const apiInstanceStore: {
   task: undefined,
   modelRepository: undefined,
   model: undefined,
+  connector: undefined,
 };
 
 export class APIProvider {
@@ -37,6 +40,7 @@ export class APIProvider {
   public static getAPI(type: 'security'): Security;
   public static getAPI(type: 'modelRepository'): ModelRepository;
   public static getAPI(type: 'model'): Model;
+  public static getAPI(type: 'connector'): Connector;
   public static getAPI(type: keyof typeof apiInstanceStore) {
     if (apiInstanceStore[type]) {
       return apiInstanceStore[type]!;
@@ -77,6 +81,11 @@ export class APIProvider {
         apiInstanceStore.model = newInstance;
         return newInstance;
       }
+      case 'connector': {
+        const newInstance = new Connector();
+        apiInstanceStore.connector = newInstance;
+        return newInstance;
+      }
     }
   }
   public static clear() {

diff --git a/public/apis/connector.ts b/public/apis/connector.ts
@@ -0,0 +1,35 @@
+/*
+ * Copyright OpenSearch Contributors
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+import {
+  CONNECTOR_API_ENDPOINT,
+  INTERNAL_CONNECTOR_API_ENDPOINT,
+} from '../../server/routes/constants';
+import { InnerHttpProvider } from './inner_http_provider';
+
+export interface GetAllConnectorResponse {
+  data: Array<{
+    id: string;
+    name: string;
+    description?: string;
+  }>;
+  total_connectors: number;
+}
+
+interface GetAllInternalConnectorResponse {
+  data: string[];
+}
+
+export class Connector {
+  public getAll() {
+    return InnerHttpProvider.getHttp().get<GetAllConnectorResponse>(CONNECTOR_API_ENDPOINT);
+  }
+
+  public getAllInternal() {
+    return InnerHttpProvider.getHttp().get<GetAllInternalConnectorResponse>(
+      INTERNAL_CONNECTOR_API_ENDPOINT
+    );
+  }
+}
diff --git a/public/apis/model_version.ts b/public/apis/model_version.ts
@@ -34,6 +34,11 @@ export interface ModelVersionSearchItem {
   last_registered_time?: number;
   last_deployed_time?: number;
   last_undeployed_time?: number;
+  connector_id?: string;
+  connector?: {
+    name: string;
+    description?: string;
+  };
 }
 
 export interface ModelVersionDetail extends ModelVersionSearchItem {
@@ -106,9 +111,11 @@ export class ModelVersion {
     nameOrId?: string;
     versionOrKeyword?: string;
     modelIds?: string[];
+    extraQuery?: Record<string, any>;
   }) {
+    const { extraQuery, ...restQuery } = query;
     return InnerHttpProvider.getHttp().get<ModelVersionSearchResponse>(MODEL_VERSION_API_ENDPOINT, {
-      query,
+      query: extraQuery ? { ...restQuery, extra_query: JSON.stringify(extraQuery) } : restQuery,
     });
   }
 

diff --git a/public/components/common/options_filter/options_filter.tsx b/public/components/common/options_filter/options_filter.tsx
@@ -3,7 +3,7 @@
  * SPDX-License-Identifier: Apache-2.0
  */
 
-import React, { useCallback, useMemo, useRef, useState } from 'react';
+import React, { useCallback, useMemo, useState } from 'react';
 import {
   EuiPopover,
   EuiPopoverTitle,
@@ -16,6 +16,7 @@ import {
 import { OptionsFilterItem } from './options_filter_item';
 
 export interface OptionsFilterProps<T extends string | number = string> {
+  id?: string;
   name: string;
   searchPlaceholder: string;
   searchWidth?: number;
@@ -33,11 +34,8 @@ export const OptionsFilter = <T extends string | number = string>({
   searchPlaceholder,
   searchWidth,
   onChange,
+  ...restProps
 }: OptionsFilterProps<T>) => {
-  const valueRef = useRef(value);
-  valueRef.current = value;
-  const onChangeRef = useRef(onChange);
-  onChangeRef.current = onChange;
   const [isPopoverOpen, setIsPopoverOpen] = useState(false);
   const [searchText, setSearchText] = useState<string>();
 
@@ -61,17 +59,19 @@ export const OptionsFilter = <T extends string | number = string>({
     setIsPopoverOpen(false);
   }, []);
 
-  const handleFilterItemClick = useCallback((clickItemValue: T) => {
-    onChangeRef.current(
-      valueRef.current.includes(clickItemValue)
-        ? valueRef.current.filter((item) => item !== clickItemValue)
-        : valueRef.current.concat(clickItemValue)
-    );
-  }, []);
+  const handleFilterItemClick = useCallback(
+    (clickItemValue: T) => {
+      onChange(
+        value.includes(clickItemValue)
+          ? value.filter((item) => item !== clickItemValue)
+          : value.concat(clickItemValue)
+      );
+    },
+    [value, onChange]
+  );
 
   return (
     <EuiPopover
-      id="popoverExampleMultiSelect"
       button={
         <EuiFilterButton
           iconType="arrowDown"
@@ -85,6 +85,7 @@ export const OptionsFilter = <T extends string | number = string>({
       isOpen={isPopoverOpen}
       closePopover={closePopover}
       panelPaddingSize="none"
+      {...restProps}
     >
       <EuiPopoverTitle style={{ padding: 12 }}>
         <EuiFieldSearch

diff --git a/public/components/experiment_warning/__tests__/index.test.tsx b/public/components/experiment_warning/__tests__/index.test.tsx
diff --git a/public/components/experiment_warning/index.tsx b/public/components/experiment_warning/index.tsx
diff --git a/public/components/model/model_versions_panel/__tests__/model_version_table.test.tsx b/public/components/model/model_versions_panel/__tests__/model_version_table.test.tsx
@@ -5,7 +5,7 @@
 
 import React from 'react';
 import userEvent from '@testing-library/user-event';
-import { within } from '@testing-library/dom';
+import { within, fireEvent } from '@testing-library/dom';
 
 import { render, screen, waitFor } from '../../../../../test/test_utils';
 import { ModelVersionTable } from '../model_version_table';
@@ -45,7 +45,6 @@ describe('<ModelVersionTable />', () => {
   it(
     'should render sorted column and call onSort after sort change',
     async () => {
-      const user = userEvent.setup();
       const onSortMock = jest.fn();
       render(
         <ModelVersionTable
@@ -66,31 +65,38 @@ describe('<ModelVersionTable />', () => {
           timeout: 2000,
         }
       );
-      await user.click(screen.getByText('Version'));
+
       await waitFor(async () => {
+        fireEvent.click(screen.getByText('Version'));
         expect(screen.getByText('Sort A-Z').closest('li')).toHaveClass(
           'euiDataGridHeader__action--selected'
         );
       });
 
       expect(onSortMock).not.toHaveBeenCalled();
-      await user.click(screen.getByText('Sort Z-A'));
-      expect(onSortMock).toHaveBeenCalledWith([{ direction: 'desc', id: 'version' }]);
+      await waitFor(() => {
+        fireEvent.click(screen.getByText('Sort Z-A'));
+        expect(onSortMock).toHaveBeenCalledWith([{ direction: 'desc', id: 'version' }]);
+      });
     },
     40 * 1000
   );
 
   it(
     'should NOT render sort button for state and status column',
     async () => {
-      const user = userEvent.setup();
       render(<ModelVersionTable versions={[]} tags={[]} />);
 
-      await user.click(screen.getByText('State'));
-      expect(screen.queryByTitle('Sort A-Z')).toBeNull();
+      expect(screen.getByTestId('dataGridHeaderCell-state')).toBeInTheDocument();
+      await waitFor(() => {
+        fireEvent.click(screen.getByTestId('dataGridHeaderCell-state'));
+        expect(screen.queryByTitle('Sort A-Z')).toBeNull();
+      });
 
-      await user.click(screen.getByText('Status'));
-      expect(screen.queryByTitle('Sort A-Z')).toBeNull();
+      await waitFor(() => {
+        fireEvent.click(screen.getByTestId('dataGridHeaderCell-status'));
+        expect(screen.queryByTitle('Sort A-Z')).toBeNull();
+      });
     },
     20 * 1000
   );