Support insight with RAG #266

qianheng-aws · 2024-08-30T08:14:14Z

Description

The alert analysis assistant has supported summary generation now. This PR provides insight with RAG if related insight has been configured. There could be 2 types of knowledge base:
1. OpenSearch’ technical knowledge base for cluster alerts, which is setup by default
2. Customized knowledge base for customer’s own alerts, which is configured by customers and setup by CloudFormation workflow.

Apart from supporting insight, this feature also refines the UX.
Screen Snapshot:

Alerting page with assistant icon
Show loading content if clicked the assistant icon
Show summary and insight tip icon if detecting related insight agent
Click insight tip icon to jump to view insight, the title has back button to go back to summary
We won't show insight tip icon if not having detected related insight agent

Issues Resolved

Check List

New functionality includes testing.
- All tests pass, including unit test, integration test.
New functionality has user manual doc added.
Commits are signed per the DCO using --signoff.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Heng Qian <[email protected]>

codecov · 2024-09-02T02:36:44Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.64%. Comparing base (4f33fa0) to head (3ba3482).
Report is 10 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #266      +/-   ##
==========================================
- Coverage   90.34%   88.64%   -1.71%     
==========================================
  Files          59       60       +1     
  Lines        1575     1700     +125     
  Branches      377      419      +42     
==========================================
+ Hits         1423     1507      +84     
- Misses        150      191      +41     
  Partials        2        2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Heng Qian <[email protected]>

public/components/incontext_insight/index.scss

CHANGELOG.md

server/routes/get_agent.ts

server/routes/summary_routes.ts

Signed-off-by: Heng Qian <[email protected]>

…ithRAG # Conflicts: # CHANGELOG.md

…ithRAG # Conflicts: # CHANGELOG.md # common/constants/llm.ts # server/plugin.ts

Signed-off-by: Heng Qian <[email protected]>

…ithRAG # Conflicts: # server/plugin.ts

Hailong-am · 2024-09-11T06:39:00Z

public/components/incontext_insight/generate_popover_body.tsx

+      try {
+        contextObj = (await incontextInsight.contextProvider?.()) ?? undefined;
+      } catch (e) {
+        console.error('Error executing contextProvider:', e);


please remove console logs

Hailong-am · 2024-09-11T06:43:53Z

server/routes/summary_routes.ts

+      const response = await assistantClient.executeAgentByName(SUMMARY_AGENT_CONFIG_ID, {
+        context: req.body.context,
+        question: req.body.question,
+      });
+      let summary;
+      let insightAgentIdExists = false;
+      try {
+        if (req.body.insightType) {
+          // We have separate agent for os_insight and user_insight. And for user_insight, we can
+          // only get it by searching on name since it is not stored in agent config.
+          if (req.body.insightType === 'os_insight') {
+            if (!osInsightAgentId) {
+              osInsightAgentId = await getAgent(OS_INSIGHT_AGENT_CONFIG_ID, client);
+            }
+            insightAgentIdExists = !!osInsightAgentId;
+          } else if (req.body.insightType === 'user_insight') {
+            if (req.body.type === 'alerts') {
+              if (!userInsightAgentId) {
+                userInsightAgentId = await searchAgentByName('KB_For_Alert_Insight', client);
+              }
+            }
+            insightAgentIdExists = !!userInsightAgentId;
+          }
+        }


nit: we can refactor these two api calls run in parallel with Promise.all() as they don't depend on each other

Hailong-am · 2024-09-11T06:45:37Z

server/routes/summary_routes.ts

+      const insightAgentId =
+        req.body.insightType === 'os_insight' ? osInsightAgentId : userInsightAgentId;
+      if (!insightAgentId) {
+        context.assistant_plugin.logger.info(


maybe debug level or remove the log

opensearch-trigger-bot · 2024-09-11T06:50:34Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/dashboards-assistant/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/dashboards-assistant/backport-2.x
# Create a new branch
git switch --create backport/backport-266-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 4e68047cd0700c33c016cf5bf0ad43f66f2ed7ba
# Push it to GitHub
git push --set-upstream origin backport/backport-266-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/dashboards-assistant/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-266-to-2.x.

songkant-aws · 2024-09-12T01:51:17Z

Looks like we need to backport this PR to 2.x. Since #267 resolved some conflicts based on this PR, could you @qianheng-aws fix the backport to unblock #267 backport?

* Support insight with RAG Signed-off-by: Heng Qian <[email protected]> * Add change in CHANGELOG.md Signed-off-by: Heng Qian <[email protected]> * Put footer in the same panel with content to match UX design Signed-off-by: Heng Qian <[email protected]> * Refine alert prompt Signed-off-by: Heng Qian <[email protected]> * set CSS scrollbar-width to thin Signed-off-by: Heng Qian <[email protected]> * Hide insight agent id from front-end Signed-off-by: Heng Qian <[email protected]> * Change summary agent config id Signed-off-by: Heng Qian <[email protected]> * Address comments Signed-off-by: Heng Qian <[email protected]> * Fix UT Signed-off-by: Heng Qian <[email protected]> * Change agent execute API Signed-off-by: Heng Qian <[email protected]> * Remove prompt from node JS server Signed-off-by: Heng Qian <[email protected]> * Replace CSS with component property Signed-off-by: Heng Qian <[email protected]> --------- Signed-off-by: Heng Qian <[email protected]> (cherry picked from commit 4e68047) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> # Conflicts: # CHANGELOG.md

* Support insight with RAG Signed-off-by: Heng Qian <[email protected]> * Add change in CHANGELOG.md Signed-off-by: Heng Qian <[email protected]> * Put footer in the same panel with content to match UX design Signed-off-by: Heng Qian <[email protected]> * Refine alert prompt Signed-off-by: Heng Qian <[email protected]> * set CSS scrollbar-width to thin Signed-off-by: Heng Qian <[email protected]> * Hide insight agent id from front-end Signed-off-by: Heng Qian <[email protected]> * Change summary agent config id Signed-off-by: Heng Qian <[email protected]> * Address comments Signed-off-by: Heng Qian <[email protected]> * Fix UT Signed-off-by: Heng Qian <[email protected]> * Change agent execute API Signed-off-by: Heng Qian <[email protected]> * Remove prompt from node JS server Signed-off-by: Heng Qian <[email protected]> * Replace CSS with component property Signed-off-by: Heng Qian <[email protected]> --------- Signed-off-by: Heng Qian <[email protected]> (cherry picked from commit 4e68047) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> # Conflicts: # CHANGELOG.md Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

Support insight with RAG

2e25985

Signed-off-by: Heng Qian <[email protected]>

qianheng-aws requested review from lezzago, joshuali925, anirudha, ruanyl, SuZhou-Joe, xluo-aws, gaobinlong, Hailong-am, wanglam and raintygao as code owners August 30, 2024 08:14

qianheng-aws added 3 commits August 30, 2024 16:15

Add change in CHANGELOG.md

21e2278

Signed-off-by: Heng Qian <[email protected]>

Put footer in the same panel with content to match UX design

662923b

Signed-off-by: Heng Qian <[email protected]>

Refine alert prompt

3ba3482

Signed-off-by: Heng Qian <[email protected]>

qianheng-aws added 3 commits September 2, 2024 11:06

set CSS scrollbar-width to thin

1956d49

Signed-off-by: Heng Qian <[email protected]>

Hide insight agent id from front-end

14f9030

Signed-off-by: Heng Qian <[email protected]>

Change summary agent config id

f799e15

Signed-off-by: Heng Qian <[email protected]>

Hailong-am reviewed Sep 4, 2024

View reviewed changes

public/components/incontext_insight/index.scss Outdated Show resolved Hide resolved

CHANGELOG.md Outdated Show resolved Hide resolved

server/routes/get_agent.ts Outdated Show resolved Hide resolved

server/routes/summary_routes.ts Outdated Show resolved Hide resolved

qianheng-aws added 9 commits September 4, 2024 11:12

Address comments

d2278f1

Signed-off-by: Heng Qian <[email protected]>

Fix UT

a0e1a89

Signed-off-by: Heng Qian <[email protected]>

Merge remote-tracking branch 'refs/remotes/origin/main' into insightW…

07a5cff

…ithRAG # Conflicts: # CHANGELOG.md

Merge remote-tracking branch 'refs/remotes/origin/main' into insightW…

4002c9b

…ithRAG # Conflicts: # CHANGELOG.md # common/constants/llm.ts # server/plugin.ts

Change agent execute API

15d44c3

Signed-off-by: Heng Qian <[email protected]>

Remove prompt from node JS server

14d6d0b

Signed-off-by: Heng Qian <[email protected]>

Merge branch 'main' into insightWithRAG

d94e244

Replace CSS with component property

cde988b

Signed-off-by: Heng Qian <[email protected]>

Merge remote-tracking branch 'refs/remotes/origin/main' into insightW…

7602a9c

…ithRAG # Conflicts: # server/plugin.ts

Hailong-am approved these changes Sep 11, 2024

View reviewed changes

Hailong-am added the backport 2.x Trigger the backport flow to 2.x label Sep 11, 2024

Hailong-am merged commit 4e68047 into opensearch-project:main Sep 11, 2024
10 of 11 checks passed

opensearch-trigger-bot bot added the failed backport label Sep 11, 2024

songkant-aws mentioned this pull request Sep 12, 2024

Add assistant capabilities to control rendering components #267

Merged

4 tasks

ruanyl added backport 2.x Trigger the backport flow to 2.x and removed backport 2.x Trigger the backport flow to 2.x labels Sep 12, 2024

opensearch-trigger-bot bot mentioned this pull request Sep 12, 2024

[Backport 2.x] Support insight with RAG #301

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support insight with RAG #266

Support insight with RAG #266

qianheng-aws commented Aug 30, 2024 •

edited

Loading

codecov bot commented Sep 2, 2024

Hailong-am Sep 11, 2024

Hailong-am Sep 11, 2024

Hailong-am Sep 11, 2024

opensearch-trigger-bot bot commented Sep 11, 2024

songkant-aws commented Sep 12, 2024

Support insight with RAG #266

Support insight with RAG #266

Conversation

qianheng-aws commented Aug 30, 2024 • edited Loading

Description

Issues Resolved

Check List

codecov bot commented Sep 2, 2024

Codecov Report

Hailong-am Sep 11, 2024

Choose a reason for hiding this comment

Hailong-am Sep 11, 2024

Choose a reason for hiding this comment

Hailong-am Sep 11, 2024

Choose a reason for hiding this comment

opensearch-trigger-bot bot commented Sep 11, 2024

songkant-aws commented Sep 12, 2024

qianheng-aws commented Aug 30, 2024 •

edited

Loading