ENG-1293: Update rasa-calm-demo to version 3.10.1
#57
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1 task
AsadHasan-Rasa
force-pushed
the
ENG-1293-update-rasa-calm-demo-to-version-3.10.1
branch
from
fcd29c0
to
6470a0e
Compare
…ailure, to assist in troubleshooting
Remove `utter_clarification_options_rasa` assertion in the following two failing tests, as now this step won't be reached: 1. e2e_tests_with_assertions/passing/disambiguation/user_sends_short_verb_only_message.yml::user sends short verb only message 2. e2e_tests_with_assertions/passing/negations/users_says_they_dont_want_the_former_option.yml::user says they don't want the former option
…nding test, corrected it now
AsadHasan-Rasa
force-pushed
the
ENG-1293-update-rasa-calm-demo-to-version-3.10.1
branch
from
19eaebb
to
63765a5
Compare
AsadHasan-Rasa
changed the title
rasa-calm-demo to version 3.10.1rasa-calm-demo to version 3.10.1
ancalita
reviewed
Sep 16, 2024
data/flows/patterns.yml
Outdated
| action: utter_clarification_options_rasa | ||
|
|
||
| pattern_clarification_with_human_handoff: |
There was a problem hiding this comment.
Is this a new pattern added to the default file in rasa-private? Or should this be overriding pattern handoff?
There was a problem hiding this comment.
iiuc this is a new pattern added to rasa-private recently in #966
There was a problem hiding this comment.
This file on main doesn't contain such pattern, so Rasa will not classify it as a pattern, but as a user defined flow: https://github.com/RasaHQ/rasa-private/blob/main/rasa/dialogue_understanding/patterns/default_flows_for_patterns.yml
There was a problem hiding this comment.
I looked at the PR and the changelog mentions that pattern_human_handoff can now be linked in other flows, therefore this flow should be overriding that pattern name.
ancalita
reviewed
Sep 16, 2024
| ``` | ||
|
|
||
| #### E2E tests with assertions |
There was a problem hiding this comment.
Thanks for updating the readme 👍🏻
ancalita
approved these changes
Sep 16, 2024
… on recent pattern-clarification changes
|
Currently CI blocked again due to: https://rasa-hq.slack.com/archives/C36SS4N8M/p1726502977324429 |
Issue seems resolved now and CI pipeline (and all tests therein) passed ✅ https://github.com/RasaHQ/rasa-calm-demo/actions/runs/10886834156 |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
djcowley
requested changes
Sep 17, 2024
… triggered in relevant conditions On these conditions: 1. When `pattern_clarification` is triggered twice 2. Or when login fails during check-portfolio at least 3 times
1 task
AsadHasan-Rasa
added a commit
that referenced
this pull request
Sep 18, 2024
* Add a new folder of updated e2e tests with assertions (#46) * update some e2e_tests to the new assertions format * adapt 2 correction test cases * adapt all cancellations, chitchat, disambiguation * adapt test cases in invalid_path, invalid_user_inputs, negations, potential_bugs * adapt digressions, flow_guards, skip_question * adapt tests for corrections * adapt all happy_path test cases * fix error in running datetime validation in the custom actions * add generative assertion test cases * update gitignore * Add assertions for failing and flaky tests * udate rasa-pro to 3.10.0rc1 * add mlflow optional dependency, add new workflow and new make commands * fix CI deprecation warning, fix failing test case * update threshold to prevent flakiness * update button payload and test case * fix flaky passing e2e test --------- Co-authored-by: Maksim Moiseikin <[email protected]> * Add example stub custom actions (#49) * add some updated existing tests in a separate folder, add a new CI workflow and make command * add chitchat generative assertions test case * update to rasa pro 3.10.0rc2 * set assertion_order_enabled to true in 1 test step * Replace hardcoded currency strings with a regex parsing [ENG-1303] This will fix a TypeError when we get a string like "50 dollars" * update to rc3 * update rasa-pro version (#54) * ENG-1293: Update `rasa-calm-demo` to version 3.10.1 (#57) * link to add_contact in pattern_clarification * link from check_portfolio to pattern_human_handoff * link from pattern_clarification to pattern_human_handoff * do not rephrase utter_human_handoff_not_available * ENG-1293: Added pattern_human_handoff flow * ENG-1293: Running tests with `--e2e-results`, and saving results on failure, to assist in troubleshooting * ENG-1293: Update `rasa-calm-demo` to version 3.10.1 * ENG-1293: Fix 2 failing tests Remove `utter_clarification_options_rasa` assertion in the following two failing tests, as now this step won't be reached: 1. e2e_tests_with_assertions/passing/disambiguation/user_sends_short_verb_only_message.yml::user sends short verb only message 2. e2e_tests_with_assertions/passing/negations/users_says_they_dont_want_the_former_option.yml::user says they don't want the former option * ENG-1293: fix mistake, had removed assertion from a wrong similar sounding test, corrected it now * ENG-1293: fix/update test `user_sends_short_verb_only_message`, based on recent pattern-clarification changes * ENG-1293: properly add `pattern_human_handoff` to default `pattern_clarification` * ENG-1293: Add tests to verify that link to `pattern_human_handoff` is triggered in relevant conditions On these conditions: 1. When `pattern_clarification` is triggered twice 2. Or when login fails during check-portfolio at least 3 times --------- Co-authored-by: Tanja Bunk <[email protected]> --------- Co-authored-by: Anca Lita <[email protected]> Co-authored-by: Maksim Moiseikin <[email protected]> Co-authored-by: Tanja Bunk <[email protected]>
This was referenced Sep 23, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Update rasa calm demo to version 3.10.1. As part of that:
add_contactinpattern_clarification#48.a. Updated/fixed effected tests accordingly.
b. Added tests to verify that link to
pattern_human_handoffis triggered in relevant conditions (when 2 clarifications reached or when 3 auth fails during portfolio-check)--e2e-resultsflag and saving test results for unsuccessful CI steps, to ease with local troubleshooting of test failures.Once above changes are merged to
release-3.10, thenrelease-3.10can be merged tomainto update to3.10.1.Note:
Currently CI pipeline is blocked though due to OpenAI rate-limit issue:Sorted: OpenAI usage budget limit is now updated/increased and this issue is resolved, so CI pipeline was able to proceed and all tests passing now ✅OpenAIException - Error code: 429 - ***'error': ***'message': 'You exceeded your current quota, please check your plan and billing details.(issue raised in dev-tribe), so currently keeping this PR as Draft.TODOs