Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🚀 Pre-release master -> staging_Mithril1 #896

Closed
17 tasks done
pcrespov opened this issue Mar 6, 2023 · 10 comments
Closed
17 tasks done

🚀 Pre-release master -> staging_Mithril1 #896

pcrespov opened this issue Mar 6, 2023 · 10 comments
Assignees
@pcrespov
Labels
release Preparation for pre-release/release
Milestone
Mithril

Comments

@pcrespov
Copy link
Member

pcrespov commented Mar 6, 2023 •
edited
Loading

In preparation for pre-release. Here an initial (incomplete) list of tasks to prepare before pre-releasing:

Motivation:

  • Tuesday weekly release

Devops check (⚠️ devops)

e2e testing check

  • Mon.
    • @pcrespov websocket CLOSED event not delivered to FE (lots of studies not working as expected)
  • Thr.
    • master is back online!
    • master e2e was off!!!
  • Fri.
    • Parallels tests failing
  • Mon.
    • Parallels failing (non-critical)

Commits (in order)

Summary

{"start": "2023-03-14T15:00:00.000Z", "end": "2023-03-14T16:30:00.000Z", "reason": "Release Mithril1"}

Release
@pcrespov pcrespov added the release Preparation for pre-release/release label Mar 6, 2023
@pcrespov pcrespov self-assigned this Mar 6, 2023
@pcrespov
Copy link
Member Author

pcrespov commented Mar 9, 2023

@mrnicegyu11 :For another test, I will check if the new deployment agent runs on aws-staging as well. To enable this without affecting inhouse users, I will pause the deployment agent on dalco-staging, potentially until the next staging-release (please take note @allexandre @crespo , thanks), but at least for today.

@pcrespov
Copy link
Member Author

pcrespov commented Mar 9, 2023 •
edited
Loading 

## What's Changed
* ⬆️ Autoscaling: requirements update by @sanderegg in https://github.com/ITISFoundation/osparc-simcore/pull/3916
* 🔨 Is3635/service settings jsonschemas by @pcrespov in https://github.com/ITISFoundation/osparc-simcore/pull/3703
* 🐛 Fixes ``/scheduled_maintenance`` entrypoint by @pcrespov in https://github.com/ITISFoundation/osparc-simcore/pull/3926
* ✨Progress is now sent to the FE when closing a study by @GitHK in https://github.com/ITISFoundation/osparc-simcore/pull/3919
* ✨ Frontend: Responsive Navigation Bar by @odeimaiz in https://github.com/ITISFoundation/osparc-simcore/pull/3925
* Bump minimist, mkdirp and extract-zip by @dependabot in https://github.com/ITISFoundation/osparc-simcore/pull/3927
* ✨ Adds entrypoint to check invitation and get associated email by @pcrespov in https://github.com/ITISFoundation/osparc-simcore/pull/3856
* ✨ S4L e2e: wait for S4L Splash Screen to disappear by @odeimaiz in https://github.com/ITISFoundation/osparc-simcore/pull/3933
* ✨Frontend User registration: Check Invitation email from token by @odeimaiz in https://github.com/ITISFoundation/osparc-simcore/pull/3935
* ⬆️ Maintenance/reqs upgrade (test/tools & service security patch)  (⚠️ devops) 🚨 by @matusdrobuliak66 in https://github.com/ITISFoundation/osparc-simcore/pull/3928
* 🐛 Using a more proper message type by @GitHK in https://github.com/ITISFoundation/osparc-simcore/pull/3936
* Removing fastapi constraints + upgrade by @matusdrobuliak66 in https://github.com/ITISFoundation/osparc-simcore/pull/3945


**Full Changelog**: https://github.com/ITISFoundation/osparc-simcore/compare/staging_ResistanceIsFutile16...staging_Mithril1

@pcrespov
Copy link
Member Author

@mrnicegyu11
Copy link
Member

DevOps changes acknowledged and implemented in ITISFoundation/osparc-ops-environments#116 (review)

@Surfict
Copy link
Contributor

Surfict commented Mar 14, 2023

acknowledged !

@pcrespov
Copy link
Member Author

  • Unclosed dy-sidecars
    • None in AWS staging
    • 23 in dalco staging. @GitHK will close and report on it below

@GitHK
Copy link
Contributor

GitHK commented Mar 14, 2023 •
edited
Loading

It was observed that after the agent redeployed rabbit service, the dy-sidecar can no longer connect to it.

WARNING: [2023-02-28 14:05:06,599/MainProcess] [aio_pika.robust_connection:connect(141)]  -  Connection attempt to "amqp://admin:******@staging_rabbit:5672/?name=dynamic-sidecar_bdd1eedf-5c45-45e8-87af-d42da3374b4b_4c947f9982af_7" failed: [Errno -2] Name or service not known. Reconnecting after 5 seconds.

Apparently there is an issue with the service name resolution on the network?

Only enough the director-v2 cli manage to correctly run when closing the service:

root@osparc-dalco-08-staging-simcore_staging_director-v2-1:/home/scu# simcore-service-director-v2 close-and-save-service bdd1eedf-5c45-45e8-87af-d42da3374b4b
Starting cleanup for service bdd1eedf-5c45-45e8-87af-d42da3374b4b
* disabling service observation
* deleting service containers
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:01 finished
* saving service state
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:03 finished
* pushing service outputs
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:02 finished
* deleting service docker resources and removing service
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:01 finished

@mrnicegyu11
Copy link
Member

mrnicegyu11 commented Mar 14, 2023 •
edited
Loading

Issues encountered:

  • Deployment agent v0.11.1 throws error when upon init not all git repos match in their tags, doesnt keep polling
  • Deployment agent v0.11.1 complains/spams in mattermost when commits of watched git-repos are re-tagged (fix via https://stackoverflow.com/a/58438257 )
  • On dalco-staging, SERVICEUUID.services.testing.osparc-staging.speag.com is served with a self-signed certificate
  • The e2e-ops filestash test fails due to the filestash version upgrade on aws-staging

All of these issues dont impact osparc core operations and can be fixed at a later point.

@pcrespov
Copy link
Member Author

@Surfict
Copy link
Contributor

Surfict commented Mar 14, 2023

Quick fix done for https://git.speag.com/oSparc/osparc-ops-environments/-/issues/497 : staging-manager from m6i.2xlarge to m6i.4xlarge

@mrnicegyu11 mrnicegyu11 mentioned this issue Mar 14, 2023
Merged
@pcrespov pcrespov added this to the Mithril milestone Mar 17, 2023
mrnicegyu11 added a commit to ITISFoundation/osparc-deployment-agent that referenced this issue Mar 20, 2023
@mrnicegyu11
🐛 Fixes: Bugs observed during Mithril1 prerelease (#173)
6af20a7 
    🐛 Fix bug: Deployment agent v0.11.1 complains/spams in mattermost when commits of watched git-repos are re-tagged (fix via https://stackoverflow.com/a/58438257 )
    🐛 Fix bug: Deployment agent needs to die when init fails due to missing tag-sync, since the GitUrlWatcher object will not be initialized and not be ready for polling. A failure and full restart of the deployment agent is (without further refactoring) the more graceful option out of this

See as well: ITISFoundation/osparc-issues#896
@pcrespov pcrespov mentioned this issue Mar 23, 2023
Closed
31 tasks
@sanderegg sanderegg mentioned this issue Mar 27, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pcrespov pcrespov

Labels
release Preparation for pre-release/release
Projects
None yet
Milestone
Mithril
Development

No branches or pull requests
4 participants
@Surfict @GitHK @mrnicegyu11 @pcrespov