Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

optimize locking in katalogus.py, reuse available data #3752

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

underdarknl
Copy link
Contributor

Changes

We see long running locks when many organizations are present due to fetching the plugin information from the katalogus.
This blocks the boefje and normalizer threads.
We also do tree requests for every organization to the same katalogus endpoint, once for plugins, once for boefjes and once for normalizers.

This Pr minimizes the lock duration by building the new dict locally before copying. it also reuses this info for the boefjes and normalizer loops.

Issue link

You have to create an issue to link to this PR. If this really is not possible, write a very detailed description here and add this PR to the project board directly.

Please add the link to the issue after "Closes".

Closes ...

Demo

Please add some proof in the form of screenshots or screen recordings to show (off) new functionality, if there are interesting new features for end-users.

QA notes

Please add some information for QA on how to test the newly created code.


Code Checklist

  • All the commits in this PR are properly PGP-signed and verified.
  • This PR only contains functionality relevant to the issue.
  • I have written unit tests for the changes or fixes I made.
  • I have checked the documentation and made changes where necessary.
  • I have performed a self-review of my code and refactored it to the best of my abilities.
  • Tickets have been created for newly discovered issues.
  • For any non-trivial functionality, I have added integration and/or end-to-end tests.
  • I have informed others of any required .env changes files if required and changed the .env-dist accordingly.
  • I have included comments in the code to elaborate on what is not self-evident from the code itself, including references to issues and discussions online, or implicit behavior of an interface.

Checklist for code reviewers:

Copy-paste the checklist from the docs/source/templates folder into your comment.


Checklist for QA:

Copy-paste the checklist from the docs/source/templates folder into your comment.

@underdarknl underdarknl requested a review from a team as a code owner October 29, 2024 12:28
Copy link

sonarcloud bot commented Oct 29, 2024

Quality Gate Failed Quality Gate failed

Failed conditions
7.0% Coverage on New Code (required ≥ 80%)

See analysis details on SonarCloud

@jpbruinsslot
Copy link
Contributor

Related: #3357

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants