Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor - IAMbic should respect resource limit #391

Open
smoy opened this issue May 5, 2023 · 0 comments
Open

Refactor - IAMbic should respect resource limit #391

smoy opened this issue May 5, 2023 · 0 comments
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@smoy
Copy link
Contributor

smoy commented May 5, 2023

Is your feature request related to a problem? Please describe.
Current IAMBic AWS plugin implementation scales by number of AWS Accounts. If there are N accounts, the parent process will attempt to use (N + 1) * 50 TCP connections due to how it's using boto3 clients. If administrator needs to put a bound on how much resources (let's say ulimit of 1024), then IAMBic will not gracefully work within the limit. Yes, higher limit can make IAMbic work faster; lower limit should not lead to a crash. (Related to #386 ). lower limit should just mean it takes longer wall time to complete.

Describe the solution you'd like
IAMbic once consume allowed resources, it will queue pending work and consume the remaining work as resources are free up by previous work.

Describe alternatives you've considered
Knob tuning. One can just pre-tune all the parallelization parameters; however, no one like knob tuning. A typical work queue, producer, consumer is preferred.

Additional context

@smoy smoy added the enhancement New feature or request label May 5, 2023
@smoy smoy added the help wanted Extra attention is needed label May 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant