[WIP] Autoscaling management #188
Conversation
| @@ -275,6 +275,10 @@ def strip_sensitive_data(event, hint): | |||
| 'task': 'mcserver.tasks.delete_pingdom_sessions', | |||
| 'schedule': crontab(hour='*', minute=0) | |||
| }, | |||
| 'submit_cloudwatch_metric': { | |||
| 'task': 'mcserver.tasks.submit_cloudwatch_metrics', | |||
| 'schedule': crontab(minute='*') | |||
There was a problem hiding this comment.
@olehkorkh-planeks @sashasimkin @suhlrich is it possible to have a finer resolution or is 1 minute the best we can do? It seems a little slow
There was a problem hiding this comment.
@antoinefalisse this is what I suggested here in point #1.
There was a problem hiding this comment.
Can you explain what you mean with aggregates of 1 minutes?
There was a problem hiding this comment.
@antoinefalisse check this documentation about target tracking(search for "aggregated at a one-minute granularity" text if highlight doesn't work).
Few more arguments against higher resolution metrics:
- From what I understand the trials processed around 1-60 minutes, so I don't expect rapid changes when all trials are processed
- I've configured scale in/out cooldowns to 5m to avoid instances flapping
- I saw in the code logic to cool-down the worker instance before unprotecting it, which adds to the delay
mcserver/utils.py
Outdated
| """ | ||
| client = boto3.client( | ||
| 'cloudwatch', | ||
| aws_access_key_id=settings.AWS_ACCESS_KEY_ID, |
There was a problem hiding this comment.
You don't need to explicitly specify these because I assigned task role to the celery task which will allow boto3 to infer the credentials from environment.
mcserver/utils.py
Outdated
| 'cloudwatch', | ||
| aws_access_key_id=settings.AWS_ACCESS_KEY_ID, | ||
| aws_secret_access_key=settings.AWS_SECRET_ACCESS_KEY, | ||
| region_name='us-west-2', |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
No description provided.