Change GitHub clone URL implementation #5266
Conversation
✅ Deploy Preview for nextflow-docs-staging canceled.
|
Signed-off-by: Nathan Thorpe <[email protected]>
nathanthorpe
force-pushed
the
github-rate-limit-fix
branch
from
5579930
to
cc1c22d
Compare
| Map response = invokeAndParseResponse( getEndpointUrl() ) | ||
|
|
||
| def result = response.get('clone_url') | ||
| if( !result ) | ||
| throw new IllegalStateException("Missing clone URL for: $project") | ||
|
|
||
| return result | ||
| return "${config.server}/${project}.git" |
There was a problem hiding this comment.
Considering the method invokeAndParseResponse is annotated with @memoized it should be invoked only once.
Not sure why this is happening.
There was a problem hiding this comment.
It only happens when there are many concurrent nextflow runs.
It is being performed in AWS Batch, so it doesn't have the opportunity to use the cache.
There was a problem hiding this comment.
Understand. However I believe this would solve much because in case other API calls will be made to access the config file, cloning the repo, etc.
Is this not happening because the GitHub API token is not specified?
There was a problem hiding this comment.
Ah right ok, didn't realize the config file was read that way.
Cloning the repo luckily doesn't use the rate quota.
I would rather not have users configure GitHub creds, but maybe I can find another solution for this.
We've come across an issue where the 60 requests per hour rate limit causes the workflow to error out. It stems from having to lookup the Clone URL via the API.
I propose that the Clone URL be hardcoded so that we don't need to call the GitHub API.
Alternatively, I could put it under try / catch for the RateLimitExceeded exception