opengrok-reindex-project Triggers API-Ratelimit that doesn't reset

[dkr91]

Describe the bug
I am using two cronjobs and scripts to sync and reindex our current codebase hourly and our LTS releases weekly, including pipeing the logs to sperate files for better debuging. Doing so has triggerd Opengrok to return this error

The Script uses this

opengrok-mirror -c /opengrok/etc/mirror.yml --api_timeout 1000 -I -U http://localhost:8080/search/ "$PROJECT" -b -L /opengrok/logs/ --strip-outgoing
/usr/bin/python3 /usr/local/bin/opengrok-reindex-project --printoutput --api_timeout 1000 --jar /opengrok/lib/opengrok.jar -U http://localhost:8080/search/ -P "$PROJECT" -- --connectTimeout 8 -r dirbased -G -m 1024 --leadingWildCards on -c /usr/local/bin/ctags -H "$PROJECT" "$INDEXER_OPT" >> "$LOGFILE"

Cannot get the 'sourceRoot' config value from the web application: 
HTTPConnectionPool(host='localhost', port=8080): Max retries exceeded with url: /search/api/v1/configuration/sourceRoot 
(Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8943626e00>: Failed to establish a new connection: [Errno 111] Connection refused'))

To Reproduce
Steps to reproduce the behavior: Build a script maunually triggeringopengrok-mirror and opengrok-reindex-project

Expected behavior
I can run the script with a Cronjob and fine-adjust when which projects get reindexed.

[vladak]

The Max retries exceeded with url error comes from the Python requests module used by the Python tools, the underlying reason listed as Connection refused. If the reason is indeed rate limiting on the side of the server (which I don't see anywhere in the description) then the answer lies in changing the server's configuration to allow more simultaneous connections to the given URL. In my deployment I usually run the opengrok-sync program with the number of workers set to 16 (which leads to bunch of GET requests to the server from both the opengrok-mirror and opengrok-reindex-project programs) processing tens of projects and have not seen such problem.