Error fetching 'feeds/us-ma.json'

[transitous-bot]

CC: @kbloom

An error occured while fetching a feed from feeds/us-ma.json.
If the error is not temporary, please consider replacing or removing the feed.
Thanks!

Here are the logs of the error(s):
On 2024-09-12 09:18:31 UTC:

Postprocessing us-ma-MBTA with gtfstidy…
Parsing GTFS feed in '/woodpecker/src/github.com/public-transport/transitous/out/.tmp-us-ma_MBTA.gtfs.zip' ... done.
Removing trips travelling too fast...done. (-58 trips [-0.08%])
Removing redundant stops... done. (-18 stops [-0.18%])
Removing redundant routes... done. (-0 routes [-0.00%])
Removing service duplicates... done. (-47 services [-47.96%])
Removing redundant trips... done. (-617 trips [-0.81%])
Removing service duplicates... done. (-586 services [-91.71%])
Outputting GTFS feed to '/woodpecker/src/github.com/public-transport/transitous/out/.tmp-us-ma_MBTA.gtfs.zip'... done.

Fetching us-ma-MBTA…
Fetching us-ma-LoganExpress…
Postprocessing us-ma-LoganExpress with gtfstidy…
Parsing GTFS feed in '/woodpecker/src/github.com/public-transport/transitous/out/.tmp-us-ma_LoganExpress.gtfs.zip' ... done.
Removing trips travelling too fast...done. (-0 trips [-0.00%])
Removing redundant stops... done. (-0 stops [-0.00%])
Removing redundant routes... done. (-0 routes [-0.00%])
Removing service duplicates... done. (-0 services [-0.00%])
Removing redundant trips... done. (-3 trips [-0.14%])
Removing service duplicates... done. (-0 services [-0.00%])
Outputting GTFS feed to '/woodpecker/src/github.com/public-transport/transitous/out/.tmp-us-ma_LoganExpress.gtfs.zip'... done.

Fetching us-ma-MWRTA…
Error: Could not fetch us-ma-MWRTA: HTTPConnectionPool(host='vc.mwrta.com', port=80): Max retries exceeded with url: /gtfs/google_transit.zip (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7638980e74d0>: Failed to establish a new connection: [Errno -2] Name or service not known'))
Fetching us-ma-BrocktonMA…
Fetching us-ma-LexingtonMA…
Fetching us-ma-Massport…
Postprocessing us-ma-Massport with gtfstidy…
Parsing GTFS feed in '/woodpecker/src/github.com/public-transport/transitous/out/.tmp-us-ma_Massport.gtfs.zip' ... done.
Removing trips travelling too fast...done. (-0 trips [-0.00%])
Removing redundant stops... done. (-0 stops [-0.00%])
Removing redundant routes... done. (-0 routes [-0.00%])
Removing service duplicates... done. (-0 services [-0.00%])
Removing redundant trips... done. (-3 trips [-0.14%])
Removing service duplicates... done. (-0 services [-0.00%])
Outputting GTFS feed to '/woodpecker/src/github.com/public-transport/transitous/out/.tmp-us-ma_Massport.gtfs.zip'... done.

Error: 1 errors occurred during fetching.

On 2024-09-12 09:20:53 UTC

Fetching us-ma-MBTA…
Fetching us-ma-LoganExpress…
Fetching us-ma-MWRTA…
Error: Could not fetch us-ma-MWRTA: HTTPConnectionPool(host='vc.mwrta.com', port=80): Max retries exceeded with url: /gtfs/google_transit.zip (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7c5bb7835f90>: Failed to establish a new connection: [Errno -2] Name or service not known'))
Fetching us-ma-BrocktonMA…
Fetching us-ma-LexingtonMA…
Fetching us-ma-Massport…
Error: 1 errors occurred during fetching.

[kbloom]

I can reach this URL from my browser, so I think this is transient.

I think that Transitous is in a better position than I am to determine which errors are transient. Could you perhaps add monitoring for how often a particular feed is unreachable, and only file these bugs after a certain threshold has passed
Some possible thresholds:

• the feed was unavailable for a whole week, in which case it's probably gone away
• the feed was unavailable > 50% of the time over the course of a week, in which case we might not want to tolerate its flakiness

[jbruechert]

Yes, that would be nice. I'll try to look into it, not sure when yet though.

I'd want something like a json file stored in the cache to record things like this.