Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Processing Arbitrary Times #18

Open
patchneranartkomol opened this issue Jun 12, 2019 · 3 comments
Open

Support Processing Arbitrary Times #18

patchneranartkomol opened this issue Jun 12, 2019 · 3 comments

Comments

@patchneranartkomol
Copy link

Some of the scripts generate data files at the exact time the script runs by specifying:

    now = pendulum.now("UTC")

https://github.com/metro-ontime/performance_tracker/blob/master/performance_tracker/process_vehicles.py

https://github.com/metro-ontime/performance_tracker/blob/master/performance_tracker/query_vehicles.py

As an admin, I would like to process an arbitrary range of times. I should be able to pass a window of time over which generate the data files. When that isn't provided, we can default to UTC now.

@ctsexton
Copy link
Member

Partially enabled by a03c87c
A bit more work required to get this working reliably!

@ctsexton
Copy link
Member

We probably need a smarter method of grabbing schedule data. At the moment we just download the latest schedule, which is updated daily. In order to download past schedules, we need to access previous commits in this repo: https://gitlab.com/LACMTA/gtfs_rail

@ctsexton
Copy link
Member

However, the reason we want to process arbitrary dates in the first place kind of assumes we already have the data readily available in the first place. And at this point in time, we do have over 12 months of schedule data already processed and uploaded to S3 as well as (unprocessed) tracked vehicle data.

So, for now we can just focus on processing raw vehicle data for previous dates. This might just require a one-off script to access and process the data we already have.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants