patch time sync plugin to handle stack too deep errors

[zanete]

Why: Sub of #949 . The framework is implemented to deal with very granular time units (seconds), but in real life the units for measurements come in larger time intervals that the IF is unable to handle, causing overflow errors.
What: Find an MVP solution for the time being to enable working with larger time intervals. Will return to this problem at a later date to implement a more robust solution

Context

We often have patchy data, we don't have granular data, or we are only interested in values at e.g. monthly time resolution.

We don't work well in these situations, even though they are very common - we're really just set up for situations where we can load in granular time series.

We often see people (and this is what I ended up doing in v1 of the GSF site manifest) have a single timestep with a duration of 1 month or one year in seconds and then execute a single, large pipeline that eventually yields SCI. This is because it's very fiddly and repetitive to compartmentalise into individual components and often we can't / don't really need to access granular temporal data anyway.

If I just want an overall SCI value for my entire application, I'm not going to spend a week trying to source granular data, or even worse naively chunking up the single value that fits my needs into time block just because that's what a manifest favours.

As a user in this situation it's much easier to interpret my manifest when I've just constructed a pipeline, can follow through the logic and read off my single value as compared to interpreting aggregated data, which is presented in quite a non-aesthetic way.

A separate time issue is the current behaviour of time sync. It works as a plugin, which means we have to invoke it in every pipeline in the tree. But we encountered some problematic side effects associated with making it a “global” feature. On balance we decided to stick with the status quo for now, but we need to revisit the way we handle time, perhaps starting from a blank page.

The current time-sync plugin often errors out with “RangeError: Maximum call stack size exceeded” when we’re using time units greater than minutes. For example, time-sync applied to monthly data, even when we just want to create time series at a resolution of 10 or even 100 hours, throws with this error. Maybe this is because of our “base” resolution being fixed at 1s, which creates too much data when we’re trying to operate over monthly data. Either way, this needs fixing or we’re only applicable to observations over seconds -> minute ranges, not week -> month -> year ranges.

Scope of work:

• @jmcook1186 Provide a manifest file that throws the errors
• Discuss options for a quick n dirty solution
• Implement solution

[zanete]

There is also a community PR open that could fix this

[zanete]

@jmcook1186 any chance you could point us to the manifest that was failing?