Remember store status and skip updating on first run if it is current.

[partim]

This PR speeds up the initial validation run in server mode if possible by creating the data set from the store only.

The store now saves its status when a validation run has been successfully completed. This status consists of the time of update and the list of the trust anchor keys used. When the server is started, it checks that this status is available, that the update time was less then configured refresh time ago, and that the requested trust anchor keys are all included. If so, it will do a quick validation run using the store only before immediately starting a regular run.

At the start of each validation run, the store deletes the stored status so that the initial no-update validation run only happens if the data set in the store is complete. This may be unnecessarily strict – any opinions?

[DRiKE]

At the start of each validation run, the store deletes the stored status so that the initial no-update validation run only happens if the data set in the store is complete. This may be unnecessarily strict – any opinions?

Deleting the stored status will only have a noticeable effect if routinator is restarted right after a non-successful/incomplete run, correct? Because after such a non-successful run, it will fall back and serve cached data from the store until it is time for the next run anyway. And that next run will then create and write a new StoredStatus.
If that's all correct, I think there is little to gain with making it less strict. But maybe I'm missing a certain scenario here.

[partim]

I’ve realized (while looking at #770) that a stored publication point can have multiple states that we need to distinguish between if we want to create a correct data set from the store only:

• it has been successfully fetched and is valid,
• it has been successfully fetched but found to be invalid,
• an attempt to fetch it has been made but it failed,
• no attempt to fetch has ever been made,
• an attempt to fetch has been made but the stored data is corrupted.

The latter two cases are interesting because the means that the attempt to create a data set from the store only needs to be aborted and a regular validation run is necessary. Because a validation run can be aborted, we should probably also store the time a publication point was stored and treat an outdated point as ‘not stored’ as well.

[partim]

I’m starting to think this is getting way to complex and making sure that we don’t accidentally produce an incomplete data set from just the store is too tricky as there are just too many variables.

It occurred to me that a better approach is to use the collectors instead: Tell them to not update if they have completed a successful update of a requested repository within the given time span. They can then transition quietly to doing an update after all if a repository is missing or too old – or, perhaps better, abort and tell the engine to start again with updates enabled (so we don’t mix old and new data).