WP4: Support - Json Object State

[duncanpeacock]

Description

Investigate loader (xchem/fragalysis-loader) issues - db keys changing when data updated. Identify / Fix issues where necessary / possible.

Deliverables

Solution / Issues Raised

Parent: (xchem#208)

[duncanpeacock]

Arranged a meeting with Tyler for initial walkthrough / steps to recreate issue.

[duncanpeacock]

Some notes/initial ideas following the meeting with Tyler:

• Data loads are done with production-media-loader from AWX with namespace production-stack.
• Tyler thinks that it does delete and re-insert all data for the target set (and also suspects that the snapshots are probably broken because of auto-incrementing keys – I need to look at the code to see exactly what is going on.
• Names can change – and naming conventions are not fixed - so using the name as a primary key is not sensible.
• Maybe could use a SHA key as a hash on the contents of the file. If that hasn't changed then you could update the database rather than doing a delete/re-insert
• Alternatively instead of deleting records, you could make them inactive (via setting a status and/or timestamp, say). That way, they could still be available for historical snapshots. Maybe this might also help the Error 404 loader issue that Boris was seeing. - Next step is look more closely at the code.

[duncanpeacock]

This has been lowered in priority.

• The new loader API allows individual data sets to be loaded see: (WP4 : API to upload target dataset  #19).
• Action capture is being developed to replace the state for recording snapshots.