WP4 : API to upload target dataset

[duncanpeacock]

Add an Upload page to the Fragalyis backend that allows a zip file of Target datasets to be uploaded. The code/process will be based on the existing upload code for Compound_csets in the fragalysis_backend using a Celery async process.

Look at removing visits+proposals from upload directory.

Parent Issue (xchem#208)

[duncanpeacock]

Initial discussion with Rachael.

1. Initial Target is to supply a method to update a Target dataset in a zip file.
2. There is an existing process for Compound Sets that can be used as a model for the new upload process. It has a separate page rendered ( viewer/upload_cset/ ). See: ( https://fragalysis.diamond.ac.uk/viewer/upload_cset/ )
3. Snapshot breaking problem notes:

• This has been worked on – it now no longer deletes and re-adds everything – changed by Rachael to update rather than overwrite. However it still seems to "break" snapshots. We discussed some theories that I can test as part of the process.
• Not as high priority as the API because of Boris' work on recreating snapshots from history.

Currently working on an initial analysis/design.

[duncanpeacock]

First version of design document for review:
Loader API Change WP4 v0.1.docx

[duncanpeacock]

Second version of change document:
Principal changes:

• Expanded authentication section - investigation of current authentication/authorisation processes in the fragalysis backend and recommendations for improvements.
• proposal/visit now included
• configuration of proposal/visit for different users

Loader API Change WP4 v0.2.docx

[duncanpeacock]

Agreed to split wider authentication/authorisation question into separate issue/sprint.
Coding started on first version.

[duncanpeacock]

Coding continuing

• Code migrated across and basic page running (but fragile) with downloaded structures. However this is not the end state as the code will not come from this source - should match closely the input data from loader.

Next stages (for this sprint):

1. Fix compound results page (had to break this for the above)
2. Tidy code
3. Get working with 'real' input data
4. Proposals+Visits (public data)
5. Test with compound sets
6. Add authentication

[duncanpeacock]

Initial version of API working on: fragalysis-duncan-default.xchem-dev.diamond.ac.uk/
Remaining to do for Minimum viable product:

1. Validation of results (Rachael taking a look at this) - and anything changes following from that.
2. Basic authentication (check on html page that user is logged in)
3. Small bug fixes (create tmp folder when missing, tidy up data when validation fails)

[duncanpeacock]

Authentication and bug fixes done.

[duncanpeacock]

Review, Pull request made and accepted. Separate issue created as place holder for follow up work.