Automating file transfer off the FlowCam

[metazool]

Short summary of discussion with Science Computing folks, also relevant to #16

1. There's an approved route for attaching lab computers to a restricted part of the internal network which has access to a "transfer proxy" (e.g. a machine that will collect data from "quarantined" machines like the FlowCam and forward it on); it does anti-virus scanning and has options to route data direct to cloud storage
2. On the instrument end this could be as basic as a PowerShell script running in Task Scheduler, which checks a directory for updates and then sends a JSON document to a webhook on the transfer proxy
3. We don't know whether permission levels on the FlowCam are elevated enough to make this work, but that would be a helpful way of finding out what else we can't do on it

I've contacted Kelly in the lab to suggest a least-effort test of stage 2...

[metazool]

@albags if this approach works in practise then it could leave us with different choices about data transfer

We've discussed the different cases of reuse value for your work https://github.com/AMI-system/ami-api ("API to push data (images and audio files) into an S3 server.") - potentially seen as a backend for a GUI experience of small to medium sized image datasets

In the internal Teams discussion the suggestion is that the transfer proxy could send the source images (in this case the collages before they are segmented) directly to cloud storage (JASMIN/s3)

It still leaves the decollaging needing to run somewhere but at least detaches that from a named individual's on-premise virtual machine...

[metazool]

From Microscope to Metadata talk from last year's RSECon - caveat, haven't watched and don't know if it has an image ML angle

[metazool]

I've moved this into the TODO part of the backlog in the "Project" view after getting some feedback from Kelly in the lab - they're able to run Powershell commands in the CLI with administrator privilege - which seems very promising for setting up a scheduled task to do file transfer.

It will need followup with @DamienUKCEH's team about possibility of using a transfer proxy with webhook from the FlowCam built-in PC. It's not an action til November, I've added it TODO so we remember to look at it then.