Integration new microservice Boutiques

Integration of a new microservice into Shanoir-NG, example MS Boutiques

The MS Boutiques in Shanoir-NG connects Shanoir-NG to Boutiques (http://boutiques.github.io/). This gives Shanoir access to the processing pipelines available in Boutiques. An user can select a pipeline of interest and configure the pipeline with the required input data, that are in Shanoir: Datasets.

The MS Boutiques needs the DICOM or NIfTI files available in Shanoir to process them. We decided to put these files into a temporary folder (/tmp), as we do not want to risk that a pipeline could delete origin production data. Therefore the data will have to be copied into the /tmp folder for an authorized user. MS Datasets is assuring that only authorized users can access and download the data. We decided, that the user choses in the Angular front-end the list of files attached to one dataset, he wants to process and he configures the pipeline with the files selected or a list of files. The user continues and starts the processing pipeline (async answer, for not waiting). The front-end is either communicating with MS Boutiques or MS Datasets and no waiting time is created for the best user experience.

As we want to avoid inter-ms communication, that slows down the system, creates dependencies and fail-over problems, we decided for the most simple solution at the moment. The user starts the processing and the front-end calls MS Datasets to download the files into /tmp. MS Datasets assures the authorization of the user in session to do that. While the user has received a notification, that his processing has been started and finished his interaction, the front-end calls MS Datasets to do the download and provisioning of the files (for the moment sync call). When this is finished, MS Datasets returns a random session id (512 long number + name of the folder in /tmp) and the front-end calls MS Boutiques to start the processing on using the random session id + with the list of files. As only the microservices can access to the disk and only authorized users can call the microservices, with the random session id we assure, that no other user can easily access to other data when manipulating the json to MS Boutiques. Attention: when this is ongoing the user needs to be alerted in the front-end, that when he leaves the front-end will not continue his job OR we could use a worker in the browser here to assure the processing finish.

Another interesting option has been discussed: MS Datasets returns a "JWT/json" with a signature on using his private key, that MS Boutiques could on using the public key verify the signature of the "JWT/json" and download the data itself. All that to avoid inter-ms communication as much as possible.