Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Docker-compose #27

Draft
wants to merge 5 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .env
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# configurable options for `docker-compose`
# Description (service-name)

# Directory to store DICOM files (dcm-receiver)
DICOMDIR=./dicoms

# Port to listen for DICOM transfer (dcm-receiver)
PORT=8080

# Directory for conversion output (heudiconv)
HEUDICONV_OUTDIR=./output


# ADVANCED: additional heudiconv arguments to use (heudiconv)
# args used: --files, -f, -o, -b
HEUDICONV_XARGS=""
17 changes: 17 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,23 @@ tools, any environment providing them would suffice, such as
other Debian-based systems with NeuroDebian repositories configured,
which would provide all necessary for ReproIn setup components.


## `docker-compose` + usage

This repository provides a `docker-compose` file that includes (2) services:
- DICOM receiver ([dcmtk](https://dicom.offis.de/dcmtk) `storescp`)
listening and ready to receive through a port.
- `reproin` (`heudiconv`) converter set to process all folders in the DICOM
directory through a cron job (default is set to everyday at 4am).

Refer to the `.env` file - many options above are configurable within it.

To spin up the reproin service, run `docker-compose up` in the project root -
this will route all logs to the console. If you wish to run it in the service
in the background, append the `--detach` flag, and use `docker-compose logs`
to monitor. Since this is a new feature, it may not behave as intended. If you
encounter any errors, please open an issue and let us know.

## TODOs/WiP/Related

- [ ] add a pre-configured DICOM receiver for fully turnkey deployments
Expand Down
10 changes: 10 additions & 0 deletions dcmrecv/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
FROM debian:stretch

ARG DEBIAN_FRONTEND="noninteractive"

RUN apt-get update -qq \
&& apt-get install -y -q --no-install-recommends ca-certificates apt-utils dcmtk \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

WORKDIR /dicoms
4 changes: 4 additions & 0 deletions dcmrecv/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# docker-compose up to start application

# dcm-receiver app will launch docker container running storescp
# listening for DICOMs on port 8080 and saving to an output directory
17 changes: 17 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
version: '3'
services:
dcm-receiver:
build: ./dcmrecv
ports:
- "${PORT}:${PORT}"
volumes:
- ${DICOMDIR}:/dicoms
command: storescp -v -sp --output-directory /dicoms ${PORT}

heudiconv:
build: ./heudicron
volumes:
- ${DICOMDIR}:/dicoms:ro
- ${HEUDICONV_OUTDIR}:/output
depends_on:
- dcm-receiver
18 changes: 18 additions & 0 deletions heudicron/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
FROM nipy/heudiconv:latest

RUN apt-get update -qq \
&& apt-get install -y -qq --no-install-recommends cron \
&& apt-get clean

RUN git config --global user.email "user@heudiconv" \
&& git config --global user.name "heudiconv-erter"

COPY heudiconv.sh /

# save copy of current PATH to be used in cron job (hacky)
RUN echo ${PATH} > /opt/heudiconv.path

RUN chmod +x /heudiconv.sh \
&& (crontab -l; echo "* 4 * * * /heudiconv.sh > /proc/1/fd/1 2>/proc/1/fd/2") | crontab

ENTRYPOINT cron -f
5 changes: 5 additions & 0 deletions heudicron/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# heudicron

Automate received DICOMs through heudiconv's reproin conversion. Users are able
to alter automation schedule via the `heudicron` script. By default, `heudiconv`
conversions will trigger every day at 3am.
38 changes: 38 additions & 0 deletions heudicron/heudiconv.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
#!/bin/bash

echo "Running scheduled heudiconv conversion"

HEUDICONV_DICOMDIR="/dicoms"
HEUDICONV_OUTDIR="/output"
HEUDICONV_XARGS=${HEUDICONV_XARGS:-""}
HEUDICONV_HISTORY="${HEUDICONV_OUTDIR}/.heudiconv.history"

set -eu

# cron uses strip down PATH
# pull container creation PATH variable
if [ -f /opt/heudiconv.path ]; then
echo "Setting heudiconv PATHs"
export PATH=$(cat /opt/heudiconv.path)
fi

if [ ! -d ${HEUDICONV_DICOMDIR} ]; then
echo "DICOM directory ${HEUDICONV_DICOMDIR} not properly set or mounted"
exit 1
elif [ ! -d ${HEUDICONV_OUTDIR} ]; then
echo "Output directory ${HEUDICONV_OUTDIR} not properly set or mounted"
exit 1
elif [ ! -f ${HEUDICONV_HISTORY} ]; then
touch ${HEUDICONV_HISTORY}
fi

for DICOMS in $(ls ${HEUDICONV_DICOMDIR}); do
if cat ${HEUDICONV_HISTORY} | awk '{print $1}' | grep "${DICOMS}" > /dev/null; then
# existing conversion was found
continue
fi

CMD="heudiconv -f reproin --bids --datalad -o ${HEUDICONV_OUTDIR} --files ${HEUDICONV_DICOMDIR}/${DICOMS} ${HEUDICONV_XARGS}"
${CMD}
printf "${DICOMS}\t$(date)\n" >> ${HEUDICONV_HISTORY}
done
4 changes: 4 additions & 0 deletions heudicron/heudicron
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# key: min (0-59) | hr (0-23) | dom (1-31) | mon (1-12) | dow (0,7=sun) | command

# trigger heudiconv every day at 3am
* 3 * * * /heudiconv.sh >> /var/log/cron.log
11 changes: 11 additions & 0 deletions heudicron/start-cron
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/bin/bash

crontab /etc/heudicron

service cron start

CRONLOG="/var/log/cron.log"

while true; do
tail -f /var/log/cron.log
done