C STORE Backup and Retries

C-STORE Backup and Retries

In C-STORE to STOW-RS mode, the Import Adapter can use additional flags to improve the reliability of file uploading. Before uploading a file to the dicomweb endpoint, the file is saved to temporary storage from which the Import Adapter tries to upload the file the specified number of times. The file can be saved locally or loaded into a GCS bucket. After a successful upload, the temporary file will be deleted. Also, the user can independently configure a TTL to automatically delete files from GCS.

The following arguments are used to configure the mode:

• --persistent_file_storage_location: temporary location for storing files before send.
• --persistent_file_upload_retry_amount: upload retry amount. (default 0).
• --min_upload_delay: minimum delay before upload backup file (ms) (default 100ms).
• --max_waiting_time_between_uploads: maximum waiting time between uploads (ms) (default 5000ms).
• --http_error_codes_to_retry: list of HTTP error codes to retry. By default error codes >= 500 will be retried.

If the flag --persistent_file_storage_location is not used then loading occurs without writing a temporary file.

If the value for the flag --persistent_file_storage_location is specified in the format gs://bucket-id/some-directory, the file will be written to the specified project bucket. When using GCS to backup files is also necessary to set the following flags:

• --gcs_backup_project_id=GCS_PROJECT_ID, where GCS_PROJECT_ID is the GCP project id in which the bucket was created
• --oauth_scopes=https://www.googleapis.com/auth/cloud-platform

If you are using Kubernetes, then in the file dicom_adapter.yaml change the arguments in the file as follows:

          args:
            - "--persistent_file_storage_location=/tmp/backupfile"
            - "--persistent_file_upload_retry_amount=5"
            - "--min_upload_delay=100"
            - "--max_waiting_time_between_uploads=5000"

If you are using import adapter locally:

gradle run -Dexec.args="--dimse_aet=IMPORTADAPTER --dimse_port=4008 --dicomweb_address=http://localhost:80 --persistent_file_storage_location=/tmp/backupfile --persistent_file_upload_retry_amount=5 --min_upload_delay=100 --max_waiting_time_between_uploads=5000"

STOW-RS Retries due to HTTP 409

If you try to insert a DICOM instance with a Study/Series/Instance UID (triplet) equal to one previously inserted, then you will receive an HTTP 409 (Conflict) error code. This use case can occur often when there are slight updates to patient information or other metadata in a study, before it is read by a radiologist.

This is the primary reason we added the --stow_overwrite flag. When provided, and a number of retries > 1 is specified, the import adapter will effectively perform an HTTP DELETE on the DICOM instance that has returned with an HTTP 409 error code and then reattempt the STOW-RS operation, up to the number of retries configured. This flag exists to allow the previously discuss workflow to occur using DICOMweb.

To use the --stow_overwrite flag effectively, it must be used with the retry and persistence flags discussed earlier. The following flags must minimally be defined together:

• --stow_overwrite: delete instance on receiving an HTTP 409 and retry up to --persistent_file_upload_retry_amount (default false).
• --persistent_file_storage_location: temporary location for storing files before send.
• --persistent_file_upload_retry_amount: upload retry amount (default 0).

The primary reason the overwrite flag needs to be used in conjunction with the persistence flags is because we do not want to keep DICOM instances buffered in memory while retries occur.

If you are using Kubernetes, then in the file dicom_adapter.yaml change the arguments in the file as follows:

          args:
            - "--persistent_file_storage_location=/tmp/backupfile"
            - "--persistent_file_upload_retry_amount=5"
            - "--min_upload_delay=100"
            - "--max_waiting_time_between_uploads=5000"
            - "--stow_overwrite"

If you are using import adapter locally:

gradle run -Dexec.args="--dimse_aet=IMPORTADAPTER --dimse_port=4008 --dicomweb_address=http://localhost:80 --persistent_file_storage_location=/tmp/backupfile --persistent_file_upload_retry_amount=5 --min_upload_delay=100 --max_waiting_time_between_uploads=5000 --stow_overwrite"