Towards Surgical Context Inference and Translation to Gestures

This repo for the paper titled 'Towards Surgical Context Inference and Translation to Gestures'.

Suturing

Needle Passing

Knot Tying

Getting Started

run the following commands with python 3.9 or above

python -m venv context-env
.\context-env\Scripts\activate
pip install -r requirements.txt
python .\run_pipeline.py Knot_Tying ALL 2023_DL - to run the script generates context labels based on the deeplab instrument masks without kinematics

Naming Conventions

TASKS can be Needle_Passing, Knot_Tying, Suturing

Masks belong to MASK SETS such as 2023_ICRA, COGITO_GT, 2023_DL, ...

Each task subject trial combination represents a unique TRIAL <Task>_S<Subject number>_T<Trial number>

Folder Structure

context_inference
- context_inference.py -- Contains context inference logic
- contour_extraction.py
- contour_template.json
- metrics.py -- IOU between predicted context and consensus context
- utils.py
data
- context_labels
  - consensus
  - <Labeler>
- contours
- masks
  - 2023_DL
    - ring
    - thread
    - leftgrasper
    - needle
    - rightgrasper
      - <Task>_<Subject>_<Trial>.png
        
        frame_0001.png
- images
  - <Task>_<Subject>_<Trial>.png
    - frame_0001.png
eval
- contour_images
- labeled_images
  - <Task>_<Subject>_<Trial>.png
    - frame_0001.png
- pred_context_labels
  - 2023_DL
    - <Task>_<Subject>_<Trial>.txt
seg
- Image segmentation scripts
run_pipeline.py -- runs context prediction pipeline