Suggest：nnUNet can be designed to run as a server，so as to improve performance

[goodsave]

If nnUNet can be designed to run as a server, keep Pytorch waiting after loading the model, stay in memory, and use a queue to process the received segmentation tasks, this will greatly improve performance and avoid wasting time caused by repeated model loading.

[ancestor-mithril]

I don't think this is feasible, because nnUNet is designed to predict using an ensemble of models trained on different folds. As a consequence, each model is loaded (sequentially) to do inference on a single input file.
If the models are not loaded, all models will have to be kept in memory.

[goodsave]

Okay, thank you very much for your reply.