Parallelism of ONNX session objects with multiple threads #15300
Unanswered
carbotaniuman
asked this question in
Other Q&A
Replies: 1 comment 6 replies
-
You should create one session and use it in multiple threads. ORT is designed for this. Creating multiple sessions for each thread is a huge waste of resources and we strongly recommend against it. |
Beta Was this translation helpful? Give feedback.
6 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
If I want to run multiple ONNX sessions of the same model with various objects, I can either create one session object and have multiple threads call it simultaneously, or create one session object per thread and have each thread with it's own. Asides from providers like DirectML that do not have parallelism, are there are benefits to one or the other?
Beta Was this translation helpful? Give feedback.
All reactions