multithread run inference on the same model using 2 sessions（c++）

[shiqingzhangCSU]

Is it thread-safe for multiple threads to call a session?
If I use multiple sessions of the same model and then call them with multiple threads, can I share the weights of this model?
Hopefully someone can give an example of a multithreaded call

[wschin]

Calling inference session from different is safe. Yes, weights are shared. See testRunModelMultipleThreads test in onnxruntime/onnxruntime/test/python/onnxruntime_test_python.py. For C++ example, see RunWithOneSessionMultiThreadsInference test in onnxruntime/onnxruntime/test/providers/tensorrt/tensorrt_basic_test.cc

[shiqingzhangCSU]

Hello! Thanks for replying! I focus on C++ inference，and I would also like to ask two questions：

1. In c++, session.Run(）is thread safe ? For multiple threads calling this, no need to lock?
2. I use multiple sessions of the same model , but the memory is basically multiplied. Is there any configuration or method for parameter sharing?