Why is the response time of ONNX Runtime increasing?

[RezaHajimohammadi]

Hi, I'm using from ONNX Runtime library to inference my deep neural network model in c++. My problem is that the processing time increases after a while. I have given more details in the link below:
https://stackoverflow.com/questions/75246419/why-is-the-response-time-of-onnx-runtime-increasing

[ytaous]

Can u pls loop the Run for 10 times and see?
Inferencing in GPU may need some time to prepare data the first time from CPU->GPU.
I am not sure about time increasing, I wonder if CPU is doing some GC that may affect the perf.

[RezaHajimohammadi]

Yes in the first process time is too high and its normal, after it time is low but after about 1 minute, time increased. About loop, i'm testing several test in any run, whit diffrent interval, result is the same.
What is GC?

[RyanUnderhill]

Are you doing this in a tight loop? There's also the possibility of thermal throttling of your CPU or GPU (easy to see in monitoring tools if your clock speed drops).

[RezaHajimohammadi]

No its not in a loop, i send command for test. I test it whit different interval, result is the same.
About your hypothesis, one of my test was this, i processed whit low interval in the start time, time didn't increase untile about one minute, in second scenario, i didn't process after start for about 1 minute, after about 1 minute i start to test, time was like first scenario at same time!