[CUDA] Questions about memory release

[FFFrog]

The temp_mask_buffer variable will execute its destructor because it goes out of scope, and then call cudaFree to release the memory it refers to; since the cuda kernel is executed asynchronously relative to the cpu, the cuda kernel DropoutKernel may not be executed when the memory is released.

My understanding may not be correct, could someone take the time to explain it? Thanks a lot.

onnxruntime/onnxruntime/core/providers/cuda/nn/dropout.cc

Line 110 in a8701c2

IAllocatorUniquePtr<void> temp_mask_buffer{}; // buffer to use if mask is not provided

onnxruntime/onnxruntime/core/providers/cuda/nn/dropout_impl.cu

Line 32 in a8701c2

__global__ void DropoutKernel(const CUDA_LONG N, const CUDA_LONG mask_element_count, const int step_size,

[pranavsharma]

The temp buffer is allocated using an arena whose memory is not freed. Hence async execution is not affected.