How can I use the free memory

[cadop]

I saw this post #171

but also noticed this property: https://nvidia.github.io/warp/modules/devices.html#warp.context.Device.free_memory

Maybe its obvious, but I cant figure out how to use it. wp.free_memory doesn't exist.

[shi-eric]

It's a Device property, so you should be able to do something like:

cuda_device = wp.get_device("cuda:0")
cuda_device_free_mem_mib = cuda_device.free_memory / (1024*1024)

[cadop]

thanks!

[cadop]

which OS property is it referring to?

nvidia-smi and task manager show 12gb available, warp says 22.122078125

[shi-eric]

That's strange... I'm on Linux so this is my experiment:

>>> import warp as wp
>>> wp.init()
>>> cuda_device = wp.get_device("cuda:0")
>>> test_array = wp.zeros(1000000000)
>>> cuda_device.free_memory / (1024*1024)
20108.1875

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.14              Driver Version: 550.54.14      CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3090        Off |   00000000:1B:00.0 Off |                  N/A |
| 30%   38C    P5             88W /  350W |    4144MiB /  24576MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA GeForce RTX 3090        Off |   00000000:68:00.0  On |                  N/A |
|  0%   44C    P8             39W /  350W |     636MiB /  24576MiB |     25%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

What happens if you run a similar comparison on your machine?

[shi-eric]

This is the driver API function being called under the hood: https://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__MEM.html#group__CUDA__MEM_1g808f555540d0143a331cc42aa98835c0