GPU memory usage for VGG16 prediction

[rachelmint]

Hi,

I found that image prediction with VGG16 took 770M GPU memory (for 1 image). I felt like mxnet use more memory than caffe in this case. Is that a normal memory cost?

[sandeep-krishnamurthy]

VGG is a dense network with many parameters. GPU memory is for complete model. I think this is normal.
What is the memory usage you are seeing for Caffe?

[rachelmint]

Caffe needs 500-550M GPU memory, but mxnet needs 770M

[eric-haibin-lin]

are you using gluon or symbols? Did you try mxnet master or pip install --pre? We just fixed some memory issues a few weeks ago #10847

[rachelmint]

Hi eric,
Thanks for your reply.
I am using symbols now and I use mxnet master but I installed it a while ago. I will try the newest version and will give an update.

[ThomasDelteil]

@rachelmint do you have an update?
@eric-haibin-lin I see much higher memory allocation than reported here using the VGG16 model. (Though the parameter file is indeed ~500MB) How do we explain this memory usage? Do we pre-allocate memory for feature maps?

Using gluon I find that MXNet allocates 1.92GB in memory when loading the model.
When running one image through the network, memory peak at 2.10GB and then go down to 2.02GB

Using the Module API, it allocates 1.4GB

import mxnet as mx
from mxnet import gluon

# load model from model zoo
net = gluon.model_zoo.vision.vgg16(pretrained=True, ctx=mx.gpu()) #1.9GB
net(mx.nd.ones((1,3,224,224), mx.gpu())) #2.1GB

# export the parameters
net.hybridize()
net(mx.nd.ones((1,3,224,224), mx.gpu()))
net.export('vgg16')

# Load in symbol
sym, arg_params, aux_params = mx.model.load_checkpoint('vgg16', 0)
mod = mx.mod.Module(symbol=sym, context=mx.gpu(0), label_names=None)
mod.bind(for_training=False, data_shapes=[('data', (1,3,224,224))])
mod.set_params(arg_params, aux_params) # 1.4 GB

[rachelmint]

Hi ThomasDelteil,

I tried in the newer version of mxnet, it takes ~850M to make a prediction on one image. I uses the code in /example/image-classification/predict-cpp/image-classification-predict.cc, not sure why it needs so much memory.

[eric-haibin-lin]

Specify the workspace option in the Convolution op will reduce the temp space used for it. Are you using the default value for workspace? You can look it up in the json file

[rachelmint]

I tried to set the workspace to 512 and 256 (the original value is 1028), the memory is not reduced, it stays exact same size.

[piyushghai]

@mxnet-label-bot [Memory]

[piyushghai]

@sandeep-krishnamurthy Can you remove the label "Pending Requester Info"