What is a good layout for problem with dimensions N = image size (512 * 512 or 1024 * 1024, for example), M = 64, K = 4?

[dsilvavinicius]

Hi, I have a GEMM problem with dimensions N = image size (512 * 512 or 1024 * 1024, for example), M = 64, K = 4 and I'm trying to setup Cutlass to work on it (aiming for Turing architecture). I cannot find a layout that works with that small K = 4 in mixed precision (results in mem. misaligment). To provide more context, M is the size of a multilayer perceptron hidden layer and K is the size of the input layer. My goal is to make inference on each pixel of the image.

[hwu36]

what is your data type? You can use cutlass profiler to try all the possible ones and pick the best one.

https://github.com/NVIDIA/cutlass/blob/master/media/docs/profiler.md

[dsilvavinicius]

Data type is cutlass::half_t for A and B and float for output. I will try to use the profiler too!

[hwu36]

Use cmake .. -DCUTLASS_NVCC_ARCHS=75 -DCUTLASS_LIBRARY_KERNELS=cutlass_tensorop_s1688gemm_f16 and then
make cutlass_profiler to build the profiler. You can then pick the best one from all the kernels by using the profiler.

[dsilvavinicius]

Allright, thanks for the advice!