Welcome to cutlass Discussions!

[hwu36]

👋 Welcome!

We’re using Discussions as a place to connect with other members of our community. We hope that you:

• Ask questions you’re wondering about.
• Share ideas.
• Engage with other community members.
• Welcome others and are open-minded. Remember that this is a community we
  build together 💪.

To get started, comment below with an introduction of yourself and tell us about what you do with this community.

[Peter9606]

Glad to be the first! I'm with a team working on GEMM acceleration used by our DNN framework. Still a lot of details are not clear for me, so really happy if there's a webniar through which the maintainer team can share something more.

[kerrmudgeon]

We have recorded several talks held over the previous three GPU Technology Conference events in San Jose.

• Developing CUDA kernels to push Tensor Cores to the Absolute Limit on NVIDIA A100
• see 'part 2' of this talk which focuses on CUTLASS and Volta Tensor Cores
• CUTLASS: CUDA TEMPLATE LIBRARY FOR DENSE LINEAR ALGEBRA AT ALL LEVELS AND SCALES (this is an older talk on CUTLASS 1.0)

Beyond this, could you suggest some of the topics you'd like a future webinar to focus on?

[Peter9606]

For me, I'd like to know why optimization tips shown in the code is chosen, for example, memory access is hard coded to 128 bits, why it's the optimal. Another example is there's a LaneMmaShape for SIMT, seems it's used to make the mma shape more squared. A topic on details of those tips would be welcomed.

[hwu36]

memory access is hard coded to 128 bits, why it's the optimal.

The max number of bits that one thread can load/store from/to global/shared memory is 128bits. We want to fully utilize the memory bandwidth, so we use the "biggest" load/store instructions.

Another example is there's a LaneMmaShape for SIMT, seems it's used to make the mma shape more squared.

Suppose we can load L elements totally from operand A and operand B. If we load a elements from operand A, then we can load (L - a) elements form operand B. The element number of the output product is a (L - a). We want to get as many results as possible from one load. max{a(L-a)} = (L^2) / 4 when a = b = L/2. Another way to think about it is that we want to reuse the loaded input operand data as much as possible., i.e. generate as many results as possible from as few loads as possible.