embedding group

[xiexbing]

Can you explain how the embedding group can contribute to the better performance of embedding layer?

[tiankongdeguiji]

During training, TorchEasyRec's EmbeddingGroup leverages TorchRec's EmbeddingBagCollection and EmbeddingCollection to fusion a large number of feature embedding lookup operations into as few operations as possible.

During inference, TorchEasyRec's EmbeddingGroup further optimizes the embedding computation on the user side, reducing the required calculations from batch_size times to just once.

[xiexbing]

for the training, how the fusion helps? more specifically, at low level, where the benefit comes from? e.g., does the fusion helps on the better utilization on the communication in the forward pass by better batching the communication data? does the fusion helps on reduce the number of cuda kernel launching on pooling in forward? please help provide details or point to the detailed documentations/papers.

[tiankongdeguiji]

the fusion helps on reduce the number of embedding lookup cuda kernel launching and better utilization on the communication.