Questions about the Initializer.

[RyougiKukoc]

We stuck with deploying some complicated operations on FINN. The main problem is that it will throw out an error when streamlining:

Initializer for mul (or matmul) weights is not set.

This happens when we use nn.GELU, which will generate a mul node for $x * \Phi(x)$, and the input[1] of this node does not have an Initializer; or when we use torch.matmul or torch.bmm to implement an attention layer manually, which will generate matmul nodes when multiplying the query and the key, or with the value, similarly, their inputs don't have an initializer.

We do understand the importance of the static shape, but we have already fixed the batch dim when exporting to qonnx. Here are two questions:

1. Can we set up a random array for the initializer of either mul or matmul node?
2. Beyond that, is there any convinent or traditional way to manually set up an initializer for each of these nodes?

Thank you.

[fpjentzsch]

Hi,

why would you want to set random initializers for these operators? Attention and GELU are not (yet) supported by FINN.