GRU Perfomance tuning

[grazder]

Hello! I'm currently trying to speedup my model and using ORT for inference. I've made a profiling and it showed up that GRU blocks takes a lot of time compared to other blocks. For the longest GRU, the input shapes are:

input_type_shape: 
[{float: [1,
          1,
          256]},
 {float: [1,
          1536]},
 {float: [1,
          1,
          256]}]

onnxruntime_profile__2024-02-08_13-00-46.json

I'm trying to figure out is it possible to speedup model without lowering size of it? Are there some different algorithms types? Can GRUCell be faster?

[skottmckay]

GRU does a lot of calculations, so 'takes a long time' may be completely expected.

The ONNX spec doesn't have different algorithms to choose from.

https://github.com/onnx/onnx/blob/main/docs/Operators.md#GRU