[E2E] Torchbench amp_bf16 training Super_SloMo accuracy failed

[mengfei25]

🐛 Describe the bug

Looks like there is a random issue for Super_SloMo, and it will be passed with WHL install from prebuild but failed with source build.
In latest weekly,
WHL Passed: https://github.com/intel/torch-xpu-ops/actions/runs/10742335908
Source build Failed: https://github.com/intel/torch-xpu-ops/actions/runs/10741560513

And I tested WHL locally multiple times and it is passed randomly.

Versions

torch-xpu-ops: 1206590

[weishi-deng]

This issue passed in the latest weekly test and local reproducer.

[chuanqi129]

Hi @weishi-deng This is a random failure, we may need to figure out the root cause of it