AssertionError: IPGen failed

[happyxiaopeng0820]

How to deal with it?

[auphelia]

Hi @happyxiaopeng0820, have you checked the log file that is generated in the tmp folder? The path is specified in the error message.

[fpjentzsch]

Hi, a crash like this could happen if the tool runs out of memory. Could you keep an eye on the RAM usage while Vitis/Vivado is running?

[SAYANTANDUTTA1993]

I am facing the same issue. I have the correct Vivado and Vitis versions (2022.2). I am using AlexNet, and my system has 64 GB of RAM. On 12th step,, my memory is getting filled. Is this normal or any other issues / memory?

[fpjentzsch]

If Vitis HLS is indeed running out of RAM, you could try reducing the NUM_DEFAULT_WORKERS environment variable to 1, so that FINN runs fewer Vitis HLS instances in parallel.

[SAYANTANDUTTA1993]

I tried the NUM_DEFAULT_WORKERS to 1 (shown in third picture). Except that I made some changes (shown fourth picture). But got the same error (picture 2 shows ip_gen error and picture 1 shows memory full). What is the problem then




!

[fpjentzsch]

Maybe your model is simply too large for HLS to handle. You could try a smaller variant of your model, reduce the SIMD/PE parallelism, set mem_mode of the MVAU layers to "internal_decoupled" (which is easier on the synthesis) or switch to the new RTL back-end in the current dev branch.

[SAYANTANDUTTA1993]

I tried above mentioned SIMD/PE(SIMD=4/16 at max, and PE=1) , and MVAU layers(as internal_decoupled) (You can find it in below attached auto_folding_config.json file). Also changed NUM_DEFAULT_WORKERS=1 (as import os /n os.environ['NUM_DEFAULT_WORKERS'] = '1', inside jupyter notebook) but getting error ip_gen failed and memory full (my system ram is 64gb). I am using Alex-net model which has 4,097,000 parameters and require approx 250MB size. So what is the system specification you are using or memory requirement to run such type of network.
I apologies if it's something on our end, but we're still running into the same ram issue. Please let us know if we're using to right configuration, as we want to rule out other possibilities before trying to upgrade our system as a last resort.
Thank you so much for your time
auto_folding_config.json

[dvisa7178]

Hello! Did you solve the problem? I'm having the same issue when running a test that was supposed to work, and it's also failling at the 12th step.