AI Instruction Set Integration and VHDL Code Verification

[Tahamermer]

Hello,

I've recently started coding with Neorv32 and have been experimenting with an AI instruction set. I have coded these experiments in VHDL. Now, I'm trying to integrate this new instruction set into Neorv32. However, I'd like to leverage the community's knowledge and experience for a proper integration and verification process.

I have some specific questions regarding this:

What are the fundamental considerations when adding a new instruction set to the Neorv32 architecture?
Are there recommended tools or methods for verifying and testing VHDL codes?
Should I focus on specific metrics (such as performance or energy efficiency) during the verification process?
Do you have any suggestions on detecting potential errors in the early stages?
Thank you in advance for any feedback, suggestions, and resources. I hope to progress my project more robustly and efficiently with the community's assistance.

Thanks.

[stnolting]

Hey there!

That sounds really interesting! Could you provide some more information about the actual use case / what kind of AI instruction you are implementing? 😉

What are the fundamental considerations when adding a new instruction set to the Neorv32 architecture?

Well, the most important thing is to think about the actual implementation. Do your AI accelerator(s) implement single operations or does it process an entire application (like a complete CNN)?

For "real" instruction the Custom Functions Unit (CFU) is the right place to got. But if you have an entire accelerator then you might want to implement it as dedicated co-processor via the Custom Functions Subsystem (CFS).

If you go for the "instructions" approach then you should think about stuff like instruction encoding (e.g. how many instructions do you have), operands (2, 3 or 4 source registers, immediates, ...) and internal states (e.g. do you need to provide additional data to the instructions like CNN coefficients).

Are there recommended tools or methods for verifying and testing VHDL codes?

Are you talking about general HDL verification or verification/testing in terms of the NEORV32?

Should I focus on specific metrics (such as performance or energy efficiency) during the verification process?

That is up to you 😉

You can optimize for area, performance, energy efficiency, high clock speed, ... This depends on your actual application / use-case.

Do you have any suggestions on detecting potential errors in the early stages?

Hm, hard question... 😅 The rtl simulation will show general design flaws. The synthesizer will show if your logic can actually be synthesized. But after that... it is up to you.

Just a small hint: when you integrate your custom instructions into the CPU you can run custom programs to check for the correct functionality. If your RTL design still has some bugs the CPU won't crash (just issuing exceptions) so that might help to setup a higher-level test platform.

[Tahamermer]

Hello,

That sounds really interesting! Could you provide some more information about the actual use case / what kind of AI instruction you are implementing? 😉

Indeed, it's an interesting topic! I'm considering using a CNN (Convolutional Neural Network) for this. I plan to break the task down into manageable pieces and progress slowly to achieve the best result. 😉

Well, the most important thing is to think about the actual implementation. Do your AI accelerator(s) implement single operations or does it process an entire application (like a complete CNN)?

For an initial approach, I want to test functions like convolution and relu using the CFU and then review the results. Depending on the progress, I'm actually considering continuing with the CFS.

Just a small hint: when you integrate your custom instructions into the CPU you can run custom programs to check for the correct functionality. If your RTL design still has some bugs the CPU won't crash (just issuing exceptions) so that might help to setup a higher-level test platform.

Actually, I was thinking of doing something similar. I had in mind to run the added instructions as CFU in the C code to test whether they work properly or not. Would this be suitable? If so, could you provide an example ? Thank you.

Best Regards,

[stnolting]

Indeed, it's an interesting topic! I'm considering using a CNN (Convolutional Neural Network) for this. I plan to break the task down into manageable pieces and progress slowly to achieve the best result. 😉

So what kind of "basic" operations do you plan to implement? Maybe some matrix multiplication stuff and a max pooling?

For an initial approach, I want to test functions like convolution and relu using the CFU and then review the results. Depending on the progress, I'm actually considering continuing with the CFS.

If you are targeting maximum performance then the CFS-approach seems to be the best fitting one. But if you want to keep the hardware small while also being quite flexible (i.e. if the actual algorithm might change) then the CFU with it's custom instruction might be the better choice 😉

Actually, I was thinking of doing something similar. I had in mind to run the added instructions as CFU in the C code to test whether they work properly or not. Would this be suitable? If so, could you provide an example ?

Sure, just have a look at the default CFU setup (example):

• 📚 documentation: Custom Functions Unit (CFU)
• ⚙️ hardware module (highly commented): rtl/core/neorv32_cpu_cp_cfu.vhd
• 💾 software example: sw/example/demo_cfu/main.c

[Tahamermer]

So what kind of "basic" operations do you plan to implement? Maybe some matrix multiplication stuff and a max pooling?

In my CNN implementation, I will be using 5 convolutional layers, 3 fully connected layers, 3 max pooling layers, and ReLU activation functions.We can say initial for this project.

[stnolting]

So the most compute-intensive operation might be matrix multiplications, right?

These could be easily added as custom instructions by providing a MAC instruction (multiply-add) with an internal state storage (i.e. keep the accumulator inside the instruction and do return the result for every single iteration - reducing register file pressure). Obviously, one would need to execute one instruction for every matrix element:

for (...) {
  mac_iterate(x[i], y[i]); // multiply x[i] with y[i]
}
mac_res = mac_result(); // get the result from the instruction's accumulator