Main file -> Interrupt handler (which will jump to 0x20000) -> jump back to Main file

[SteveZ-Cal]

FPGA Setup

NEORV32 on Cyclone V De0-CV
IMEM: 128KB
DMEM: 64KB

Hardware

Version: V1.8.6 (Purpose: compatible to Zephyr latest version)

Task I'm Trying to Achieve

A simple Main file "hello_world.bin" uploaded to 0x00000 [Around 64KB] (Integrated with Zephyr OS)
A task file "task.bin" that will be triggered by interrupt handler uploaded to 0x20000 [Around 4KB] (Just NEORV32)

I'm interested in the cfs interrupt, and did some configuration for its corresponding vhdl to capture one pulse.

In addition, I stored the previous PC before the interrupt handler in NEORV32_CFS->REG[61], and the current PC in NEORV32_CFS->REG[63]

( If I press key1 on FPGA, it will trigger a cfs one pulse interrupt)

ZEPHYR

Within that Interrupt handler (Zephyr OS API for IQR_Connect, irq_enable)

/*******Define how ISR handle Interrupt ********/

// printf allowed in Zephyr

void my_isr(void *arg){
	printf("INSIDE my_isr\n");
 	__asm__ volatile ("jalr ra, %0" : : "r" (0x00020000)); 
	printf("Exit my_isr\n");
}

void my_isr_installer(void){
	printf("Inside my_isr_installer\n");
	IRQ_CONNECT(MY_DEV_IRQ, MY_DEV_PRIO, my_isr, MY_ISR_ARG, MY_IRQ_FLAGS);
	printf("ISR-Enabled\n");
   	irq_enable(MY_DEV_IRQ);
	printf("IRQ-Enabled\n");
}

hello_world.bin

my_isr_installer();

(some other tasks)
neorv32_uart0_printf("Before Interrupt\n")

...
I pressed key 1 on FPGA
(calling corresponding interrupt handler, trigger my_isr function which will jump to 0x200000) -> The API is correct, and nothing wrong with that part
...

neorv32_uart0_printf("After Interrupt\n")

NEORV32

task.bin

  /***JUMP Back to the Main Program***/
  neorv32_uart0_printf("Inside Interrupt handler task function \n");
  uint32_t firq_PC = NEORV32_CFS->REG[61];
  uint32_t PC = NEORV32_CFS->REG[63];
  firq_PC = firq_PC%4 + firq_PC;
  neorv32_uart0_printf("Current PC: [%x]\n", PC);
  neorv32_uart0_printf("jump back to FIRQ -> MAIN [%x]\n", firq_PC);
  asm volatile ("jalr ra, %0" : : "r" (firq_PC));

Error I Got

.... 
(After jump to task.bin)
......
Current PC: [0x00020224]
Jump Back to Main: [0x0000f474]
<RTE> [M] Load access fault @ PC = 0x000008F2, MTVAl = 0x24203A60 </RTE>
<RTE> [M] Instruction access fault @ PC = 0x24203A50, MTVAl = 0x00000000 [FATAL EXCEPTION] Halting CPU. </RTE>]

What could be potential reason I got this errors, and does it mean in order to normally execute after getting back from the interrupt, I need to disable its Machine Mode? If so, how?

[stnolting]

I'm not very familiar with the Zephyr OS kernel or its interrupt handling... 🙈 However, the general interrupt flow seems to be working in your setup.

In addition, I stored the previous PC before the interrupt handler in NEORV32_CFS->REG[61], and the current PC in NEORV32_CFS->REG[63]

Do you do this using software or did you wire up the CFU to the CPU's execution/control logic?

So you want to execute the "task.bin" program when your CFU interrupt triggers, right?

I am not sure, but somehow this feels like a stack corruption issue... 🤔

__asm__ volatile ("jalr ra, %0" : : "r" (0x00020000));

This might be dangerous as this overrides the ra register without the C environment noticing it. Furthermore, your code in task.bin will also override registers and the calling program is not aware of this.

Did you compile "task.bin" on its own (i.e. not linked with the base program)? Could you show the C-code of that?

If that task is just a "free-standing" single function (no linking with your base/main program) then you could try to use the naked attribute for it and make a manual context save/restore of all registers. Declaring that function as interrupt would add this context save/restore stuff but that would also place an mret at the corrupting the interrupt stack.

Another option would be to link everything during compilation. That would make things much easier as your task would become just another C function (where everyone is aware of) 😅

[SteveZ-Cal]

Do you do this using software or did you wire up the CFU to the CPU's execution/control logic?

Yes, I wired the fetch_PC from V1.8.6 NEORV32 CPU to CFS module (I just added another input port for PC in my CFS module)

So you want to execute the "task.bin" program when your CFU interrupt triggers, right?

Yes

Did you compile "task.bin" on its own (i.e. not linked with the base program)? Could you show the C-code of that?

Yes, It's a separate file loaded during boot time on 0x00020000; in addition, I tested and it's running fine with its own starting base_rom at 0x00020000.

task.bin

int main() {

  neorv32_uart0_printf("<<Before Exit Task>>\n");

  /***JUMP Back to the Main Program***/
  uint32_t firq_PC = NEORV32_CFS->REG[61];
  uint32_t PC = NEORV32_CFS->REG[63];
  firq_PC = firq_PC%4 + firq_PC;
  neorv32_uart0_printf("Current PC: [%x]\n", PC);
  neorv32_uart0_printf("jump back to FIRQ -> MAIN [%x]\n", firq_PC);
  asm volatile ("jalr ra, %0" : : "r" (firq_PC));

  neorv32_uart0_printf("<<Exit Task>>\n");

  return 0;
}

In addition below is how I define my Interrupt in CFS

(defined in architecture neorv32_cfs_rtl of neorv32_cfs)
signal irq_cfs : std_ulogic;
signal irq_cfs_dly: std_ulogic;

cfs_in_i(0) --> my key1 on FPGA

// just need to capture one pulse wave for interrupt
irq_process: process(rstn_i, clk_i)
begin:
       if (rstn_i = '0') then

           irq_cfs <= '0';
           irq_cfs_dly <= '0';

      elsif rising_edge(clk_i) then

           irq_cfs_dly <= irq_cfs;

           if cfs_in_i(0) = '1' then
               irq_cfs <= '1';
           else
              irq_cfs<='0';
           end if;

     end if;
end process irq_process;
irq_o <= irq_cfs and (not irq_cfs_dly); -- getting CFS one pulse interrupt
           

Lastly, what would be a corresponding inline assembly that let you directly jump to 0x00020000, without overrides any other registers.

[stnolting]

Well, the most "comfortable" way would be to put everything into a single compilation unit / link everything.

However, you can also use your concept of having two entirely separated programs. For the "main" program I would use a pointer-based calling of the task program so the compiler / C-runtime is aware of this jump (and you do not have to worry about polluting the ra register):

  // call task program
  uint32_t task_program_base_addr = 0x00020000U;
  void (*task_pnt)(void);
  task_pnt= (void*)task_program_base_addr;
  (*task_pnt)();

The task program is required to perform a full context save when entering and also a full context restore when exiting. Since this program does not "know" which registers need a backup you will need to do this by hand by declaring that function as "naked" and adding some assembly code to push/pop all register to/from the stack:

void __attribute__((naked)) main(void) {

  // save context, do not backup x0 and sp
  asm volatile (
#ifndef __riscv_32e
    "addi sp, sp, -30*4 \n"
#else
    "addi sp, sp, -14*4 \n"
#endif
    "sw x1,   0*4(sp) \n"
    "sw x3,   1*4(sp) \n"
    "sw x4,   2*4(sp) \n"
    "sw x5,   3*4(sp) \n"
    "sw x6,   4*4(sp) \n"
    "sw x7,   5*4(sp) \n"
    "sw x8,   6*4(sp) \n"
    "sw x9,   7*4(sp) \n"
    "sw x10,  8*4(sp) \n"
    "sw x11,  9*4(sp) \n"
    "sw x12, 10*4(sp) \n"
    "sw x13, 11*4(sp) \n"
    "sw x14, 12*4(sp) \n"
    "sw x15, 13*4(sp) \n"
#ifndef __riscv_32e
    "sw x16, 14*4(sp) \n"
    "sw x17, 15*4(sp) \n"
    "sw x18, 16*4(sp) \n"
    "sw x19, 17*4(sp) \n"
    "sw x20, 18*4(sp) \n"
    "sw x21, 19*4(sp) \n"
    "sw x22, 20*4(sp) \n"
    "sw x23, 21*4(sp) \n"
    "sw x24, 22*4(sp) \n"
    "sw x25, 23*4(sp) \n"
    "sw x26, 24*4(sp) \n"
    "sw x27, 25*4(sp) \n"
    "sw x28, 26*4(sp) \n"
    "sw x29, 27*4(sp) \n"
    "sw x30, 28*4(sp) \n"
    "sw x31, 29*4(sp) \n"
#endif
  );

    neorv32_uart0_printf("<<Before Exit Task>>\n");

    // other stuff ...

    neorv32_uart0_printf("<<Exit Task>>\n");

  // restore context, do not restore x0 and sp
  asm volatile (
    "lw x1,   0*4(sp) \n"
    "lw x3,   1*4(sp) \n"
    "lw x4,   2*4(sp) \n"
    "lw x5,   3*4(sp) \n"
    "lw x6,   4*4(sp) \n"
    "lw x7,   5*4(sp) \n"
    "lw x8,   6*4(sp) \n"
    "lw x9,   7*4(sp) \n"
    "lw x10,  8*4(sp) \n"
    "lw x11,  9*4(sp) \n"
    "lw x12, 10*4(sp) \n"
    "lw x13, 11*4(sp) \n"
    "lw x14, 12*4(sp) \n"
    "lw x15, 13*4(sp) \n"
#ifndef __riscv_32e
    "lw x16, 14*4(sp) \n"
    "lw x17, 15*4(sp) \n"
    "lw x18, 16*4(sp) \n"
    "lw x19, 17*4(sp) \n"
    "lw x20, 18*4(sp) \n"
    "lw x21, 19*4(sp) \n"
    "lw x22, 20*4(sp) \n"
    "lw x23, 21*4(sp) \n"
    "lw x24, 22*4(sp) \n"
    "lw x25, 23*4(sp) \n"
    "lw x26, 24*4(sp) \n"
    "lw x27, 25*4(sp) \n"
    "lw x28, 26*4(sp) \n"
    "lw x29, 27*4(sp) \n"
    "lw x30, 28*4(sp) \n"
    "lw x31, 29*4(sp) \n"
#endif
#ifndef __riscv_32e
    "addi sp, sp, +30*4 \n"
#else
    "addi sp, sp, +14*4 \n"
#endif
    "ret              \n"
  );
}

[stnolting]

I hope this helps 😉

I just tested such a setup and it seems to work just fine.

[SteveZ-Cal]

That definitely help!! Thanks a lot :)

I have a few follow up questions:

[Q1]

Just want to confirm that regardless of what environment (Zephyr, NEORV32, etc) we compiled the .bin file; the registers recognized by each file is just from x0 to x31. (RISC-V)
If that's the case, given that I don't know which registers need a backup, then save context / restore context should be exactly the same as above and applicable to all files regardless of their environment?

[Q2]

For now, for the testing purpose, both the main file and task file are compiled with NEORV32 (V1.8.6), with main file stored at 0x00000, and task file stored at 0x20000

I want to test if interrupt handler installed in one file (task file) can be used in another file (main file) --> this is what I want to achieve currently

task file mainly used for manage interrupt handler, main file is just where it be called by interrupt handler from task file, since it starts from 0x00000 will run neorv32 crt0 start-up code (I'm not exactly sure what got reset in this case --> this is also what I want to know) from neorv32.ld

Below is just sample programs for main and task file (Hardware, FPGA setup is same as above):

NEORV32_CFS->REG[3] used as flag in cfs_irq_handler below, NEORV32_CFS->REG[5] used as pointer for get back to cfs_irq_handler after finishing main file, and NEORV32_CFS->REG[6] used for saving the sp in cfs_irq_handler

main file:

task1(){

    unsigned int pc;

    __asm__ __volatile__ (
        "auipc %0, 0\n"    // Load the PC into %0
        : "=r" (pc)
    );

    neorv32_uart0_printf("Enter task1,  pc at [%x]\n", pc); // check to see if it's at around [0x2000]

   neorv32_rte_print_logo(); // content part of task1, but in future will subjected to change to the one I want
   
   NEORV32_CFS->REG[3] = 1; // signal cfs_irq_handler, set flag to 1

    unsigned int ra_value = NEORV32_CFS->REG[5]; // jump back to the unfinished cfs_irq_handler part
    void (*task_pnt)(void);
    task_pnt= (void*)ra_value;
    (*task_pnt)();

}

int main(){
 
task1();

}

task file:

void cfs_irq_handler(){

  /** pointer-based calling to get to main file**/
  uint32_t task_program_base_addr = NEORV32_IMEM_BASE;
  void (*task_pnt)(void);
  task_pnt = (void*)task_program_base_addr;

  unsigned int pc;

  __asm__ __volatile__ (
      "auipc %0, 0\n"   
      : "=r" (pc)
  );

// if flag set then it will just exit, else will go the main file

 if(NEORV32_CFS->REG[3]!=1){

    NEORV32_CFS->REG[5] = pc;

   neorv32_cpu_csr_clr(CSR_MIP, 1 << CFS_FIRQ_PENDING);

// save context, do not backup x0 and sp
    asm volatile (
    #ifndef __riscv_32e
      "addi sp, sp, -30*4 \n"
    #else
      "addi sp, sp, -14*4 \n"
    #endif
      "sw x1,   0*4(sp) \n"
      "sw x3,   1*4(sp) \n"
      "sw x4,   2*4(sp) \n"
      "sw x5,   3*4(sp) \n"
      "sw x6,   4*4(sp) \n"
      "sw x7,   5*4(sp) \n"
      "sw x8,   6*4(sp) \n"
      "sw x9,   7*4(sp) \n"
      "sw x10,  8*4(sp) \n"
      "sw x11,  9*4(sp) \n"
      "sw x12, 10*4(sp) \n"
      "sw x13, 11*4(sp) \n"
      "sw x14, 12*4(sp) \n"
      "sw x15, 13*4(sp) \n"
    #ifndef __riscv_32e
      "sw x16, 14*4(sp) \n"
      "sw x17, 15*4(sp) \n"
      "sw x18, 16*4(sp) \n"
      "sw x19, 17*4(sp) \n"
      "sw x20, 18*4(sp) \n"
      "sw x21, 19*4(sp) \n"
      "sw x22, 20*4(sp) \n"
      "sw x23, 21*4(sp) \n"
      "sw x24, 22*4(sp) \n"
      "sw x25, 23*4(sp) \n"
      "sw x26, 24*4(sp) \n"
      "sw x27, 25*4(sp) \n"
      "sw x28, 26*4(sp) \n"
      "sw x29, 27*4(sp) \n"
      "sw x30, 28*4(sp) \n"
      "sw x31, 29*4(sp) \n"
    #endif
    );
    asm volatile("mv %0, sp" : "=r" (NEORV32_CFS->REG[6]));
    (*task_pnt)();

  }

// restore context, do not restore x0 and sp
  asm volatile("mv sp, %0" : : "r" (NEORV32_CFS->REG[6]));
  asm volatile (
    "lw x1,   0*4(sp) \n"
    "lw x3,   1*4(sp) \n"
    "lw x4,   2*4(sp) \n"
    "lw x5,   3*4(sp) \n"
    "lw x6,   4*4(sp) \n"
    "lw x7,   5*4(sp) \n"
    "lw x8,   6*4(sp) \n"
    "lw x9,   7*4(sp) \n"
    "lw x10,  8*4(sp) \n"
    "lw x11,  9*4(sp) \n"
    "lw x12, 10*4(sp) \n"
    "lw x13, 11*4(sp) \n"
    "lw x14, 12*4(sp) \n"
    "lw x15, 13*4(sp) \n"
    #ifndef __riscv_32e
      "lw x16, 14*4(sp) \n"
      "lw x17, 15*4(sp) \n"
      "lw x18, 16*4(sp) \n"
      "lw x19, 17*4(sp) \n"
      "lw x20, 18*4(sp) \n"
      "lw x21, 19*4(sp) \n"
      "lw x22, 20*4(sp) \n"
      "lw x23, 21*4(sp) \n"
      "lw x24, 22*4(sp) \n"
      "lw x25, 23*4(sp) \n"
      "lw x26, 24*4(sp) \n"
      "lw x27, 25*4(sp) \n"
      "lw x28, 26*4(sp) \n"
      "lw x29, 27*4(sp) \n"
      "lw x30, 28*4(sp) \n"
      "lw x31, 29*4(sp) \n"
    #endif
    #ifndef __riscv_32e
      "addi sp, sp, +30*4 \n"
    #else
      "addi sp, sp, +14*4 \n"
    #endif
   // "ret              \n"
  );

   NEORV32_CFS->REG[3] = 0; // reset flag to any (not 1)

  neorv32_uart0_printf(">>>>>Exit NEORV32 CFS_IRQ_HANDLER<<<<<\n");

return;
}

int main(){

    /** setup interrupt handler in task file only (required)**/

    neorv32_rte_setup();
    neorv32_rte_handler_install(CFS_RTE_ID, cfs_irq_handler); // CFS to RTE
    neorv32_cpu_csr_set(CSR_MIE, 1 << CFS_FIRQ_ENABLE); // enable CFS FIRQ1
    neorv32_cpu_csr_set(CSR_MSTATUS, 1 << CSR_MSTATUS_MIE); // enable machine-mode interrupts;

}

Problem:

2.1

Initially it's not working as intended, when I interrupt -> main file, it's not getting back to the cfs_irq_handler();
--> [Q3] temporarily resolved

2.2

In addition to the issue from [Q3] below by potentially need to change __crt0_stack_begin for two files

I was able to interrupt only once, and the next time it won't recognize the interrupt after getting back from the main file. Based on that I thought the interrupt handler I installed in task file was cleared after calling main file since it needs to execute the crt0 start up code.

Is there a way for interrupt handler to be recognized?

[Q3]

Based on neorv32.ld:
/* Export symbols for neorv32 crt0 start-up code */
PROVIDE(__crt0_stack_begin = (ORIGIN(ram) + LENGTH(ram)) - 4);

Does it indicates that I also have to change the _crt0_stack_begin as well since main file and task file compiled separately and in order to avoid stack pollution, I have to define a separate _crt0_stack_begin for each file?

I tried that later, and it do resolve the issue for main file to jump back to cfs_irq_handler(); but that changing doesn't seem to the most clean/logical way?

[stnolting]

[Q1]

Thats right - as long as your compile for a rv32i architecture (which is the default one). If you use the "embedded" rv32e architecture then only registers x0 to x15 exist.

Some "special" registers like gp won't be touched by GCC (as far as I know). However, I would recommend to backup all registers just be safe (except for x0, which is hardwired to zero).

[Q2]

First of all, I would recommend to upgrade your NEORV32 setup to the latest version. We had some minor bug fixes since version 1.8.6 (-> CHANGELOG).

I want to test if interrupt handler installed in one file (task file) can be used in another file (main file) --> this is what I want to achieve currently

This should be no problem.

Regarding the start-up code, here is a list of the things crt0 is doing: https://stnolting.github.io/neorv32/#_start_up_code_crt0

[Q3]

This is a hard question... Basically, both executables (main and task) are compiled using the same linker script, the same start-up code and the same address map. The crt0-part of main will be executed right after reset but the crt0-part of your task executable should never be executed as you (should) never jump to the beginning of that memory region - just to the base addresses of the according functions.

So - in theory - you do not need to modify anything here. Main's crt0 will setup the stack and all the code from the task executable will just use that.

But there is a problem. Unless you compile your task code (the one located at address != 0) as "position independent code" the compiler will think the code is located at address zero and might create code to compute offsets based on this address.

And there is another problem. A global stack is fine as we just pull/push data. But .bss and .data sections might be a problem. Both executables "think" they are alone in the world of NEORV32. But in fact, they are not. So they might override .data and/or .bss regions that are used by the other executable.

I still think the "simplest" solution would be to link the sources of both executables together so that they are aware of each other.