JTAG scan chain interrogation failed: all ones

[cmenendezzz]

Hi, I have encountered a technical issue working on a project in the NEXYS A7 100t board. Specifically, I am facing difficulties in establishing a connection with the JTAG interface with OPENOCD.

I am following the example of the blinking led. Due to the fact that J10 ports are not being recognized by Vivado, I have resorted to using a PMOD to extract the signals for JTAG. I have connected the JTAG HS2 and modified the OpenOCD configuration file accordingly. However, I am currently unable to establish a successful connection.
Then, I have installed riscv-openocd and edit the configuration file accordingly. Nevertheless, I encounter the same error. It seems to be
that TDO is stuck at 1 so no communication with your target is going to work. Is this a recurrent error? Do I have to modify the blinking led example?

Thank you in advance.

$ sudo openocd -f riscv-openocd/tcl/interface/ftdi/digilent-hs2.cfg -f neorv32/sw/openocd/openocd_neorv32.cfg
Open On-Chip Debugger 0.12.0+dev-02948-g92c031926 (2023-07-05-09:32)
Licensed under GNU GPL v2
For bug reports, read
	http://openocd.org/doc/doxygen/bugs.html
none separate

Info : clock speed 2000 kHz
Error: JTAG scan chain interrogation failed: all ones
Error: Check JTAG interface, timings, target power, etc.
Error: Trying to use configured scan chain anyway...
Error: neorv32.cpu: IR capture error; saw 0x1f not 0x01
Warn : Bypassing JTAG setup events due to errors
Error: dtmcontrol is 0. Check JTAG connectivity/board power.
Warn : target neorv32.cpu.0 examination failed
Info : starting gdb server for neorv32.cpu.0 on 3333
Info : Listening on port 3333 for gdb connections
Error: Target not examined yet

[stnolting]

Hey @cmenendezzz!

What hardware setup are you using (i.e. processor configuration)?

It seems to be that TDO is stuck at 1 so no communication with your target is going to work

Hmm, I know this problem, but in most cases there is just problem with the wiring between FPGA and JTAG. If you pull-down your TDO wire (so, the one going to the JTAG adapter) do you get an "all zero" error?

[cmenendezzz]

Reading you answer, I guess i probably have to specify the memory sizes in the openocd configuration file?

[cmenendezzz]

I also tested
MEM_INT_IMEM_SIZE : natural := 161024;
MEM_INT_DMEM_SIZE : natural := 81024

But I still have the same error...

[stnolting]

The openOCD file seems just fine. I was talking about the linker script sw/common/neorv32.ld. The region configuration in this file has to be sync to the hardware configuration. I.e. this section:

neorv32/sw/common/neorv32.ld

Lines 59 to 61 in a299bf7

	/* Default rom/ram (IMEM/DMEM) sizes */
	__neorv32_rom_size = DEFINED(__neorv32_rom_size) ? __neorv32_rom_size : 2048M;
	__neorv32_ram_size = DEFINED(__neorv32_ram_size) ? __neorv32_ram_size : 8K;

For your setup I would constrain both regions to 8kB:

/* Default rom/ram (IMEM/DMEM) sizes */
__neorv32_rom_size = DEFINED(__neorv32_rom_size) ? __neorv32_rom_size : 8K;
__neorv32_ram_size = DEFINED(__neorv32_ram_size) ? __neorv32_ram_size : 8K;

Since you are using an Eclipse-based build system... did you add the NEORV32-specific makefile and linker script to that build flow?

[cmenendezzz]

I decided to do it simplier and to leave Eclipse behind for a second.

Today I managed to debug your example hello_world on neorv32/sw/example as you well explained in the user guide of the NERV32.

Now, I am trying to debug a simply hello world in C:

#include <stdio.h>
int main() {
   printf("Hello, World!");
   return 0;
}

In order to compile the application , I did as you said, I copy an example and I replace the main.c for my main.c. Then I followed the same procedure as for the other example: open opencod, run gdb... And this is the actual situation:
(above the gdb and below the openocd)

We can see the same memory errors on the OpenOCD window.
I guess there is some problem loading the rom ...
One last question , I though that my main.c was loaded in the RAM, but is it true? With the latest error seams to be loaded in the ROM.

Thank you in advance again!

[stnolting]

Today I managed to debug your example hello_world on neorv32/sw/example as you well explained in the user guide of the NERV32.

Great to hear! 👍

We can see the same memory errors on the OpenOCD window.

I think I know where this error comes from... your executable is just way too big:

The size is 0x11318 - that's about 70kB, so much more than the 8kB IMEM. Do you use any standard C library functions like printf?

One last question , I though that my main.c was loaded in the RAM, but is it true? With the latest error seams to be loaded in the ROM.

If the bootloader is enabled (INT_BOOTLOADER_EN => true) then the internal instruction memory (IMEM) gets implemented as RAM as you need to be able to write the executable to it. Section Boot Configuration of the data sheet gives some more information about this.

[cmenendezzz]

The size is 0x11318 - that's about 70kB, so much more than the 8kB IMEM. Do you use any standard C library functions like printf?
Yes, you are right. I used the printf function, which I guess it consumes a significant amount of memory. That is why you created your own printf function that utilizes UART communication, which is more memory-efficient?

By the way, I've made great progress with debugging via Eclipse. I think it is very useful to understand what is happening behind but once you know it is easier to launch everything from a single click! However, I'm currently facing one last issue (hopefully): I can't access a variable in GDB.

Here is the situation:
I have a main.cpp file that calls two functions: twiceTwo (which multiplies a number by two) and square (which returns the square of a number). Both of these functions are in another file, and I've included a header file in main.cpp to use them. You can find the main.cpp file below.

During the debugging process, I can see that my functions are being called correctly. However, I encounter an error when I try to access the value of 'a' in GDB:

dwarf2_find_location_expression: Corrupted DWAEF expression.

Have you faced this error before?
(this error occurs when I use print (or p) a and print &a)

Thank you again for your help! And once I solve this problem, I can show you/ send you a file where I wrote down all the steps to debug via Eclipse (if you are interested!)

[stnolting]

I am not familiar with this error, but it might be related to GCC's compile switches.

Most of the the time I use -g to add debug symbols to the final elf file. Do you set such a switch in your compilation setup? You could try to use -gdwarf-2 instead to add debug information in DWARF-2 format.

Thank you again for your help! And once I solve this problem, I can show you/ send you a file where I wrote down all the steps to debug via Eclipse (if you are interested!)

That would be really cool! 👍

[cmenendezzz]

Most of the the time I use -g to add debug symbols to the final elf file. Do you set such a switch in your compilation setup? You could try to use -gdwarf-2 instead to add debug information in DWARF-2 format.

I managed to solve the problem by realizing that the processor had already optimized the variable, rendering it useless in the end, and I couldn't access it. Consequently, I made a change in the makefile, replacing EFFORT ?= -Os with EFFORT ?= -O0.

Additionally, I've completed the guideline document. How would you like me to send it to you?

[stnolting]

I managed to solve the problem by realizing that the processor had already optimized the variable, rendering it useless in the end, and I couldn't access it.

👍

Consequently, I made a change in the makefile, replacing EFFORT ?= -Os with EFFORT ?= -O0.

-O0 is heavy, as this results in very large (and inefficient) executable. However, for debugging purpose this might be handy.

Additionally, I've completed the guideline document. How would you like me to send it to you?

That's really cool! Thank you so much for sharing! Maybe this is something we could integrate into the user guide? Just file a PR or open a new discussion for that so everyone can have a look. 😉

[cmenendezzz]

Hello again!
I've been working on testing the UART port using your hello_world file, but unfortunately, I've encountered some issues. I want to share the relevant parts of the main.c file:

#include <neorv32.h>
#define BAUD_RATE 19200

int main() {
  neorv32_rte_setup();
  neorv32_uart0_setup(BAUD_RATE, 0);
  neorv32_rte_check_isa(0); // silent = 0 -> show message if isa mismatch
  neorv32_rte_print_logo();
  neorv32_uart0_puts("Hello world! :)\n");
  return 0;
}

At the moment, the first function, neorv32_rte_setup(), executes correctly. However, as soon as I try to access the second function, an immediate interrupt occurs. Specifically, the interrupt corresponds to RTE_TRAP_S_ACCESS = 7, indicating a store (bus) access fault. It appears that there might be an issue with accessing memory related to the peripherals.

Do I have to defined the memory mapping for the peripherals? (I thought that is was already definded)

Here a picture of the situation: after executing UARTx->CTRL=0 from the neorv32_uart_setup function ( so after entering in one of the UART registers) an interrupt occurs.

[stnolting]

Well, this should not happen... 😅

Do you have UART0 enabled in your setup (i.e. IO_UART0_EN => true,)? Could you show the according UART output when the IRQ kicks in? The RTE? The RTW (runtime environment) should output something starting with <RTE> ... in this case. Alternatively, could you also show the mtval CSR in your debugging session?

[cmenendezzz]

Do you have UART0 enabled in your setup (i.e. IO_UART0_EN => true,)?

Yes it is enabled.

Then here is the mtval CSR :

When the IRQ kicks in, that is what it does:

1. It enters neorv32_rte_setup()

2. It goes into neorv32_uart0_setup(BAUD_RATE,0) (BAUD_RATE=19200)

3. Executes UARTx->CTRL=0

4. RTE core is activated with rte_cause=7

5. Then goes into _neorv32_rte_debug_handler
   

6. Executes the neorv32_uart0_function_available()
   

7. Executes the first condition and and interrupt happens

8. Now the rte_mcause is equals to 5 (which corresponds to " Load (bus) access fault" I guess)
   

9. Never finishes the neorv32_uart0_function_available() function.

[stnolting]

First at all, this is not an interrupt - it is an exception (my mistake 😅). The CPU tries to write to an address that seems not be mapped to any module. Hence, the bus transaction is not acknowledged leading to a bus timeout that finally raises "store access" exception. The actual address the CPU was trying to access is stored to mtval CSR in this case.

If I understand your IDE layout correctly then mtval is 0xEF009043, right?

This address is not mapped to any module at all by default (see DOC: Address Space).

The CPU is executing UARTx->CTRL=0 while this exception kicks in. But that does not make sense as UART0 is located at base address 0xFFFFF500 - far away from 0xEF009043... 🤔 This seems to be a problem with the program itself / the compilation.

• Could you provide the UART output as well?
• Could you provide a minimal example program (including a disassembly would be great)?
• Does the program work if you upload it via the UART bootloader?
• Can you try to compile the program in the console using the default makefile?

[stnolting]

This is also very strange:

clock should not be optimized away here... What compiler switches do you use?

[cmenendezzz]

The CPU tries to write to an address that seems not be mapped to any module. Hence, the bus transaction is not acknowledged leading to a bus timeout that finally raises "store access" exception. The actual address the CPU was trying to access is stored to mtval CSR in this case.

I see...

If I understand your IDE layout correctly then mtval is 0xEF009043, right?

yes!

clock should not be optimized away here... What compiler switches do you use?

I am using optimization O0... It shouldn't appear , isn't it?

Could you provide a minimal example program (including a disassembly would be great)?

I am using your hello world program, I didn't touch anything.

Can you try to compile the program in the console using the default makefile?

I just did it and I obtain the same thing...

Does the program work if you upload it via the UART bootloader?

I am going to try right now! But how doing it could solve the memory acces problem?

[stnolting]

I am using optimization O0... It shouldn't appear , isn't it?

GCC should not optimize out any variables at all.

I am going to try right now! But how doing it could solve the memory acces problem?

The runtime environment prints out a lot of information when encountering an exception. That might help to isolate the cause.

[cmenendezzz]

I also try commenting the neorv32_rte_setup() function in order to see if this function has something to do about it.

And I found that now the neorv32_uart0_setup(BAUD_RATE,0) executed till the end. Nevertheless, as you can see below in the screenshoot, UARTX->CTRL does not take the tmp value...

[stnolting]

Interesting...

I also try commenting the neorv32_rte_setup() function

Better keep that. Otherwise the CPU might go crazy if an exception is triggered.

[cmenendezzz]

That would be the reason why he could not find the ctr0.S? Because, when I commented that line, this error shows up (but does not prevent from finishing the execution of the rest of the functions).

[cmenendezzz]

Could you provide the UART output as well?

I am using minicom and nothing outputs

[stnolting]

Also I am using 1.8.5 version

Could you upgrade to the latest version? We had some severe bug until version 1.8.5.3 (#626).

Can you enable the bootloader (INT_BOOTLOADER_EN => true) and re-synthesize the design?
Then connect via UART terminal, press reset and check if the bootloader menu appears (do not start a debugging sessions during this).

[cmenendezzz]

So I create a new project with the latest version . And I followed your instructions (I also read the corresponding section in NEORV32's documentation) and no menu appears . I tried with minicom and cutecom and in none of them a menu appears... During those tests I disconnected the JTAG adapter .

Nevertheless, now I don't encounter exceptions during the hello world debugging. But unfortunatly the message does not appers in the serial port. SO I can say that the latest version fixed some bugs, it's an avancement!

[stnolting]

So I create a new project with the latest version . And I followed your instructions (I also read the corresponding section in NEORV32's documentation) and no menu appears . I tried with minicom and cutecom and in none of them a menu appears... During those tests I disconnected the JTAG adapter .

Have you connected some LEDs to the gpio_o port? At least one LED should be connected to gpio_o(0) to show the bootloader status (should be blinking after reset).

Btw, I always mess up the UART's RXD and TXD pins when wiring them to a serial adapter - so better check twice 😉

Nevertheless, now I don't encounter exceptions during the hello world debugging. But unfortunatly the message does not appers in the serial port. SO I can say that the latest version fixed some bugs, it's an avancement!

There is a general problem when debugging code from ROM (i.e. the setup with the bootloader disabled). In that case the debugger cannot use "normal" breakpoints (temporarily inserting ebreak instructions into the memory). So it has to use "hardware breakpoints" (via the CPU's trigger module). I am not sure if your IDE handles that correctly. When running GDB straight from the console the user is responsible of using the "right" breakpoint type.

Anyway, good to hear that we have one problem less now 😅👍

[cmenendezzz]

Have you connected some LEDs to the gpio_o port? At least one LED should be connected to gpio_o(0) to show the bootloader status (should be blinking after reset).

Yes! I connected 8 LEDs! And they blink after reset.

Btw, I always mess up the UART's RXD and TXD pins when wiring them to a serial adapter - so better check twice

And I was actually using the USB-UART Bridge (I told myself that it would avoid confusions). But probably that is where the problem is.

There is a general problem when debugging code from ROM (i.e. the setup with the bootloader disabled)

I see but this whole time I've been using the test_on_chip_debugger example where the bootloader is enabled!

When running GDB straight from the console the user is responsible of using the "right" breakpoint type.

Eventhought, I have runned my program from Eclipse and directly from GDB and I obtained the same results. I don't think the problem comes from there.

[stnolting]

Yes! I connected 8 LEDs! And they blink after reset.

Great news! So the bootloader seems to be working correctly. Now we just need to get some output from it. Did you check your serial adapter? For example wiring RX and TX together to test if the "echo" works?

I see but this whole time I've been using the test_on_chip_debugger example where the bootloader is enabled!

👍

Eventhought, I have runned my program from Eclipse and directly from GDB and I obtained the same results. I don't think the problem comes from there.

Cool, so I think your IDE is capable of handling that correctly.

[cmenendezzz]

Did you check your serial adapter? For example wiring RX and TX together to test if the "echo" works?

It works! The problem was the USB-UART Bridge. I changed the constraint file and connect it to a pmod and it worked right away! Thank you very much!!

And concerning the guidelines to debug via Eclipse, I have to add some details but I will post it once it is done!

Thank you very much again for your help!

[stnolting]

Great to hear!!! 👍 🎉