A question on volatile variable access to memory

[mahdi259]

Hi
I have a question on volatile variable access to memory. As I described in #906 , I have declared some volatile variables in my code and used @snolting 's vhdl code to capture their changes.

The odd thing is that there are more accesses to one memory location than I wrote in the code.

[stnolting]

Could you show the entire code (including the variable declaration)?

Do you have a cache enabled?

[mahdi259]

Sure. Give me some minutes.

[mahdi259]

Question.zip
Cache is not enabled.

[mahdi259]

Firstly thanks for replying.
It writes 0x00000001 in step variable three times. The following text file contains the last console outputs.
console.txt

[stnolting]

Ahh ok.

So, there are 6 writes to cls in that log file. The very first write (writing zero) is is the clearing of the .bss in the application start code.

Now we need to find out where the remaining 5 writes are coming from... Where did you put the "VHDL monitor code"? Could you please also share that code snippet?

[mahdi259]

Yes, that's in the HW folder of Question.zip.
neorv32_tb.simple.vhd

[stnolting]

Ahh I see. You are using the default testbench for simulation which has the external bus cache enabled:

XBUS_CACHE_EN => true, -- enable external bus cache (x-cache)

The additional write accesses your are seeing in the terminal are caused by the block write back of the XCACHE.

[mahdi259]

Sorry, What does it do exactly?

[stnolting]

The XCACHE caches all access that go through the processor's external bus interface.

Here is an overview of the processor's bus system: https://stnolting.github.io/neorv32/#_bus_system

And here is the documentation of the XBUS + XCACHE: https://stnolting.github.io/neorv32/#_processor_external_bus_interface_xbus

[mahdi259]

Thanks a lot. That was the case...

[mahdi259]

I think a block size bigger than 1 may cause this case (more memory writes than specified), right?🤔

From datasheet: The write-back strategy will gather all writes locally inside the cache until the according cache block is about to be replaced. In this case, the entire modified cache block is written back to main memory.

[mahdi259]

As you guided I use volatile variables to capture operation results in simulation.

[stnolting]

Yeah, but this is not a really good trick I have to admit 😅

Are you using any caches in your design? If so, which ones and why?

You can "manually" bypass the cache during load/stores (without touching it - so no misses etc.) if you use atomic LR/SC operations as these instructions will always bypass the cache. But beware, it makes memory coherency slightly more complicated.

[mahdi259]

As I described in #923, When I only use external memory, the function with more compression rate experiences more latency due to instruction fetch latency. So I use instruction cache and also dcache.

Thanks I will try to do it. Maybe fence will synchronize and should be used after atomic LR/SC operations. right?

(I use external memory to compare different memory latencies.)

[stnolting]

When I only use external memory, the function with more compression rate experiences more latency due to instruction fetch latency. So I use instruction cache and also dcache.

The "optimal" cache configuration highly depends on your external memory system and also on your actual application. Maybe the XCACHE is already sufficient (no need for i-cache and d-cache). 🤔

Thanks I will try to do it. Maybe fence will synchronize and should be used after atomic LR/SC operations. right?

Yes, that is a good idea. 😉

Here is a cut-out from the processor test program where I am synchronizing atomic memory accesses:

neorv32/sw/example/processor_check/main.c

Lines 1932 to 1941 in cc8af90

	amo_var = 0x00cafe00; // initialize
	asm volatile ("fence"); // flush/reload d-cache
	tmp_a = neorv32_cpu_load_reservate_word((uint32_t)&amo_var);
	amo_var = 0x10cafe00; // break reservation
	asm volatile ("fence"); // flush/reload d-cache
	tmp_b = neorv32_cpu_store_conditional_word((uint32_t)&amo_var, 0xaaaaaaaa);
	tmp_b = (tmp_b << 1) | neorv32_cpu_store_conditional_word((uint32_t)&amo_var, 0xcccccccc); // another SC: must fail
	tmp_b = (tmp_b << 1) | neorv32_cpu_store_conditional_word((uint32_t)ADDR_UNREACHABLE, 0); // another SC: must fail; no bus exception!
	asm volatile ("fence"); // flush/reload d-cache

However, fences will take a lot of time as they first flush the cache(s) and then reload the cache(s).

[mahdi259]

Thanks a lot.