Add the HPDcache as cache subsystem

[cfuguet]

Add the HPDcache as another alternative for the cache subsystem.

The HPDcache is a highly configurable L1 Dcache that mainly targets high-performance systems.

[cfuguet]

Hello everyone !

Finally 😃 the pull-request to add the HPDcache as one of the cache subsystems for the CVA6. In this pull-request:

• I propose to import the HPDcache sources using the vendor.py script. I import the sources inside vendor/openhwgroup/cvhpdcache (Update: a submodule into core/cache_subsystem/hpdcache is used instead)
• The HPDcache is always compiled as the other cache subsystems (using the Flist.cva6). The target cache is selected in the configuration package (cv[32|64]a6_*_config_pkg.sv).
• As for the other cache subsystems, there is a cache_subsystem module, that integrates both the HPDcache and the CVA6_icache. However, the definition of this cache_subsystem module is in the target/cva6 directory of the HPDcache Git repository. I hesitated if putting this part in the HPDcache or in the CVA6 repository. Changes in the interface between the CVA6 and the caches will be likely easier to handle if the cache_subsystem module is part of the CVA6 repository. Something to discuss (Update: CVA6-specific files are migrated to the core/cache_subsystem directory)
• For the moment there is no test in the CI using a configuration with the HPDcache. It could be a good thing to run at least one or two tests using the HPDcache to be sure that future modifications in the CVA6 still work with the HPDcache.

Let me know if you think this is the good way,

Thanks

[cfuguet]

By the way, CMOs and Hardware Prefetcher interfaces to the cache are wired to 0 for the moment.

Regarding the CMOs, I have a patch that allows to support Zicbom (cache-block management operations, e.g. inval, flush) and Zicbop (cache-block prefetch operations). But we can discuss about integrating these extensions in the CVA6 after. I know that @Jbalkind is also working in these extensions to integrate with OpenPiton.

Regarding the hardware prefetcher, most of the configuration registers will be actually part of the HPDcache CSR register map. Thus, signals will not come from the CVA6 pipeline, but from the HPDcache itself. This interface will then disappear.

[zarubaf]

Awesome, very exciting - looks good! I've only gone through the cva6 related files.

I am okay with merging since it doesn't touch CVA6 sources much, but I would suggest we add some sort of sanity check (verilator compilation + dhrystone test for example?) to the CI to make sure there are no breaking changes happening :-)

[cfuguet]

Thanks @zarubaf,

I made the modification you proposed.

For the CI, I think that having at least Dhrystone as you proposed is a good idea.

[github-actions]

❌ failed run, report available here.

[JeanRochCoulon]

Error in vcs-testharness config:

[JeanRochCoulon]

@cfuguet @zarubaf As sanity check, I would propose to change the 64bit configuration cache from WT to HPD cache. This configuration is tested by smoke_test in veri-testharness, vcs-uvm and vcs-testharness. That's a way to test HPD regression wihtotu adding extra tests.

[cfuguet]

@cfuguet @zarubaf As sanity check, I would propose to change the 64bit configuration cache from WT to HPD cache. This configuration is tested by smoke_test in veri-testharness, vcs-uvm and vcs-testharness. That's a way to test HPD regression wihtotu adding extra tests.

That is Ok for me. However, people may be surprised of having a new cache subsystem with their currently used configuration.

[zarubaf]

I would prefer an extra sanity test in GitHub CI. It would indeed be surprising if the cache suddenly changed. The extra test should not add much additional delay as we are in general anyway waiting for the Thales CI.

[zarubaf]

Hm, since the CI passed on @niwis PR, could it be that you have some similar named function/module/etc. in the HPDCache and verilator is stumbling over it?

[github-actions]

❌ failed run, report available here.

[cfuguet]

Hm, since the CI passed on @niwis PR, could it be that you have some similar named function/module/etc. in the HPDCache and verilator is stumbling over it?

Yes I agree. It looks like a side-effect. I don't have any function with the same or similar name. Somehow, compiling the extra files for the HPDcache, now verilator is having some trouble with the inlining of code, even if the HPDcache is not actually used in the configurations failing. It clearly looks like a Verilator bug...

[github-actions]

❌ failed run, report available here.

[cfuguet]

Hm, since the CI passed on @niwis PR, could it be that you have some similar named function/module/etc. in the HPDCache and verilator is stumbling over it?

@zarubaf and @JeanRochCoulon, I'm starting to getting out of ideas... I commented out all HPDcache files in the Flist.cva6 file and the CI still fails on tests using the WB cache. WT tests go farther but they are cancelled after a while because of the WB failing. Do you have any ideas ?

Thanks !

[github-actions]

❌ failed run, report available here.

[niwis]

@cfuguet labelling the WB generate block seems to fix this: pulp-platform@e8f51a7
I guess verilator internally handles the three-case if-elseif-else statement differently than the previous two cases and stumbles across the missing label.

[cfuguet]

@cfuguet labelling the WB generate block seems to fix this: pulp-platform@e8f51a7 I guess verilator internally handles the three-case if-elseif-else statement differently than the previous two cases and stumbles across the missing label.

Good catch. That is great. Thank you @niwis

[cfuguet]

Additional changes introduced in the pull-request:

• Migrate AXI related definitions from the ariane_soc_pkg to the ariane_axi_soc_pkg. This allows to correctly propagate the AXI ID width from the CVA6 configuration package to the testbench. This feature is very useful for the HPDcache which can use more than 4 bits of AXI ID bits (the constant value that was defined in the ariane_soc_pkg) to issue more than 16 miss requests or write transactions to the memory.
• Minor fix in the CVA6 makefile to correctly support VCD tracing when using Verilator

[cfuguet]

By the way, @niwis your patch fixes the issue. Thank you again !

[github-actions]

❌ failed run, report available here.

[cfuguet]

Hi @JeanRochCoulon,

Could you please send me the failing report in the Thales CI ? It is currently failing on UVM tests and I guess it is because I changed the ariane_axi_soc_pkg and ariane_soc_pkg files and I did not modify completely the UVM testbench in cv_verif.

I don't have yet a local working cv_verif because some tooling issues. If the CI failing issue is not straightforward, I will make the effort of making the cv_verif work in my environment.

Thank you !

[JeanRochCoulon]

Hi @cfuguet Of course I can help, but I am busy Today. I let you know.

[cathales]

Here is the error

Error-[P1ARGS-CANTOPN-F] Cannot open file
  Unable to open '${HPDCACHE_DIR}/rtl/hpdcache.Flist' due to 'No such file or 
  directory'.

[github-actions]

❌ failed run, report available here.

[github-actions]

❌ failed run, report available here.

[cathales]

ERROR: [Synth 8-1031] hpdcache_pkg is not declared [/gitlab-runner/riscv-public/builds/21Uxzs_1/2/riscv-ci/cva6/core/cache_subsystem/cva6_hpdcache_if_adapter.sv:13]
ERROR: [Synth 8-1031] hpdcache_pkg is not declared [/gitlab-runner/riscv-public/builds/21Uxzs_1/2/riscv-ci/cva6/core/cache_subsystem/cva6_hpdcache_if_adapter.sv:31]
ERROR: [Synth 8-1031] hpdcache_req_sid_t is not declared [/gitlab-runner/riscv-public/builds/21Uxzs_1/2/riscv-ci/cva6/core/cache_subsystem/cva6_hpdcache_if_adapter.sv:31]
ERROR: [Synth 8-1031] hpdcache_pkg is not declared [/gitlab-runner/riscv-public/builds/21Uxzs_1/2/riscv-ci/cva6/core/cache_subsystem/cva6_hpdcache_if_adapter.sv:42]
ERROR: [Synth 8-1031] hpdcache_req_t is not declared [/gitlab-runner/riscv-public/builds/21Uxzs_1/2/riscv-ci/cva6/core/cache_subsystem/cva6_hpdcache_if_adapter.sv:42]
ERROR: [Synth 8-1031] hpdcache_pkg is not declared [/gitlab-runner/riscv-public/builds/21Uxzs_1/2/riscv-ci/cva6/core/cache_subsystem/cva6_hpdcache_if_adapter.sv:44]
ERROR: [Synth 8-1031] hpdcache_tag_t is not declared [/gitlab-runner/riscv-public/builds/21Uxzs_1/2/riscv-ci/cva6/core/cache_subsystem/cva6_hpdcache_if_adapter.sv:44]
ERROR: [Synth 8-1031] hpdcache_pkg is not declared [/gitlab-runner/riscv-public/builds/21Uxzs_1/2/riscv-ci/cva6/core/cache_subsystem/cva6_hpdcache_if_adapter.sv:45]
ERROR: [Synth 8-1031] hpdcache_pma_t is not declared [/gitlab-runner/riscv-public/builds/21Uxzs_1/2/riscv-ci/cva6/core/cache_subsystem/cva6_hpdcache_if_adapter.sv:45]
ERROR: [Synth 8-1031] hpdcache_pkg is not declared [/gitlab-runner/riscv-public/builds/21Uxzs_1/2/riscv-ci/cva6/core/cache_subsystem/cva6_hpdcache_if_adapter.sv:49]
ERROR: [Synth 8-1031] hpdcache_rsp_t is not declared [/gitlab-runner/riscv-public/builds/21Uxzs_1/2/riscv-ci/cva6/core/cache_subsystem/cva6_hpdcache_if_adapter.sv:49]
ERROR: [Synth 8-1031] hpdcache_pkg is not declared [/gitlab-runner/riscv-public/builds/21Uxzs_1/2/riscv-ci/cva6/core/cache_subsystem/cva6_hpdcache_if_adapter.sv:75]
ERROR: [Synth 8-1031] HPDCACHE_REQ_OFFSET_WIDTH is not declared [/gitlab-runner/riscv-public/builds/21Uxzs_1/2/riscv-ci/cva6/core/cache_subsystem/cva6_hpdcache_if_adapter.sv:115]
ERROR: [Synth 8-1031] HPDCACHE_REQ_OFFSET_WIDTH is not declared [/gitlab-runner/riscv-public/builds/21Uxzs_1/2/riscv-ci/cva6/core/cache_subsystem/cva6_hpdcache_if_adapter.sv:116]
ERROR: [Synth 8-1031] HPDCACHE_TAG_WIDTH is not declared [/gitlab-runner/riscv-public/builds/21Uxzs_1/2/riscv-ci/cva6/core/cache_subsystem/cva6_hpdcache_if_adapter.sv:116]
ERROR: [Synth 8-1031] HPDCACHE_REQ_AMO_LR is not declared [/gitlab-runner/riscv-public/builds/21Uxzs_1/2/riscv-ci/cva6/core/cache_subsystem/cva6_hpdcache_if_adapter.sv:127]
ERROR: [Synth 8-1031] HPDCACHE_REQ_AMO_SC is not declared [/gitlab-runner/riscv-public/builds/21Uxzs_1/2/riscv-ci/cva6/core/cache_subsystem/cva6_hpdcache_if_adapter.sv:128]
ERROR: [Synth 8-1031] HPDCACHE_REQ_AMO_SWAP is not declared [/gitlab-runner/riscv-public/builds/21Uxzs_1/2/riscv-ci/cva6/core/cache_subsystem/cva6_hpdcache_if_adapter.sv:129]
ERROR: [Synth 8-1031] HPDCACHE_REQ_AMO_ADD is not declared [/gitlab-runner/riscv-public/builds/21Uxzs_1/2/riscv-ci/cva6/core/cache_subsystem/cva6_hpdcache_if_adapter.sv:130]

[github-actions]

❌ failed run, report available here.

[github-actions]

✔️ successful run, report available here.

[JeanRochCoulon]

Congratulations, and thank you for the code modifications. I am pretty sure this cache will be widely used by the OS community.

[cfuguet]

Thank you @JeanRochCoulon, I'm very happy that we finally made this integration.

Regards,