A question on running llama on neorv32

[mahdi259]

Hi
I'm thrilled to run LLaMA.cpp on neorv32 processor. Do you think that it's possible to do that?

[Sam-Vervaeck]

Hi! Can you provide more context than just a file name? I recently tried running a cpp library on the NEORV32. The cpp language should not be a problem (documentation for cpp is just a bit barebones at the moment). However, you should check first if the LLAMA library is made for 32-bit systems. My cpp library was not and that was the actual source of some implementation problems.

[Sam-Vervaeck]

Looking at that project readme the setup mentions UNIX-like and Windows systems to set it up, so it def seems like it needs an OS to run on. Running it bare-metal on NEORV32 seems like a nono. You could maybe try to get it to run on the Zephyr OS port since that is close to Linux but that would probably require quite a bit of work.

[mahdi259]

Actually I couldn't find any OS specification on readme, thanks.

[Sam-Vervaeck]

It is not explicit, but they mention windows, unix, and docker. That all points to an OS.

[stnolting]

If it is just "plain C(++)" then it "should" be no "problem"... 😅

I have only briefly skimmed the documentation of llama and so far I could not identify any requirements beyond "plain C++" code... However, things will get complicated if llama requires some kind of advanced runtime environment like virtual memory, forking and threading, OS calls, ...

[mahdi259]

Thanks a lot for your response.