AMD GPUs

[fakerybakery]

Hi,
I saw this project currently only supports Cuda. I was wondering if it might be possible to use HIPIFY to make it work on AMD GPUs. Do you know if this would be possible?
Thank you!

[balisujohn]

So I'm using the GGML library, which I think may already support AMD ROCm, though for now I'm just compiling for cuda. If you have an AMD card with ROCm, I'm happy to help you get started on working on this, but I can't do it myself since I don't have an AMD card. https://github.com/ggerganov/ggml

[fakerybakery]

Thanks! Do you know if it works on CPU?

[balisujohn]

I have an experimental CPU branch, but the numbers don't match for now because I changed the behavior of some cuda ops without making the equivalent change for the CPU version of the OP. it will take a lot of work to fix, and probably be so slow it's far less useful, so it's low priority for me.

https://github.com/balisujohn/tortoise.cpp/tree/exp-processor

[balisujohn]

I should add that I'm supportive of adding support for all platforms including CPU, and will accept contributions towards support for any platform.

[balisujohn]

@fakerybakery CPU support is now added.