Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

有什么办法既可以开启cuda自定义算子，又不会出现乱码 #371

Open
wuyougg opened this issue Aug 13, 2024 · 1 comment
Open

有什么办法既可以开启cuda自定义算子，又不会出现乱码 #371

wuyougg opened this issue Aug 13, 2024 · 1 comment

Comments

@wuyougg
Copy link

wuyougg commented Aug 13, 2024

我的750ti在开启cuda自定义算子之后出现乱码(用的是1.5b模型)，如果关闭的话差不多7s一个字(即使我使用最低精度)，我曾经在ollama使用llama3.1的7b版本，大约2s一个字，现在没有资金换显卡，请问有什么办法既可以开启cuda自定义算子，又不会出现乱码，或者怎么在ollama上使用rwkv模型
@BlinkDL
Copy link

BlinkDL commented Aug 19, 2024

可以用WebGPU后端试试

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@BlinkDL @wuyougg