Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Calling ggml from a Python script #120

Open
StellaAthena opened this issue Apr 30, 2023 · 1 comment
Open

Calling ggml from a Python script #120

StellaAthena opened this issue Apr 30, 2023 · 1 comment

Comments

@StellaAthena
Copy link

C n00b here, but I was wondering how nicely this library plays with Python. I have some existing codebases that are written in Python and would like make ggml an optional backend for people who don’t have GPUs.

Specifically, the main functionality I would like to be able to call would be:

  1. Computing a continuation based on an input prompt
  2. Computing the loglikelihood of a specific target continuation based on an input prompt

How easy is this to do?
@pikalover6
Copy link
Contributor

Some things that might help you out:
gpt-j python bindings
llama python bindings
similar issue
You could look around in kobold.cpp to see how they deal with it

@StellaAthena StellaAthena mentioned this issue Apr 30, 2023
Closed
CCLDArjun pushed a commit to CCLDArjun/ggml that referenced this issue Dec 18, 2023
@bitRAKE
add SIGINT support for _WIN32 environments (ggerganov#120)
2d15d6c 
* add SIGINT support for _WIN32 environments

* perhaps more consistent
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@StellaAthena @pikalover6