Large language model inference for power users. At the current development stage it is more or less a ChatGPT CLI/TUI.
pipx install git+https://github.com/danisztls/llmux
Go to platform.openai.com and log-in with your OpenAI account (register if you don't have one). Click on your name initial in the top-right corner, then select "View API keys". Finally click on "Create new secret key". That's it.
You may also need to add a payment method, clicking on Billing --> Payment methods. New accounts should have some free credits, but adding a payment method may still be mandatory. For pricing, check this page.
There are three alternatives.
a. Export the key as an env var: export OPENAI_API_KEY="<YOUR_KEY>"
b. Create a global configuration at $XDG_CONFIG_HOME/llmux/config.yaml or a local at ./llmux.yaml:
api-key: "YOUR_KEY"c. Use the command line option --key <YOUR_KEY>.
The configuration priority order is: Command line option > Environment variable > Local configuration file > Global configuration file.
Run llmux. Then just chat!
The number next to the prompt is the tokens used in the conversation at that point.
Use the /q command to quit and show the number of total tokens used and an estimate of the expense for that session, based on the specific model in use.
model: "gpt-3.5-turbo"
markdown: false
max_tokens: 500
temperature: 1ChatGPT CLI, by default, uses the original gpt-3.5-turbo model. In order to use other ChatGPT models, edit the model parameter in the config.yml file ore use the --model command line option. Here is a list of all the available options:
| Name | Pricing (input token) | Pricing(output token) |
|---|---|---|
gpt-3.5-turbo |
0.0015 | 0.002 |
gpt-3.5-turbo-0613 |
0.0015 | 0.002 |
gpt-3.5-turbo-16k |
0.003 | 0.004 |
gpt-4 |
0.03 | 0.06 |
gpt-4-0613 |
0.03 | 0.06 |
gpt-4-32k |
0.06 | 0.12 |
gpt-4-32k-0613 |
0.06 | 0.12 |
Pricing is calculated as $/1000 tokens.
Check this page for the technical details of each model.
Add the --multiline (or -ml) flag in order to toggle multi-line input mode. In this mode use Alt+Enter or Esc+Enter to submit messages.
Use the --context <FILE PATH> command line option (or -c as a short version) in order to provide the model an initial context (technically a system message for ChatGPT). For example:
llmux --context notes.txt
Both absolute and relative paths are accepted. Note that this option can be specified multiple times to give multiple files for context. Example:
llmux --context notes-from-thursday.txt --context notes-from-friday.txt
ChatGPT CLI saves all the past conversations (including context and token usage) in $XDG_DATA_HOME/llmux. In order to restore a session the --restore <YYYYMMDD-hhmmss> (or -r) option is available. For example:
llmux --restore 20230728-162302 restores the session from the $XDG_DATA_HOME/llmux/session-20230728-162302.json file. Then the chat goes on from that point.
It is also possible to use the special value last:
llmux --restore last
In this case it restores the last chat session, without specifying the timestamp.
Note that, if --restore is set, it overwrites any --context option.