[Bug]: ollama not wroking

[kittenvr]

Is there an existing issue for the same bug?

• I have checked the troubleshooting document at https://docs.all-hands.dev/modules/usage/troubleshooting
• I have checked the existing issues.

Describe the bug

when i start openhands with ollama it just doesnt work

Current OpenHands version

ghcr.io/all-hands-ai/openhands

Installation and Configuration

ollama serve

# The directory you want OpenHands to modify. MUST be an absolute path!
export WORKSPACE_BASE=$(pwd)/workspace

docker run \
    -it \
    --pull=always \
    --add-host host.docker.internal:host-gateway \
    -e SANDBOX_USER_ID=$(id -u) \
    -e LLM_API_KEY="ollama" \
    -e LLM_BASE_URL="http://host.docker.internal:11434" \
    -e LLM_OLLAMA_BASE_URL="http://host.docker.internal:11434" \
    -e WORKSPACE_MOUNT_PATH=$WORKSPACE_BASE \
    -v $WORKSPACE_BASE:/opt/workspace_base \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    ghcr.io/all-hands-ai/openhands:main

Model and Agent

model: deepseek-coder-v2:latest

Operating System

linux chachyos

Reproduction Steps

i dont know how to reproduce on other installs

Logs, Errors, Screenshots, and Additional Context

https://discord.com/channels/1222935860639563850/1281641914210455643

[enyst]

Can please you try this: open the app on localhost:3000, and enter in Settings the model, API key, and base URL? Does it work to see the openhands interface and enter these settings?

[kittenvr]

Can please you try this: open the app on localhost:3000, and enter in Settings the model, API key, and base URL? Does it work to see the openhands interface and enter these settings?

https://www.logpasta.com/paste/99927887-c724-425d-9ae5-07580bc64e13

[enyst]

I think the docker problem is the main issue there, but first, please also try to simplify the command a bit, and pull the 0.9 tag instead of main.

export WORKSPACE_BASE=$(pwd)/workspace

docker run \
    -it \
    --pull=always \
    --add-host host.docker.internal:host-gateway \
    -e SANDBOX_USER_ID=$(id -u) \
    -e LLM_OLLAMA_BASE_URL="http://host.docker.internal:11434" \
    -e WORKSPACE_MOUNT_PATH=$WORKSPACE_BASE \
    -v $WORKSPACE_BASE:/opt/workspace_base \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    ghcr.io/all-hands-ai/openhands:0.9

About the docker issue: can you try to check if docker networking is working? Could start by restarting Docker and checking if it's up to date. That error doesn't seem related to openhands, but docker installation or configuration, I would say.

[ParkerrDev]

It works for me when I add --network host \ on 0.12

Commands I used to get it working:

ollama serve

docker pull docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik

sudo docker run -it --privileged --rm --pull=always \
    --network host \
    -e LLM_API_KEY="ollama" \
    -e LLM_BASE_URL="http://127.0.0.1:11434" \
    -e LLM_OLLAMA_BASE_URL="http://127.0.0.1:11434" \
    -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    --add-host host.docker.internal:host-gateway \
    --name openhands-app \
    docker.all-hands.dev/all-hands-ai/openhands:0.12

[kittenvr]

It works for me when I add --network host \ on 0.12

Commands I used to get it working:

ollama serve

docker pull docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik

sudo docker run -it --privileged --rm --pull=always \
    --network host \
    -e LLM_API_KEY="ollama" \
    -e LLM_BASE_URL="http://127.0.0.1:11434" \
    -e LLM_OLLAMA_BASE_URL="http://127.0.0.1:11434" \
    -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    --add-host host.docker.internal:host-gateway \
    --name openhands-app \
    docker.all-hands.dev/all-hands-ai/openhands:0.12

thanks but im not using ollama anymore, could you check this one out tho #4586 (comment)

[jimmyzhao-57blocks]

It works for me when I add --network host \ on 0.12

Commands I used to get it working:

ollama serve

docker pull docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik

sudo docker run -it --privileged --rm --pull=always \
    --network host \
    -e LLM_API_KEY="ollama" \
    -e LLM_BASE_URL="http://127.0.0.1:11434" \
    -e LLM_OLLAMA_BASE_URL="http://127.0.0.1:11434" \
    -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    --add-host host.docker.internal:host-gateway \
    --name openhands-app \
    docker.all-hands.dev/all-hands-ai/openhands:0.12

when I add --network host, I can't open the localhost:3000 in the browser.

[BridgerB]

It works for me when I add --network host \ on 0.12

Commands I used to get it working:

ollama serve

docker pull docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik

sudo docker run -it --privileged --rm --pull=always \
    --network host \
    -e LLM_API_KEY="ollama" \
    -e LLM_BASE_URL="http://127.0.0.1:11434" \
    -e LLM_OLLAMA_BASE_URL="http://127.0.0.1:11434" \
    -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    --add-host host.docker.internal:host-gateway \
    --name openhands-app \
    docker.all-hands.dev/all-hands-ai/openhands:0.12

This is not working for me.
I am able to use

ollama run llama3.1

or

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1",
  "prompt": "2+2?",             
  "stream": false
}'

But when I try your command I get this:

INFO:     Started server process [10]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:3000 (Press CTRL+C to quit)
INFO:     127.0.0.1:55928 - "GET /api/options/models HTTP/1.1" 200 OK
03:41:36 - openhands:ERROR: listen.py:311 - Expected 3 websocket protocols, got 1
INFO:     ('127.0.0.1', 55932) - "WebSocket /ws" 403
INFO:     connection rejected (403 Forbidden)
INFO:     connection closed
INFO:     127.0.0.1:55928 - "GET /api/options/agents HTTP/1.1" 200 OK
INFO:     127.0.0.1:55928 - "GET /api/options/security-analyzers HTTP/1.1" 200 OK
03:41:39 - openhands:ERROR: listen.py:311 - Expected 3 websocket protocols, got 1
INFO:     ('127.0.0.1', 55940) - "WebSocket /ws" 403
INFO:     connection rejected (403 Forbidden)
INFO:     connection closed