bot-banter

This is a PoC demonstrating how two bots can autonomously "speak" to each other using an LLM and TTS. It uses NATS jetstream for message routing, ollama for generating text using an LLM of the user's choice and playht API for TTS speech synthesis.

Important

This project was built purely for educational purposes and thus is likely ridden with bugs, inefficiencies, etc. You should consider this project as highly experimental.

Click to watch/listen a sample conversation:

Bot Conversation Flow

sequenceDiagram
    participant GoTTS as TTS
    participant GoLLM as LLM
    participant Gobot
    participant Rustbot
    participant RustLLM as LLM
    participant RustTTS as TTS
    Gobot->>+Rustbot: Hi Rustbot!
    Rustbot->>RustLLM: Hi Rustbot!
    RustLLM->>RustTTS: Hi Gobot!
    RustLLM->>Rustbot: Hi Gobot!
    Rustbot->>-Gobot: Hi Gobot!
    activate Gobot
    Gobot->>GoLLM: Hi Gobot!
    GoLLM->>GoTTS: Teach me about Rust!
    GoLLM->>Gobot: Teach me about Rust!
    Gobot->>-Rustbot: Teach me about Rust!

Loading

Architecture

Zoomed in view on the high-level architecture:

flowchart TB
    subgraph " "
        playht(PlayHT API)
        ollama(Ollama)
    end
    bot <-->ollama
    bot <-->playht
    bot <--> NATS[[NATS JetStream]]

Loading

Tasks, Goroutines, Channels

Note

Mermaid does not have proper support for controlling layout or even basic graph legends There are some terrible workarounds, so I've opted not to use them in this README, hence the diagram might feel a bit unwieldy

flowchart TB
    ollama{Ollama}
    playht{PlayHT}
    llm((llm))
    tts((tts))
    jetWriter((jetWriter))
    jetReader((jetReader))
    ttsChunks(ttsChunks)
    jetChunks(jetChunks)
    prompts(prompts)
    ttsDone(ttsDone)
    subgraph NATS JetStream
        Go(go)
        Rust(rust)
    end
    Go-- 1. -->jetReader
    jetWriter-- 7. -->Rust
    jetReader-- 2. -->prompts
    prompts-- 3. -->llm
    llm-->ollama
    llm-- 4. -->ttsChunks
    llm-- 4. -->jetChunks
    jetChunks-->jetWriter
    ttsChunks-->tts
    tts-- 5. -->playht
    tts-- 6. -->ttsDone
    ttsDone-->jetWriter

Loading

1. jet.Reader receives a message published on a JetStream subject
2. jet.Reader sends this message to the prompts channel
3. llm worker reads the messages sent to the prompts channel and forwards them to ollama for LLM generation
4. ollama generates the response and the llm worker sends it to both ttsChunks and jetChunks channels
5. tts worker reads the message and sends the message to PlayHT API and streams the audio to the default audio device;
6. once the playback has finished tts worker notifies jet.Writer via the ttsDone channel that it's done playing audio
7. jet.Writer receives the notification on the ttsDone channel and publishes the message it received on jetChunks channel to a JetStream subject

HOWTO

There are a few prerequisites:

• nats
• ollama
• Create an account on playht
• sound/audio libraries on some platforms e.g. Linux

Run NATS

Both bots use nats as their communication channel.

Homebrew

Install

brew tap nats-io/nats-tools
brew install nats nats-server

Run:

nats-server -js

Nix

nix-shell -p nats-server natscli
nats-server -js

Run Ollama

Download it from the official site or see the Nix install below.

Nix

nix-shell -p ollama

Run a model you decide to use

ollama run llama2

Audio libraries

If you are running on Linux you need to install the following libraries -- assuming you want to play with the bot-speaking service

Note

This is for Ubuntu Linux, other distros have likely different package names

sudo apt install -y --no-install-recommends libasound2-dev pkg-config

playht API credentials

Once you've created an account on playht you need to generate API keys. See here for more details.

Now, you need to export them via the following environment variables which are read by the client libraries we use (go-playht, playht_rs):

export PLAYHT_SECRET_KEY=XXXX
export PLAYHT_USER_ID=XXX

Run the bots

Important

Once you've started gobot you need to prompt it. gobot reads prompt from stdin which kickstarts the conversation: rusbot waits for gobot before it responds!

Start the gobot:

go run ./gobot/...

Start the rustbot:

cargo run --manifest-path rustbot/Cargo.toml