Modify the command.cpp to change the Activation Phrase/prompt

[normatovjj]

Since the command "Ok Whisper, start listening for commands." is hard coded, here is a modification to the command.cpp to change the prompt to whatever you want. Instead of the line 565 below:

565 std::string k_prompt = "Ok Whisper, start listening for commands.";

we change it to:

std::string k_prompt;

std::ifstream configFile("./examples/command/phrase.txt");
if (configFile.is_open()) {
    std::getline(configFile, k_prompt);
    configFile.close();
}

if (k_prompt.empty()) {
    k_prompt = "Hey Whisper";  // Default prompt if config.txt is empty or not found
}

if (!params.prompt.empty()) {
    k_prompt = params.prompt;
}

You would need to create a new file "phrase.txt" in the "whisper.cpp/examples/command/" folder and write whatever you want, simple text without "". Also, if you already have the command compiled, you would need to delete the executable from the main folder whisper.cpp (just one file), and recompile again, by simply running make command from the main directory.

Also, would be great to reset the assistant to listening for the activation phrase after giving it a command, like regular voice assistants do. After one command is identified, it would be ideal to again activate the assistant by saying the activation phrase.

[normatovjj]

Update: in order to achieve the second idea from above, we can change the lines 574-680 while (is_running) {... to the following (also pay attention to correct indentation and remember to compile again after changing command.cpp file):

while (is_running) {
        // handle Ctrl + C
        is_running = sdl_poll_events();

        // delay
        std::this_thread::sleep_for(std::chrono::milliseconds(100));

        if (ask_prompt) {
            fprintf(stdout, "\n");
            fprintf(stdout, "%s: Say the following phrase: '%s%s%s'\n", __func__, "\033[1m", k_prompt.c_str(), "\033[0m");
            fprintf(stdout, "\n");

            ask_prompt = false;
        }

        {
            audio.get(2000, pcmf32_cur);

            if (::vad_simple(pcmf32_cur, WHISPER_SAMPLE_RATE, 1000, params.vad_thold, params.freq_thold, params.print_energy)) {
                fprintf(stdout, "%s: Speech detected! Processing ...\n", __func__);

                int64_t t_ms = 0;

                if (!have_prompt) {
                    // wait for activation phrase
                    audio.get(params.prompt_ms, pcmf32_cur);

                    const auto txt = ::trim(::transcribe(ctx, params, pcmf32_cur, "prompt", logprob_min0, logprob_sum0, n_tokens0, t_ms));

                    const float p = 100.0f * std::exp(logprob_min0);

                    fprintf(stdout, "%s: Heard '%s%s%s', (t = %d ms, p = %.2f%%)\n", __func__, "\033[1m", txt.c_str(), "\033[0m", (int) t_ms, p);

                    const float sim = similarity(txt, k_prompt);

                    if (txt.length() < 0.8*k_prompt.length() || txt.length() > 1.2*k_prompt.length() || sim < 0.8f) {
                        fprintf(stdout, "%s: WARNING: prompt not recognized, try again\n", __func__);
                        ask_prompt = true;
                    } else {
                        fprintf(stdout, "\n");
                        fprintf(stdout, "%s: The prompt has been recognized!\n", __func__);
                        fprintf(stdout, "%s: Waiting for voice commands ...\n", __func__);
                        fprintf(stdout, "\n");

                        // save the audio for the prompt
                        pcmf32_prompt = pcmf32_cur;
                        have_prompt = true;
                    }
                } else {
                    // we have heard the activation phrase, now detect the commands
                    audio.get(params.command_ms, pcmf32_cur);

                    // prepend 3 second of silence
                    pcmf32_cur.insert(pcmf32_cur.begin(), 3.0f*WHISPER_SAMPLE_RATE, 0.0f);

                    // prepend the prompt audio
                    pcmf32_cur.insert(pcmf32_cur.begin(), pcmf32_prompt.begin(), pcmf32_prompt.end());

                    const auto txt = ::trim(::transcribe(ctx, params, pcmf32_cur, "root", logprob_min, logprob_sum, n_tokens, t_ms));

                    const float p = 100.0f * std::exp(logprob_min);

                    // find the prompt in the text
                    float best_sim = 0.0f;
                    size_t best_len = 0;
                    for (size_t n = 0.8*k_prompt.size(); n <= 1.2*k_prompt.size(); ++n) {
                        if (n >= txt.size()) {
                            break;
                        }

                        const auto prompt = txt.substr(0, n);

                        const float sim = similarity(prompt, k_prompt);

                        if (sim > best_sim) {
                            best_sim = sim;
                            best_len = n;
                        }
                    }

                    fprintf(stdout, "%s:   DEBUG: txt = '%s', prob = %.2f%%\n", __func__, txt.c_str(), p);
                    if (best_len == 0) {
                        fprintf(stdout, "%s: WARNING: command not recognized, try again\n", __func__);
                    } else {
                        // cut the prompt from the decoded text
                        const std::string command = ::trim(txt.substr(best_len));

                        fprintf(stdout, "%s: Command '%s%s%s', (t = %d ms)\n", __func__, "\033[1m", command.c_str(), "\033[0m", (int) t_ms);

                        // reset the loop to wait for the activation phrase again
                        have_prompt = false;
                        ask_prompt = true;
                    }

                    fprintf(stdout, "\n");
                }

                audio.clear();
            }
        }
    }

    return 0;

[ulatekh]

Why add an extra file? Why not just make it a command-line parameter?

[normatovjj]

Sure, there you go. I found a parameter in the og file called "required activation prompt" but changing it in the command line did not change anything. Now you can set your own activation phrase by using --phrase or -ap followed by a string ("Hey, Ulatekh"). If that parameter is not specified in the command line, then it defaults to "Hey, Whisper". This version also comes with listening to only one command after activation phrase, then reseting to wait for the activation phrase, like explained in the post above.

Here is the command.cpp with necessary changes (cannot upload .cpp here, so just change the extension to .cpp and you should be good). Don't forget to compile the program after replacing command.cpp in the ./example/command/ folder
command.txt