May whisper.cpp help ?

[kobenaxie]

whisper.cpp may offer an efficient way for inference ?

[yihong0618]

yes, but not so simple for now.
have dig it.
for now, it does not have python binding.
if we want to use it, we need build both it and its python binding.

[maltoze]

it can be used by calling a dll, but there is no performance improvement when I tried it(cpu: E5-2640).

whisper_ctx = whisper_cpp.whisper_init(model_path)

params = whisper_cpp.whisper_full_default_params(0)
params.n_threads = 8
params.language = whisper_cpp.String(b"zh")

audio = load_audio(file_path)
rc = whisper_cpp.whisper_full(
    whisper_ctx,
    params,
    audio.ctypes.data_as(ctypes.POINTER(ctypes.c_float)),
    len(audio),
)
if rc != 0:
    raise Exception(f"Error from whisper.cpp: {rc}")

n_segments = whisper_cpp.whisper_full_n_segments(whisper_ctx)
result = []
for i in range(n_segments):
    t0 = whisper_cpp.whisper_full_get_segment_t0(whisper_ctx, i)
    t1 = whisper_cpp.whisper_full_get_segment_t1(whisper_ctx, i)
    txt = whisper_cpp.whisper_full_get_segment_text(whisper_ctx, i)
    result.append({"start": t0 * 10, "end": t1 * 10, "text": txt.decode()})
whisper_cpp.whisper_free(whisper_ctx)

[momobobe]

yes, but not so simple for now. have dig it. for now, it does not have python binding. if we want to use it, we need build both it and its python binding.

https://github.com/abdeladim-s/pywhispercpp has done it