Available --:--local Start a project
← All articles

( · 5 min read )

Faster Whisper: how to transcribe audio locally, on CPU or GPU

A practical guide to faster-whisper for local Whisper transcription: model sizes, CPU vs GPU, int8 quantization, voice activity filtering and live transcription.

  • Speech to text
  • Local AI
  • Whisper

Most transcription services work the same way: you upload your audio, wait, and get text back. That's fine for a podcast. It's not fine for a client call, a medical consultation, an internal meeting or anything you'd rather not hand to a third party.

Running Whisper locally solves that. The audio never leaves your machine, there's no per-minute bill, and it works offline. The easiest way I know to do it well is faster-whisper. I used it to build LiveTranscribe, a small open-source tool that transcribes any app's audio live on my own computer.

Here's what faster-whisper is, how to choose a setup, and what to watch out for.

What faster-whisper is

Whisper is OpenAI's open speech-recognition model. faster-whisper is a reimplementation of it on top of CTranslate2, an inference engine for Transformer models. Same model weights, different engine underneath.

The project's README says it is "up to 4 times faster than openai/whisper for the same accuracy while using less memory."

Three features make it practical:

  • Quantization. You can run the model in 8-bit (int8) on both CPU and GPU, or in float16 on GPU. Smaller numbers mean less memory and faster inference, usually with little loss in accuracy.
  • Voice activity detection. With vad_filter=True, it uses the Silero VAD model to skip stretches without speech. That saves time and cuts down on the odd phrases Whisper sometimes invents during silence.
  • Timestamps. You get segments with start and end times, and word-level timestamps if you ask for them.

A minimal local Whisper example

Install it with pip install faster-whisper. Then:

from faster_whisper import WhisperModel

model = WhisperModel("small", device="cpu", compute_type="int8")

segments, info = model.transcribe("meeting.wav", vad_filter=True)
print("Detected language:", info.language)

for segment in segments:
    print(f"[{segment.start:.1f}s -> {segment.end:.1f}s] {segment.text}")

One detail that surprises people: segments is a generator. The transcription only actually runs as you loop over it. If you want the full result in memory, wrap it in list().

To run on an NVIDIA GPU, change the first line to device="cuda" and compute_type="float16" (or int8_float16 to save memory). The current version of CTranslate2 expects CUDA 12 and cuDNN 9. If your machine has older drivers, check the README for the pinned versions that still work.

Choosing a model size

Whisper comes in several sizes, from tiny to large. The trade-off is simple: bigger models are more accurate, especially on accents, noisy audio and less common languages, but slower and hungrier for memory.

A rough guide:

  • tiny and base: quick tests, or very clean audio on a weak machine.
  • small: a sensible default on CPU. Good enough for clear English and French speech, and fast enough to keep up with live audio on a modern laptop.
  • medium and large: when accuracy matters more than speed, for noisy recordings, strong accents or Arabic. On a decent GPU they run comfortably.

There are also English-only variants (small.en and so on) that do a little better on English if that's all you need.

The honest answer is to test. Take ten minutes of your real audio, run two or three sizes, and read the output. Benchmarks on clean datasets won't tell you how it handles your meeting room or your customers' phone lines.

CPU or GPU?

You don't need a GPU to transcribe locally. With int8 and a small model, a recent CPU handles a lot. A GPU becomes worth it when:

  • you want the larger models at reasonable speed,
  • you're processing hours of recordings in batch,
  • you're transcribing live and need low latency with a bigger model.

On a single consumer GPU you can comfortably run the large models. That's the same principle I apply to other local AI work: a well-chosen model on hardware you own often beats paying per request.

Live transcription: chunks and latency

Whisper isn't a streaming model. It transcribes a window of audio at a time. To make it feel live, you feed it short chunks and print text as each one comes back.

That creates a few trade-offs:

  • Chunk length. Short chunks mean low latency but less context, so words at the edges get cut or misheard. Longer chunks are more accurate but feel laggy. A few seconds is a reasonable middle ground.
  • Silence. Running VAD on each chunk avoids wasting time on silence and reduces invented text.
  • Sample rate. Whisper works on 16 kHz mono audio, so resample before you transcribe.
  • Keeping up. If one chunk takes longer to transcribe than it lasts, you fall behind. That's the moment to pick a smaller model or move to a GPU.

How LiveTranscribe uses it

LiveTranscribe is a command-line tool I wrote that captures the audio of one specific application, like a browser tab, a video call or a media player, and transcribes it live in the terminal.

It works like this:

  1. A PipeWire virtual sink splits the app's audio. One branch goes back to your speakers, so you keep hearing it normally. The other branch feeds the transcriber.
  2. faster-whisper transcribes 3-second, 16 kHz chunks with voice activity filtering and word-level timestamps. You choose the model size at start.
  3. Words stream to the terminal as they're recognized, grouped into paragraphs by silence, and the transcript is saved with timestamps.
  4. When you exit, the original audio routing is restored.

Nothing leaves the machine at any point.

Languages, Arabic and dialects

Whisper is multilingual and detects the language automatically, or you can set it with language="fr" or language="ar". Setting it yourself is more reliable when the first seconds are silence or music.

French and English work well. Modern Standard Arabic works reasonably. Dialects like Moroccan Darija, and conversations that switch between Arabic and French mid-sentence, are harder: expect more errors, and use a larger model. If dialects matter for your use case, test on real recordings before you commit. I wrote more about this in building Arabic AI that understands dialects.

Why local transcription matters

Audio is some of the most sensitive data a company has: voices, names, health details, negotiations. Transcribing it locally means you don't need to decide whether a cloud provider's terms are acceptable, or whether sending it abroad is allowed under your data protection law. I go into those questions in private AI and on-premise LLMs.

Transcription is also often the first step of a bigger pipeline: summarise the meeting, extract action items, feed a voice note into a WhatsApp assistant. If that first step runs locally, the rest can too.

Working together

I'm Driss Briksine, a software and ML engineer based in Morocco. I build local and multimodal AI systems end to end, from speech and vision models to the software around them. See LiveTranscribe and my other work.

If you need transcription, or a voice pipeline, that keeps audio on your own hardware, send me a short brief.