Skip to main content
ZICQ

Wiki Models & Products

Whisper

Models & Products
Aliases: Whisper OpenAI Whisper ASR ·2026-09-14

Whisper

Whisper is OpenAI's open-source automatic speech recognition (ASR) model released in 2022. With its multilingual robustness and open-source availability, it quickly became the industry ASR de facto standard.

Model specifications

Size Params VRAM Relative speed
tiny 39M ~1 GB ~32x
base 74M ~1 GB ~16x
small 244M ~2 GB ~6x
medium 769M ~5 GB ~2x
large-v3 1.5B ~10 GB 1x

Core capabilities

  • Multilingual recognition: 99 languages, WER near human level on major languages.
  • Translation: directly translate foreign speech to English text.
  • Timestamps: word-level / segment-level timestamp output.
  • Long audio: chunked processing supports arbitrary length.
  • Robustness: much better than traditional Kaldi systems on accents, noise, phone audio.

One-line startup

pip install openai-whisper
whisper audio.mp3 --language en --model medium

Chinese experience

  • Chinese recognition accuracy is very high (medium+ is good enough).
  • Numbers / proper nouns occasionally wrong, needs post-processing correction.
  • Speaker diarization needs additional tools (pyannote.audio).

Deployment

  • Local CLI: the whisper command above, simplest.
  • Python library: whisper.load_model("large-v3"), embed in apps.
  • whisper.cpp: C++ rewrite, runs on CPU / Apple Silicon / Android.
  • faster-whisper: CTranslate2 backend, 4-5x faster, half the memory.
  • whisper-jax: JAX backend, TPU acceleration.

Limitations

  • No built-in speaker diarization.
  • Real-time streaming needs custom wrapper (whisper_streaming etc).
  • Long silence / song change scenarios may produce hallucinated text.