Whisper is an open-source speech recognition model released by OpenAI in September 2022. It runs on your own machine, so recordings never leave your computer — useful for interviews, meetings and anything confidential.

What you need

  • Python 3.9 or later
  • ffmpeg, the audio/video toolkit Whisper uses to read files
  • A GPU is optional; small models run fine on a CPU, just more slowly

Step 1 — Install

pip install -U openai-whisper

Install ffmpeg with your system’s package manager (for example brew install ffmpeg on macOS or winget install ffmpeg on Windows).

Step 2 — Transcribe a file

whisper interview.mp3 --model small --language en

Whisper writes the transcript in several formats, including plain text and SRT/VTT subtitle files with timestamps.

Step 3 — Choose a model size

Whisper comes in sizes from tiny to large. Smaller models are faster and need less memory; larger ones are more accurate, especially with accents, noise and technical vocabulary. A practical approach: test small first, and move up only if the errors bother you.

Step 4 — Translate to English

whisper lecture-it.mp3 --model medium --task translate

The translate task outputs English text from speech in other supported languages.

Tips for better results

  • Record close to the microphone and reduce background noise.
  • Set --language when you know it; auto-detection can fail on short clips.
  • Proofread names and numbers: speech recognition still makes mistakes, and occasionally produces text that was never spoken during long silences.

Going further

Faster community implementations of Whisper exist for laptops and servers, and many note-taking apps use Whisper-family models under the hood.

Common problems and fixes

  • It’s very slow. You’re probably running on the CPU. Use a smaller model, or install a GPU-enabled version of PyTorch if you have a compatible graphics card. Optimized community implementations can also be much faster on ordinary hardware.
  • Out of memory. Choose a smaller model size; the largest models need several gigabytes of memory.
  • Wrong language detected. Specify the language explicitly instead of relying on automatic detection, especially for short clips.
  • Repeated or invented phrases. Whisper can “hallucinate” text during long silences or music. Trim silent sections, split long files, or use voice-activity detection before transcription.
  • Names and jargon spelled wrong. Provide an initial prompt containing the correct spellings of names and technical terms, and correct the transcript afterwards.

Privacy advantages of running locally

Running Whisper on your own computer means the audio never leaves your device. That matters for interviews, medical or legal recordings, internal meetings and anything confidential. You still need consent to record people, and you should store transcripts securely, but you avoid sending sensitive audio to a third-party service.

Example workflow for interviews

  1. Record with a good microphone close to speakers, in a quiet room.
  2. Convert the file to a common format such as WAV or MP3 if needed.
  3. Run Whisper with the language specified and a prompt listing names and terms.
  4. Export subtitles (SRT or VTT) if you need timestamps, or plain text for notes.
  5. Review the transcript while listening at higher speed, correcting errors.
  6. Store the audio and transcript securely, and delete them when no longer needed.

Accuracy tips by situation

  • Phone calls: use the larger models, as audio quality is lower.
  • Lectures: provide a prompt with the subject’s technical terms.
  • Multiple languages: transcribe each segment with the right language setting.
  • Noisy recordings: apply noise reduction first.
  • Strong accents: larger models usually handle accents and dialects more accurately.

What to watch next

  • Faster and smaller speech recognition models that run on laptops and phones.
  • Better speaker identification (diarization) in open-source tools.
  • Real-time transcription in local apps.
  • Accuracy improvements for low-resource languages.

Key takeaways

  • Whisper is an open-source speech recognition model you can run on your own computer.
  • Choose a model size that balances speed and accuracy for your hardware.
  • Specify the language and provide spelling hints for names and jargon.
  • Watch for invented text during silences, and review important transcripts.
  • Local transcription keeps sensitive audio on your device.

Frequently asked questions

Is Whisper free?

Yes. OpenAI released Whisper as open-source software with open model weights, so you can run it locally at no cost beyond your hardware.

Do I need a GPU?

No, but a compatible GPU makes transcription much faster, especially with larger models.

Can Whisper identify different speakers?

Whisper itself transcribes speech without labelling speakers; additional tools can add speaker identification.