Localize product videos into 8 languages with WhisperX word-level timing
Turn a single Demo or tutorial recording into natural English, Japanese, Spanish, and German dubbed videos with word-accurate SRT subtitles, locked technical glossaries, and FFmpeg audio ducking.
Why default setups fall short
Standard subtitle translators fail on technical product demos because translated sentences in Spanish or German often run 25% longer than the original speech window, causing dubbed voiceovers to desynchronize from screen clicks. By combining WhisperX phoneme-level timestamps with a syllable-aware LLM translation prompt and FFmpeg sidechain ducking, indie founders localize a 3-minute product demo into 8 languages in under 10 minutes.
Prerequisites before starting
- Python 3.10+ with whisperx and ffmpeg installed
- OpenAI / Anthropic API key for syllable-constrained translation
- ElevenLabs API key or free Edge-TTS for multi-language voice synthesis
Step-by-step implementation
1. Extract word-level timestamps with WhisperX forced alignment
Unlike standard Whisper (which drifts by up to 1.5 seconds on pauses), WhisperX runs a wav2vec2 phoneme alignment pass so every subtitle segment has millisecond-accurate start and end boundaries.
whisperx input-demo.mp4 --model large-v3 --align_model WAV2VEC2_ASR_LARGE_LV60K_960H \ --output_format json --output_dir ./aligned/
2. Translate with a glossary lock and character-budget constraint
Feed each timed segment into your LLM along with a JSON glossary of brand/UI terms that must remain untranslated (such as API names, CLI flags, and menu labels) and enforce a maximum character budget based on segment duration.
max_chars = int(segment_duration_sec * 14.5) # Keep speaking rate <= 14.5 chars/sec
3. Synthesize per-segment TTS and time-stretch within a 1.08x ceiling
Generate audio per subtitle segment and use FFmpeg atempo (capped between 0.95x and 1.08x) so the dubbed phrase lands on the exact UI click without sounding sped up.
ffmpeg -i seg_04.mp3 -filter:a "atempo=1.05" -vn seg_04_synced.wav
4. Mix with sidechain audio ducking and burn clean styling into MP4
Lower the original track to -18dB during active speech so keyboard clicks and UI sound effects remain audible underneath the localized voiceover.