Audio Transcription App

Audio Transcription App is a local-first speech-to-text pipeline built with Streamlit that combines two state-of-the-art models: OpenAI Whisper for transcription and pyannote.audio for speaker diarization. Users upload audio (up to 1GB, any FFmpeg-supported format), select a Whisper model tier, and get a speaker-labeled transcript exported as PDF or DOCX. All inference runs locally — no audio leaves the machine. The app handles large files via proportional FFmpeg segmentation with timestamp offsets so the diarization timeline stays aligned across segments.

Transcription services like Otter.ai and Rev send audio to cloud servers — unacceptable for confidential interviews, medical consultations, or legal recordings. Local Whisper deployments handle transcription but not speaker identification. pyannote.audio handles diarization but requires aligning its output with Whisper's timestamps. The challenge was integrating both models into a pipeline where large files don't exhaust memory and speaker labels stay aligned after segmented transcription.

Build a local-first transcription pipeline that: (1) runs entirely on-device with no cloud API calls; (2) handles audio files up to 1GB via segmentation; (3) assigns speaker labels to each Whisper segment using pyannote diarization; (4) exports labeled transcripts as PDF and DOCX; (5) provides a user-friendly Streamlit UI with bilingual support.

• ◈Solo developer — ML pipeline architecture, audio processing, UI, export formats

1. 01Proportional FFmpeg segmentation for large files — instead of fixed-duration chunks (which can cut mid-sentence), total_duration is divided by the number of required segments to produce balanced chunks. Each chunk is resampled to 16kHz mono WAV (Whisper's expected input) and its start offset tracked. After transcription, segment timestamps are shifted by chunk_start to restore global timeline position.
2. 02Midpoint matching for speaker assignment — for each Whisper segment, the speaker is determined by finding which diarization interval contains the segment's midpoint: (segment_start + segment_end) / 2. This handles the common case where a speaker changes mid-segment — the segment is assigned to whoever was speaking at the center, not the start.
3. 03fp16=False for transcription quality — Whisper supports half-precision (fp16) inference which is faster but reduces accuracy, especially for non-English speech and noisy audio. Deliberately using full 32-bit float precision prioritizes transcription quality over speed. Users who need speed select a smaller model tier instead.
4. 04pyannote/speaker-diarization-3.1 as gated model — pyannote's model requires Hugging Face account approval before download. This is intentional: it filters for serious use cases and reduces abuse. The app handles the HF token via environment variable; the gated access adds one setup step but ensures a production-grade diarization model.
5. 05Streamlit session_state for speaker renaming — diarization returns generic labels (SPEAKER_00, SPEAKER_01). Streamlit's session_state persists the user's custom name mappings (e.g., 'Interviewer', 'Candidate') across interactions without re-running the pipeline. The transcript re-renders with updated names immediately when the mapping changes.
6. 06Queue-based audio delivery for robustness — audio chunks are pushed to a Python queue between FFmpeg extraction and Whisper transcription. This decouples the extraction pace from the inference pace, allowing the pipeline to handle variable-length chunks without backpressure blocking FFmpeg.

• →Midpoint matching is more robust than start-time matching for speaker assignment — early versions assigned speaker by checking which diarization interval contained segment_start. This failed when a speaker change happened 0.1s into a segment. Using the midpoint reduced misattributions significantly.
• →Proportional segmentation requires tracking segment offsets explicitly — the first version concatenated Whisper outputs without offset correction. Diarization ran on the full file but transcription timestamps were relative to each chunk, causing speaker labels to be applied to the wrong sentences. Adding chunk_start offset to every segment timestamp fixed the alignment.
• →Local LLM inference hardware matters enormously — the large-v3 model that produces best-quality transcripts runs in real time on a GPU but takes 3× wall-clock time on CPU for a 30-minute file. Exposing model selection to users with clear performance guidelines (tiny for quick drafts, large-v3 for final output on GPU) was more practical than picking one tier.
• →Streamlit session_state is the right tool for multi-step ML workflows — the pipeline has 4 stages (upload, transcribe, diarize, label). Each stage is expensive. session_state caches each stage's output so editing speaker names doesn't re-run transcription. This pattern (stage → cache → next stage) is the correct Streamlit architecture for ML pipelines.

Functional — local desktop tool for private transcription workflows