How Myna Player Generates Context-Aware Subtitles Ahead of Playback
This is the final article in my product-engineering series. The earlier posts covered Toolboard, isolated Codex Desktop profiles, CodexPigeon, and CodexControl. This one is about the newest and most technically ambitious project: Myna Player.
AI subtitles create a timing problem.
Translate every tiny fragment immediately and the text may arrive on time, but important context disappears. Wait for a longer passage and the translation becomes more coherent, but the subtitle may appear after the dialogue has already passed.
That creates a practical tradeoff:
speed or context.
I wanted to see whether a media player could avoid making that choice at playback time.
Myna Player is an open-source, local-first AI video player that processes audio ahead of the current playback position, transcribes it locally, translates finalized subtitle cues with surrounding context, and renders them against the native player clock.
The project is currently a public alpha. The core macOS application works, Windows code and packaging are under active verification, and the first stable release has not been published yet.
The product boundary
Myna Player is not a transcription editor with a video preview attached.
It is a media player with a just-in-time subtitle pipeline.
That distinction shapes the architecture. The system must keep several things true at once:
- playback remains responsive;
- transcription stays ahead of the viewer;
- seeking does not corrupt previous work;
- completed processing can resume later;
- translation receives enough surrounding context;
- translated cues keep the source timing;
- local media and audio remain on the device;
- the subtitle renderer never waits for an AI request.
The simplified pipeline is:
native player clock
|
v
persistent priority scheduler
|
v
FFmpeg audio windows
|
v
local Whisper worker + VAD
|
v
timed source subtitle cues
|
+---------------------+
| |
v v
local source cache optional translation workers
| |
+----------+----------+
|
v
synchronized subtitle renderer
The expensive work happens outside the playback loop.
Why direct media access changes the problem
A live-caption application usually listens to audio while it is being played. That is necessary when the source is unknown or truly live.
Myna Player works with local media files, so it can read the source directly.
It does not have to wait for sound to leave the speakers. It can extract audio from later parts of the file and prepare subtitles before playback reaches them.
Direct file access gives the pipeline:
- deterministic time ranges;
- repeatable extraction;
- precise audio-stream selection;
- the ability to jump ahead;
- resumable cached work;
- no dependency on system-audio capture;
- no contamination from notification sounds or other applications.
The media file itself becomes the source of truth.
Canonical processing windows
The pipeline schedules deterministic, non-overlapping audio windows. The current design uses canonical 30-second ranges:
00:00–00:30
00:30–01:00
01:00–01:30
01:30–02:00
...
Stable windows give the cache stable identities.
When the same video is reopened, the app can determine which regions were completed. When the user seeks, finished windows do not need to be processed again.
The scheduler assigns priority according to the current playback position:
- the first missing window is urgent;
- roughly the next 90 seconds are promoted;
- later windows remain background work.
The goal is not simply to transcribe the whole file as fast as possible.
It is:
Keep the region the viewer is about to reach prepared, while using spare capacity to extend the cache farther into the video.
Media fingerprints and pipeline keys
A file path is not a safe cache key.
The file may change. The selected audio stream may change. The speech model, language mode, VAD configuration, extraction settings, or cue-segmentation rules may change.
Myna Player creates a media fingerprint and combines it with a pipeline key containing assumptions such as:
- selected audio stream;
- Whisper runtime and model hash;
- transcription language mode;
- VAD model;
- chunk and extraction parameters;
- cue-segmentation version.
Only checkpoints with an exact matching key are restored.
This prevents old subtitle data from being reused after the processing assumptions changed.
A cache should not merely remember that work happened. It should remember which work happened under which configuration.
Seeking needs generations, not only cancellation
Seeking creates one of the more subtle concurrency problems.
Imagine the viewer jumps from minute 2 to minute 48 while workers are still processing minute 3.
Cancelling a task is not always immediate. A native process or model worker may finish after the seek and attempt to write a result that is no longer relevant.
Myna Player uses processing generations.
When the playback context changes significantly, the scheduler advances the generation. Jobs created under the previous generation may finish, but their outputs are rejected if they no longer match the active generation and pipeline state.
This is safer than assuming cancellation always stops every external operation instantly.
The broader principle is useful beyond media applications:
When asynchronous work can outlive the state that requested it, validate the result against the current generation before committing it.
Local transcription with word timestamps
For each audio window, FFmpeg extracts mono 16 kHz PCM audio. Optional voice activity detection reduces unnecessary speech-recognition work.
A long-lived whisper.cpp worker performs local transcription and returns word-level timing.
Word timestamps are important because raw transcription segments are not automatically good subtitles. The application needs to derive readable cues while preserving alignment with speech.
Cue construction considers factors such as:
- pauses and word timing;
- maximum duration;
- readable line length;
- punctuation and sentence boundaries;
- overlap prevention;
- continuity across adjacent processing windows.
The result is a source subtitle timeline that can be displayed immediately, exported, edited, or passed to a translator.
Translation may use context, but not rewrite time
Translation needs more context than a single short cue often provides.
Pronouns, tone, names, and sentence structure may depend on the lines before and after the current subtitle.
Myna Player therefore sends finalized source cues together with limited neighboring text context.
But it enforces a strict contract:
- the translator may change subtitle text;
- it may use surrounding context to understand meaning;
- it may not change cue IDs;
- it may not change start or end timestamps;
- it may not merge or reorder cues silently.
A conceptual request looks like this:
{
"context_before": ["..."],
"cues": [
{
"id": "cue_0042",
"start_ms": 184000,
"end_ms": 187200,
"text": "Source text"
}
],
"context_after": ["..."]
}
The response is validated against the source cue set. Missing IDs, extra IDs, duplicates, or timing changes invalidate the translation result.
This separation is essential:
- transcription owns when something was said;
- cue segmentation owns how speech is divided;
- translation owns how the text is expressed in another language.
A language model should not gain authority over the timeline just because it is producing the translated text.
Separate source and translation workers
Speech recognition and translation have different resource profiles.
Transcription is local, CPU-heavy, and tied to extracted audio. Translation may be remote, rate-limited, provider-specific, and retried independently.
Myna Player separates them into different worker paths.
That allows source subtitles to appear without waiting for translation. It also means a translation-provider failure does not invalidate completed transcription work.
The current provider layer supports adapters for services including DeepL, Gemini, MiniMax, OpenAI, and OpenRouter.
Provider credentials are stored through the operating system credential store.
Persistent checkpoints with SQLite and WAL
The subtitle pipeline may run for a long time, and users may close the application halfway through a video.
Myna Player stores local state in SQLite with write-ahead logging. It can persist:
- media fingerprints;
- processing checkpoints;
- source cues;
- provider-specific translations;
- playback position;
- subtitle corrections;
- model metadata;
- pipeline state.
This makes the pipeline resumable instead of treating every application launch as a fresh transcription session.
Per-video reset removes that video's transcript, translations, checkpoints, cache record, and remembered playback position without deleting the original media file or global settings.
The renderer follows the native player clock
Subtitle timing is driven by the native playback clock rather than a separate web timer.
The player surface uses libVLC on macOS, with a Windows child-window implementation under active verification.
The application supports playback controls such as play, pause, seek, volume, speed, fullscreen, audio tracks, and embedded subtitle tracks.
Generated cues are rendered according to the native player position. They can be shown as source text, translated text, or both.
The renderer reads prepared cues from local state. It does not block playback while waiting for transcription or translation.
Local-first does not mean cloud-free
Local transcription is the default:
- FFmpeg runs locally;
- VAD runs locally;
- whisper.cpp runs locally;
- subtitle timing, editing, playback, export, and caching run locally.
Cloud translation is optional.
When enabled, the selected provider receives finalized transcript text and limited neighboring context. It does not receive the video file or extracted audio.
This keeps the largest and most sensitive media data on the user's device while still allowing users to choose a cloud translation provider when they need one.
Native runtime packaging is part of the product
A desktop AI application is not finished when it works on the developer's machine.
Myna Player depends on native components such as:
- VLC;
- FFmpeg and FFprobe;
- whisper.cpp;
- speech models;
- optional VAD models.
Packaging scripts download or build pinned components, verify checksums, stage licenses, and create native Tauri bundles.
The release process also has to account for architecture differences, signing, notarization, Windows installers, model integrity, and diagnostics.
For products that combine web UI, Rust, and native media runtimes, packaging is not a final administrative step. It is one of the core engineering layers.
What I learned
1. Reframing the timing problem creates new options
The system does not need to choose between context and latency at the exact moment dialogue is heard if it can process the source ahead of playback.
2. AI components need narrow authority
Translation benefits from context, but it should not control cue identity or timing.
3. Caches must encode assumptions
A completed result is reusable only when the media and pipeline configuration still match.
4. Cancellation alone is not enough
Generation validation protects the system from late asynchronous results after seeks and configuration changes.
5. Local-first architecture can still support user-chosen cloud services
The important question is not whether every operation is local. It is which data crosses the boundary, when, and under whose control.
6. Packaging is part of application architecture
Native dependencies, checksums, models, licenses, signing, and platform verification must be designed alongside the application itself.
Project links
- Website: https://myna-player.github.io/
- GitHub: https://github.com/ademisler/Myna-Player
- Status: public alpha
- License: GPL-3.0-or-later
Myna Player is open for review and contribution. I am especially interested in feedback about the look-ahead scheduling model, subtitle cue contract, local-first privacy boundary, and cross-platform packaging approach.
Comments
Join the discussionNo comments yet. Be the first to comment.
