VisemIQ
A visual speech recognition experiment that tries to turn speech into text from silent video, by looking only at lip movements.
- Question
- Can speech be read from video alone, without any audio?
- Status
- Prototype
- Started
- September 2026
- Tools
- Visual speech recognition
- Large language models
- Python
- Claude
The problem
Understanding what is being said in videos where the audio is missing or unusable. Lip reading is a limited and exhausting skill for people; I wanted to see how far AI can go with it.
What I built
The first version reads English speech from lip movements, with a Turkish interface and optional Turkish translation. I have also put in place the language infrastructure and automatic language detection for Turkish videos; Turkish lip reading itself is the next research phase.
Early observations
- Lip reading is a much harder problem than speech recognition from audio: the same lip shape (a viseme) can correspond to several different sounds. That is where the project’s name comes from.
- Passing the raw output through a language model to put it in context looks like the most promising way to make it readable.