Model deep-dive
Kling 3.0 Omni native lip sync, explained
"Lip sync" used to mean a second tool: you generated a video, then ran the mouth through a separate model to match an audio track. Kling 3.0 Omni, Kuaishou's omni-modal video model, does it in one pass instead — the voice and the mouth motion are generated together inside the model, at the same time as everything else in the shot. That sounds like a small distinction, but it changes how clean the result looks and how much fiddling you do afterward. Here's what native lip sync actually means, the Kling 3.0 Omni capabilities that make it useful, how to make a talking clip in Renoise, and what to use when a face has to follow your own recording.
What "native" lip sync means
A post-processing lip sync pipeline works in two stages. First a video model generates the footage; then a second model takes an audio clip and re-warps the mouth region to match the phonemes. Because the mouth is edited after the fact, the seams show up: the lower face can look pasted on, jaw and cheek motion don't always follow, and the timing drifts on fast speech.
Native lip sync folds that into the generation itself. You write the line in the prompt, and the model generates the speech and the mouth, jaw, and facial motion that fit it as part of the same render — not as an edit layered on top. Because voice and face are generated together, the mouth moves with the cheeks, the expression matches the line, and the timing is locked from the first frame.
The flip side: Kling 3.0 Omni doesn't take an audio file. The voice is the model's own, generated from your text — you can't upload a recording for it to follow.
That's the difference that matters: with a post step you're correcting a finished video; with native lip sync the talking is baked in.
Kling 3.0 Omni specs
Kling 3.0 Omni is built as an omni-modal model — lip sync is one capability among several that work together. Here's what it does, as integrated in Renoise:
| Capability | Kling 3.0 Omni |
|---|---|
| Clip length | 3–15s (≤10s when a reference video is included) |
| Resolution | 720p / 1080p |
| Aspect ratios | 3 (16:9 / 9:16 / 1:1) |
| Inputs | Text, reference images, a reference video (no audio input) |
| Lip sync | Native, voiced by the model from your prompt |
| Multi-subject consistency | Yes — tracks multiple subjects in one shot |
| Shots per generation | One clip; build a sequence shot by shot |
| Physics | Physical-dynamics simulation |
| References | Up to 7 images (≤4 with a reference video) + 1 video |
A few of these are worth unpacking, because they're what make lip sync usable beyond a single talking head.
Multi-subject consistency
A talking scene is rarely one face. Kling 3.0 Omni can hold multiple subjects consistent within a shot — two people in a dialogue, a presenter next to a product — so the right mouth moves for the right line and each subject keeps their appearance across the clip. As a model-layer improvement this is much tighter than older models, though like any AI video model it can still drift, so it's worth reviewing the result rather than assuming a perfect lock.
One shot per generation
Kuaishou's own Kling app has a multi-shot storyboard mode, but in Renoise each Kling 3.0 Omni generation is one clip. For a short dialogue scene or a multi-beat ad, generate each shot with the same reference images so the speaker keeps their look, and line the clips up on the Canvas — or ask Renoise Agent to plan the shots and join the clips for you.
Multimodal inputs and physical dynamics
The model takes text, reference images and a reference video together — but not audio. The sound comes from the model itself, which is exactly why the voice and the mouth line up so closely. On top of that, its physical-dynamics simulation keeps motion plausible (hair, cloth, gesture), so a talking subject still moves like a real one rather than a floating face.
Reference handling
You can attach up to 7 reference images to anchor a character, style, or scene. If you also supply a reference video (one video clip), the image budget drops to 4 and clip length caps at 10 seconds — a deliberate trade, because a reference video already carries a lot of motion and identity information. A reference video also switches native audio off, so there's no generated speech to sync to; keep talking shots to image references.
How to make a lip-sync clip in Renoise
On the Renoise Canvas, a talking clip is a few steps:
- Open the video tool and pick Kling 3.0 Omni as the model (or go straight to
/videos?model=kling-3.0-omni). - Add your subject. Upload a reference image of the character or scene, or write a prompt to generate one. You can attach up to 7 reference images to lock the look.
- Write the line the subject should say, in quotes, and name who says it — Kling 3.0 Omni generates that voice and drives the lip sync from it.
- Describe the delivery: the scene, camera, tone and energy, not the mouth shapes — the model handles those from the line.
- Set length and aspect ratio (3–15s; pick 9:16 for Reels/TikTok, 16:9 for YouTube). Leave out the reference video for talking shots: it caps the clip at 10 seconds and turns native audio off.
- Generate, then review the sync and the subjects before exporting. Watermark-free export is available on paid plans.
For a single still photo that talks, see the AI talking photo guide; for syncing to a real, authorized person, see celebrity-style lip sync; and for music-driven motion, the AI dance video guide.
Lip sync to your own recording
Kling 3.0 Omni won't follow a voiceover or song you already have — it accepts no reference audio, and neither do Kling 4.0 and Kling 4.0 Omni. For that job, give Renoise Agent one portrait and your recording (mp3 or wav, 4 seconds to 5 minutes, speech or singing). It runs OmniHuman 2.0 by ByteDance, which animates the mouth to follow the recording; the clip runs as long as the audio, at 720p. Pure instrumentals are rejected — there has to be a voice.
Lip sync of real people
If you want a real person's likeness to speak on camera, use only authorized real faces — your own, or one you have written consent for. Celebrities, public figures and minors are not allowed, and local laws apply. It's a consent rule, not a creative limit: with an authorized face, Kling 3.0 Omni makes it speak the lines you write like any other subject.
Tips for clean results
- Keep clips short with a reference video. With a reference video attached, you're capped at 10 seconds and 4 image references — plan the shot around that rather than fighting it.
- Match the aspect ratio to the platform up front (9:16 vertical, 16:9 horizontal, 1:1 square) so you're not re-cropping a face later.
- Write clean dialogue. Put each spoken line in quotes and name who says it; one speaker per sentence keeps the mouth timing tighter.
- Reuse references across shots. When you need multiple shots of the same speaker, attach the same reference images to every generation so the face stays consistent from clip to clip.
- Want a second take? Seedance 2.0 (ByteDance) is also live in Renoise and generates its own audio too — run the same line on both and keep the better result.