Introduction
Match recorded mouth movement to a clean replacement voice track using Lite or Basic. The recording sets the intended result length, while mode-specific controls cover scene detection or Lite noise removal and looping.
| Field | Value |
|---|---|
| Input | Video, Audio |
| Output | Video |
| Service publisher | ByteDance Lip Sync |
| Endpoint identifier | bytedance/lip-sync/lip-sync |
| Accepted visual media | MOV / MP4 / MKV; 360p–1080p, 24–60 fps, 1–30 Mbps |
| Accepted voice media | MP3 / WAV / AAC / M4A / OGG, up to 10 MB |
| Profile / audio limit | Lite: >1–240 s; Basic: >1–210 s |
| Basic control |
|
| Lite control defaults | Noise removal false; loop true; reverse false; source start 0 s |
| Delivered video | MP4, 25 fps; intended duration follows speech |
| Billing basis | Input-audio seconds rounded up; same rate in both modes |
Key features & highlights
A shorter line trims the listening-room shot
In our tests, the record-store take ran 6.04 seconds before receiving a 5.2-second recording in Lite mode. The output stopped at 5.20 seconds, with the woman's mouth following the replacement line. Audio determined the ending of the edit.
Choose the controls your scene needs
Basic exposes scene_detection for shot boundaries and speaker recognition in single-speaker footage. Lite provides background-noise removal, looping, reverse looping, and a source start position. Mode-incompatible combinations are rejected when the request is accepted, before processing begins.
Repeat a take to finish the dialogue
Our Lite control trial combined loop_video, loop_reverse, video_start_seconds set to 1, and background-noise removal. A 5.04-second source covered a 6.5-second recording and returned 6.52 seconds of video. Review the repeated movement and its change of direction in your own edit.
Inspect the picture at 25 frames a second
The listening-room result measured 864×496 at 25 fps. The woman wears a burgundy satin camisole beneath a dark jacket, with a short dark pixie cut. This sample demonstrates revised speech timing in existing footage; its dimensions describe that particular output.
Request parameters
| Parameter | Required | Description |
|---|---|---|
video_url | Yes | Required in Lite and Basic, with no default: provide MOV, MP4, or MKV at 360p–1080p, 24–60 fps, and 1–30 Mbps. H.264 is recommended. at least 1 characters |
audio_url | Yes | Required for either profile, with no default: clean human speech in MP3, WAV, AAC, M4A, or OGG, up to 10 MB. at least 1 characters |
generation_mode | No | Optional selector accepting lite or basic, default lite. Lite targets frontal single-speaker video; Basic targets complex single-speaker scenes. |
remove_background_noise | No | Optional Lite boolean, default false. Separates background sound from the recording before mouth alignment. Default: |
scene_detection | No | Optional Basic boolean, default false. Enables scene detection and speaker recognition for its single-speaker workflow. Default: |
loop_video | No | Optional Lite boolean, default true. Repeats footage to cover speech that runs longer than the video. Default: |
loop_reverse | No | Optional Lite boolean, default false; meaningful with |
video_start_seconds | No | Optional numeric source position for Lite, default 0 seconds. Choose a starting point within the supplied video; omit this field in Basic. |
Pricing
$0.048 / second
Chargeable duration is the uploaded audio rounded upward to whole seconds. Audio must exceed 1 second, with a 240-second ceiling in Lite or 210 seconds in Basic. Both profiles share the same rate; video duration and compatible optional controls do not change it. Server-busy responses are not charged and are retried automatically.
Use cases
Replace a listening-room invitation
Use the record-store example as a timing reference for an authorized dialogue revision. A shorter replacement recording can remove the unused ending while the mouth delivers the new words.
Dub a frontal presenter take
Choose Lite for a single presenter facing the camera. Finish the translated or revised speech separately, then submit it with the source video for mouth alignment.
Process a single-speaker sequence with cuts
Choose Basic and enable scene_detection when a single-speaker clip contains scene changes. Inspect synchronization around each cut after the asynchronous task completes.
Cover a longer voice-over with short footage
Lite can loop the source when speech outlasts it. Set the starting second and consider alternating playback direction, then check whether the repeated action fits the intended delivery.
Prepare a voice track with background sound
For a Lite job, remove_background_noise separates background sound from the driving voice before lip alignment. Listen to the source recording and review the returned speech; Basic requests require that control to be omitted.
Prompt guide & pro tips
Build the media pair
Provide an accepted video URL and clean speech URL. Check the audio against the file-size and duration limits for your chosen profile.
Keep fields within the profile
Set
generation_modeor use its Lite default, then include compatible controls. Submit the task and record its identifier for status checks.Collect the completed edit
Retrieve completion by polling or callback, inspect the MP4, and save the file before its 14-day deletion deadline.
ByteDance Lip Sync: select a scene profile
ByteDance Lip Sync is ByteDance's video lip-sync service, part of its video translation tools. An existing single-speaker video supplies the picture, and a separate human-voice recording drives the replacement mouth movement. The returned MP4 uses 25 fps and follows the recording's duration: excess source footage is trimmed, while short footage loops to cover longer speech.
Choose lite for frontal single-speaker footage. Select basic for complex single-speaker scenes, with optional scene detection and speaker recognition. These are scene profiles at the same rate. The endpoint does not clone voices; its soundtrack comes from the audio you upload.
Parameters and profile compatibility
Use bytedance/lip-sync/lip-sync as the request model. Both URLs are required. Optional controls must match the selected profile; incompatible combinations are rejected during acceptance before a generation starts.
| Public field | Value or default | Applicable profile and effect |
|---|---|---|
video_url | Required URL; no default | Both: MOV, MP4, or MKV; 360p–1080p, 24–60 fps, 1–30 Mbps; H.264 recommended. |
audio_url | Required URL; no default | Both: clean MP3/WAV/AAC/M4A/OGG, up to 10 MB, longer than 1 second; Lite ceiling 240 seconds, Basic ceiling 210 seconds. |
generation_mode | lite or basic; default lite | Chooses the scene profile without changing the audio-second rate. |
remove_background_noise | Boolean; default false | Lite: separate background sound before driving the mouth; omit in Basic. |
scene_detection | Boolean; default false | Basic: detect scenes and identify the speaker; omit in Lite. |
loop_video | Boolean; default true | Lite: repeat picture to cover a longer recording; omit in Basic. |
loop_reverse | Boolean; default false | Lite with looping enabled: alternate forward and reverse playback; omit in Basic. |
video_start_seconds | Number; default 0 | Lite: start picture playback at this source second; omit in Basic. |
There is no prompt or resolution selector in the public field set. Use the recording to provide the words. Video above 1080p is reduced to 1080p; below-360p footage is rejected.
Billing follows the recording
Count the input audio's seconds and round upward to the next whole second. Lite and Basic share one per-second rate; supported control changes and source-video length do not change that basis. Consult the ByteDance Lip Sync model rate for the current price.
The 5.2-second record-store recording counts as 6 billable seconds. Shortening its video would not reduce that audio-based quantity. Trim unwanted silence or unused speech before submitting, while keeping the recording longer than 1 second and within the selected mode's ceiling.
Prepare a compatible pair
Inspect the video for a clear single speaker and check its file format, resolution, frame rate, and bitrate. For Lite, use frontal footage. For a complex scene that needs shot analysis, choose Basic and enable scene_detection if appropriate.
Listen to the separate voice recording before upload. Prepare any translated wording and its recording beforehand. Lite can separate background sound through remove_background_noise; Basic does not accept that setting. When selecting Basic, also omit loop_video, loop_reverse, and video_start_seconds, including values copied from a Lite request.
Compare the two input durations. The recording defines the intended edit length. In Lite, plan repeated footage with loop_video, add loop_reverse for alternating direction, and use video_start_seconds to select a source starting point. Review repeated gestures after completion.
Observations from our tests
The record-store sample paired 6.04 seconds of 864×496 footage with a 5.2-second line in Lite, with background-noise removal disabled. The woman in the listening room delivered the replacement words through revised mouth movement. Its result measured 5.20 seconds at 864×496 and 25 fps, demonstrating trimming to the recording.
A separate Basic run with scene detection paired a 6.04-second source with 5.2 seconds of speech and returned 5.28 seconds. Another Lite run enabled looping, reverse looping, noise removal, and a source start of 1 second: 5.04 seconds of video covered 6.5 seconds of speech, returning 6.52 seconds. Small measured differences around the recording length occurred in these runs.
Three Basic attempts with background-noise removal failed in our tests. Lite processed that option successfully, so the exposed compatibility rules restrict it to Lite. These observations explain the field split without promising a result for every input.
Completion, waiting, and file limits
Submission creates an asynchronous task and returns an identifier. Poll for completion or receive a callback, then retrieve the finished MP4. The service can return a server-busy response; those responses are not charged and are retried automatically. Peak traffic can add waiting time, so task acceptance does not establish completion.
Use footage, faces, and voice recordings you are entitled to use; every person shown must be an adult. Never impersonate another person. Generated files are deleted after 14 days, so keep a downloaded copy of a completed edit you need to retain.
Compare related video lip-sync choices
Sync Lipsync 2 provides five duration-alignment modes and bills source-video seconds. Consider it when those timing choices suit your edit, accounting for the different billing measure. VEED Lipsync 2.0 and Lip Sync are additional related choices. Evaluate your own authorized video and recording against each endpoint's supported inputs.
API quick start
# Generate the idempotency key once; reuse it when retrying after a lost response.
export SPICY_IDEMPOTENCY_KEY="${SPICY_IDEMPOTENCY_KEY:-$(uuidgen)}"
curl -X POST https://api.spicyapi.ai/api/v1/jobs/createTask \
-H "Authorization: Bearer $SPICY_API_KEY" \
-H "Content-Type: application/json" \
-H "Idempotency-Key: $SPICY_IDEMPOTENCY_KEY" \
-d '{
"model": "bytedance/lip-sync/lip-sync",
"input": {
"audio_url": "https://example.com/reference.mp3",
"generation_mode": "lite",
"video_url": "https://example.com/reference.mp4"
}
}'
# Copy data.taskId from the response into TASK_ID, then poll the status.
TASK_ID="TASK_ID_FROM_RESPONSE"
curl "https://api.spicyapi.ai/api/v1/jobs/recordInfo?taskId=$TASK_ID" \
-H "Authorization: Bearer $SPICY_API_KEY"
See the API reference for authentication, the complete schema, polling, and error handling.
