Quick Answer: An AI digital human video is a talking-head clip where a generated presenter speaks a script with lip-synced motion. The open-source Digital Human Koubo agent skill makes one from a single photo: it writes the script, generates segments through Renoise MCP, cleans the audio, adds captions, and packages the edit. You need an Agent Skills-capable agent, a Renoise account, and ffmpeg, Python, and Node.js on your machine.
Most people assume the hard part of an AI presenter is the face. In the author's livestream ad demo, the face, voice, studio, captions, and price animation were all generated, and the only human input was one sentence. The total cost was 482 Renoise credits, as shown in the project README. The real bottleneck turned out to be something else, and that is what this guide explains.
What You Need Before You Start
The project is a folder with a SKILL.md file plus Python, shell, and Remotion scripts. To run the whole pipeline, your agent needs three capabilities:
- Agent Skills loading, so it can read the skill folder.
- A remote MCP server connection, here Renoise, which uses streamable HTTP and browser login with no API key to paste.
- Local command execution, for ffmpeg downloads, QA, and editing.
Local tools cover the rest: ffmpeg, Python 3 with numpy and Pillow, Node.js 18 or later for Remotion, and whisper.cpp (whisper-cli) for transcript checks. If you already have some of these, the setup step skips them.
How the Pipeline Turns One Photo Into a Presenter
The skill fixes the presenter first. Face, outfit, voice description, and prompt template live in a persona.json file, so every video features the same person. You can place her in a new scene, such as a livestream studio or a classroom, by compositing with GPT Image before she talks.
The script is then split into segments of 15 seconds or less. At roughly 5.4 Chinese characters per second, that works out to about 80 characters per segment. Each segment is generated as a lip-synced clip with Grok Video 1.5 through Renoise MCP. The repo also documents Wan 3.0 and OmniHuman for footage you record yourself.
Here's the sequence the skill follows:
- Diagnose the content and recommend two of five editing styles.
- Write
script.jsonwith segment timing. - Quote credits and wait for approval.
- Generate segment 1 for your review, then batch the rest.
- Download, transcribe with whisper, and inspect frames.
- Unify the audio, build a word-level timeline, and render in Remotion.
The Counter-Intuitive Part: Audio Drift Beats Face Quality
The common advice is to spend your effort on the photo and the model. That advice is incomplete. Once you generate 30 clips separately, each one carries its own microphone character, room tone, and loudness. The result sounds like 30 recordings stitched together, even when every face looks perfect.
The project addresses this with 1/3-octave spectral matching, de-essing, one shared small-room reverb, and loudness normalization. In the repo's tests, sibilance spread dropped from 13.2 dB to 5.9 dB. The final render targets -14 or -16 LUFS. The lesson is practical: if you're budgeting time for an AI presenter, set aside more of it for audio than you think you need.
Here's where most people get surprised. Grok sometimes burns subtitles into the picture itself. You can't remove them in post without damaging the frame. The skill detects this with four frame inspections per segment, and those segments get re-shot.
Cost Gates and Quality Control
Before any credits are spent, the agent quotes segments, seconds, credits, and your balance, then waits for your explicit "yes." That gate matters because a full video is many generations, and a bad prompt across all of them is expensive.
Generating segment 1 first is the second gate. You check the lip sync, face consistency, and tone before the batch runs. Once the clips arrive, whisper transcribes each one and compares it against the script. The repo reports whisper back-check scores of 94–100% across English, Japanese, Korean, French, Spanish, and Arabic. Those numbers are the author's tests, so treat them as a benchmark for your own languages, not a guarantee.
Most guides skip this step, and it's the one that saves you money. A clip that transcribes wrong is a clip you'll regenerate later anyway.
Choosing Among the Five Editing Styles
The skill offers five styles and recommends two based on your content. Style A is the finance and tech explainer, with charts, documents, and chapter cards. Style B suits livestream sales, with price animations. Style C puts the presenter on one side and code B-roll on the other. Style D is a step-by-step tutorial, and Style E handles multi-presenter pieces.
Each style has a default packaging recipe drawn from 48 components. The author measured those components frame by frame across 25 videos by creators such as MrBeast, Mark Rober, MKBHD, and Kurzgesagt. Measured packaging is useful, but it reflects the author's selection and method, so it's a starting template rather than a rulebook. You can add or remove techniques by asking the agent.
The trade-off is between flexibility and predictability. A style template gives you a fast, consistent result. Custom packaging gives you more control, but every extra effect adds render time and another place for mistakes.
Which Agents Work, and How Well Tested Are They?
The project's own status table shows that testing varies by agent. Only Claude Code was fully tested by the author. The rest are listed as expected or should-work, so check your product's docs before you commit.
| Agent | Skills location | Local commands | Status (per README) |
|---|---|---|---|
| Claude Code | ~/.claude/skills/ | Yes | Tested; the repo was built with it |
| Codex (CLI / desktop) | ~/.codex/skills/ | Yes | Expected to work, same scripts |
| OpenClaw | Skills folder per its docs | Yes | Should work |
| WorkBuddy | Import folder or zip | Yes (desktop agent) | Should work; menu names may differ |
| Claude desktop / claude.ai | Upload zip in Settings | Sandbox only | Planning, quoting, and generation work; local editing does not |
If your agent can only chat and call MCP, you can still plan, quote, and generate clips through Renoise. Then move the editing steps to a shell-capable agent.
Limits and Responsible Use
Three caveats matter here. First, the pipeline needs a command line for the editing stage. Non-technical users will hit a wall unless they have help. Second, the presenter is a synthetic person, so disclosure matters. The author labels every demo "内容由 AI 数字人演绎" (performed by an AI digital human). Follow the labeling rules where you publish, and get consent before you build a presenter from someone else's photo. The demo's male presenter is the author's own digital twin, made from his own photo.
Third, cost scales with length. A 30-second ad needed 482 credits in the demo, so run the quote gate before you commit to a long script.
Frequently Asked Questions
What is an AI digital human video?
An AI digital human video is a talking-head clip featuring a generated presenter who speaks a script with lip-synced motion. In this project, the presenter is defined once in a persona file and reused across videos, so the same face appears every time. Speech is produced from a script, and the clips are edited with captions and packaging.
How do I make an AI digital human video from one photo?
Install the skill in an agent that supports Agent Skills, connect Renoise MCP, and install ffmpeg, Python, and Node.js. Give the agent your photo and a one-sentence brief. It will recommend a style, write a script, quote the credit cost, and generate a test segment for you to approve before batching the rest.
Why does my AI talking video sound different from clip to clip?
Each clip is generated separately, so microphone character, room tone, and loudness drift between them. The fix is post-processing: spectral matching, de-essing, a shared reverb, and loudness normalization to a fixed target such as -14 LUFS. The repo's tests cut sibilance spread from 13.2 dB to 5.9 dB with this approach.
Can I use this with Claude Code or Codex?
Yes. Claude Code is the only agent the author fully tested, and Codex is listed as expected to work because it uses the same scripts. Both need local command execution for the editing stage. Confirm the current setup steps in the project README before installing.
Start With One Short Clip This Week
The fastest way to learn this pipeline is to skip the full ad and make one 15-second segment from a photo you have rights to use. Approve the quote, check the lip sync and the audio, then compare it against a second clip made with different audio settings. Note the result, and keep the spectral and loudness steps in the workflow before you scale to a longer AI digital human video. Pass this to someone wrestling with inconsistent audio across their presenter clips, and point them to the source repo to get started.