Quick Answer: AI video motion capture turns one generated clip into a skeletal animation. anymove films a creature from two cameras in one frame, trains a DINOv2 joint reader on that single creature, finds every joint in both views, and fits a rigged skeleton to the 3D points. The result exports as a GLB. A five-second clip costs about $0.26 on fal and fits in under a minute.
A five-second AI clip costs less than a coffee, but the motion inside it is locked in pixels. Getting that motion into a game engine is the hard part. Most people give up at the point where a dragon's wing needs to bend the way a real joint would. Here's how anymove gets past that step, what it actually costs, and where it still breaks.
Why one camera fails on a creature
A single AI video shows one view of a creature. Depth is a guess. A wing that swings toward the camera looks the same as one that folds back, and a pose estimator trained on humans has no reason to know that a dragon's tail has twenty-odd joints.
The fix isn't a bigger model. It's a second view. anymove asks the video model to film the creature from two cameras in one frame, front on the left and side on the right. Both views share the same motion, so a joint you can see in both gives you the depth you were missing.
The common advice says to pick the single best angle and work from it. That breaks down fast for creatures with wings, tails or legs that cross behind the body. The second view is the only thing that makes the 3D solve well-posed.
Training a reader on one creature
Most guides reach for a general-purpose pose model. anymove doesn't, and that's the part most people find surprising. It trains a joint reader on the one creature you're animating.
Here's the workflow:
- Build a mesh at real size. Generating it with Tripo H3.1 with HD texture costs about $0.40 in the README's setup.
- Place the skeleton on grid views in Blender, then skin and paint each limb a distinct colour so left and right never swap.
- Render about 2,200 random training poses, each in both views, with the true joints known. This takes roughly 25 minutes on one RTX 4090.
- Train a DINOv2-based reader for about 20 epochs. Training takes 17 to 20 minutes on 12 GB of VRAM.
The reader learns this dragon, not dragons in general. Its average miss falls from 40 px to 6 px on training renders. That narrow focus is why the reader can find a joint that a generic model would miss in a stylised creature.
The catch is the training set. Random poses only cover what sits inside the joint limits. If your creature never lies on its side in training, it won't read that pose later.
Fitting the rig in 3D
Once the reader has found every joint in both views, the solver does the geometry. A joint sits at the same height in both views, so each joint is where its two lines meet. The front view gives side-to-side position, the side view gives nose-to-tail, and both views together give depth.
This next part matters more than it looks. The solver ignores stray detections, so a bad point in one frame doesn't drag the whole skeleton off course. The rig bends its bones onto the surviving points frame by frame, and the output is a GLB at 24 fps with no hand edits.
Reading a clip and fitting it takes about 10 seconds plus under a minute of solving. The dragon's roar fits on a CPU in about 20 seconds, which is a useful sanity check before you invest in a GPU run.
What it costs and how long it takes
| creature | skeleton | reader error, front / side (px) | view agreement (1 = same footfalls) |
|---|---|---|---|
| wolf, 1.6 m | 30 bones, 36 joints | 6.9 / 3.8 | trot 0.94 |
| dragon, 3 m | 42 bones, 50 joints, wings | 6.2 / 5.5 | take off 1.00, roar 0.93 |
| giant spider, 2.4 m span | 28 bones, 40 joints, 8 legs | 2.9 / 3.6 | rear up 0.99, bite 0.96 |
| fire demon, 3.6 m | 38 bones, 46 joints, wings, tail | 8.4 / 9.4 | wing beat 0.99, slam 0.84 |
The demon stands 412 px tall, so its 8.4 px error is about 2% of its height. That's the number to compare against, not the raw pixel count. The wolf's exam, which renders a made-up motion with known joints, read back at 0.94 cm mean 3D joint error.
Costs in the README, priced on fal as of October 2026: a mesh is $0.40, a five-second two-camera clip with MiniMax H3 Max is about $0.26 (H3 is $0.30), and a LoRA trained for 1,200 steps is about $12 over roughly two hours. A new move for an existing creature costs about $0.52 for two clips.
Where it breaks: eight legs and lying down
Every honest capture pipeline has failure modes, and this one lists its own. Three matter most.
- Eight-legged walking doesn't fit. The two views disagree about which legs are up, with agreement scores of 0.36 to 0.49. Big single gestures work best: rearing, roaring, slamming, taking off, pouncing.
- Lying down doesn't fit. The reader trains on random poses inside the joint limits, which exclude lying on a side. The wolf's death and lie-down clips failed for this reason.
- Video models drift. Plain MiniMax H3 swapped limb colours on 2 of 5 wolf clips. With the LoRA, none of five did. That's one seed per move, a first read rather than a benchmark.
Here's the counter-intuitive part. You might assume the LoRA's training data is clean motion capture. It isn't. The LoRA's clips re-render motions solved from AI clips, so the two views stay in sync by construction, but the motion itself is AI-made. Real dog mocap exists in AI4Animation, but it carries a non-commercial licence.
Licensing matters too. MiniMax H3's open weights carry a community licence, and hosted use on fal goes through fal's terms. The wolf LoRA in the release is a LoRA on MiniMax H3, so MiniMax's community licence applies to it, not Apache-2.0. Check both before you ship anything you make with it.
Frequently Asked Questions
what is AI video motion capture?
AI video motion capture takes motion from a generated video and converts it into a skeletal animation. anymove does this by filming the creature from two cameras in one frame, locating every joint in both views, and solving those joints into 3D. The output is a rigged GLB you can load into an engine. It needs no suit, markers or studio.
how do you motion capture a creature from AI video?
Build a rigged mesh at real size, render a few thousand training poses with known joints, then train a joint reader on that one creature. Generate a two-camera clip with a prompt, run the reader on the clip, and fit the rig to the 3D joints. In the anymove repo this takes about 25 minutes to render, 20 minutes to train and under a minute to fit.
why do some AI animations fail when fitted to a rig?
The two views have to agree on every joint. Eight-legged walking often fails because the front and side views disagree about which legs are lifted, with agreement scores as low as 0.36. Poses outside the reader's training range, like lying on a side, also fail. Big single gestures such as rearing or roaring fit most reliably.
can I use AI video motion capture for commercial projects?
Check the licence first. The anymove code is Apache-2.0, but the MiniMax H3 model carries a community licence and hosted use on fal follows fal's terms. The wolf LoRA counts as a LoRA on MiniMax H3, so that licence applies to it. Read both before you ship.
Closing thoughts
The single most useful habit is to test your creature before you commit to a full run. Render one clip, read it, and check whether the two views agree on the footfalls. If they do, you've got a capture worth rigging. Try a single gesture on your own creature this week, such as a rear or a roar, and note the agreement score. Pass this to anyone wrestling with AI video motion capture for a creature that won't rig cleanly.