Quick Answer: Humanoid whole-body motion tracking means getting a robot to reproduce a reference motion with its entire body at once, not just its hands or feet. SuperMimic, an Apache-2.0 project from HybridRobotics, does this with one differentiable world model built on the mjlab framework. It offers two state representations, generalized coordinates and maximal coordinates, for the Unitree G1 humanoid.

Watch a humanoid try to match a dance clip and the hard part shows up fast. Every joint has to hit its target at the same moment, the torso has to stay upright, and the feet have to carry the load without slipping. Most tracking pipelines handle this by stacking separate pieces: a simulator, a policy, a reward function, and a hand-tuned controller. SuperMimic takes a different route, and the design choices behind it are worth understanding even if you never run the code.

What a whole-body tracker is actually optimizing

A motion tracker takes a reference trajectory, usually captured from a human or generated by an animation tool, and produces joint commands that make the robot follow it. The "whole-body" part matters. Tracking only the end effectors leaves the rest of the body to drift, and drift in a humanoid is how robots fall over.

Here's the core tension. The reference motion tells you where the body should be. It says nothing about the forces needed to get there, and a physical robot can't teleport between poses. A tracker has to reconcile kinematic targets with dynamics, contact, and balance at every control step.

That's why the representation of the robot's state is a real design decision, not a formality. The README's two task IDs reflect that choice directly.

One differentiable world model instead of a stack

The project description says SuperMimic "trains a humanoid whole-body motion tracker with one differentiable world model." A world model is a learned predictor of how the environment responds to actions. Given the current state and a control input, it predicts the next state. Model-based reinforcement learning has used this idea for years, and DreamerV3, one of the projects the README credits, is a well-known example of learning a world model and training a policy inside its imagined rollouts.

The word differentiable is the important part. Because the model is differentiable, gradients of a tracking loss can flow backward through predicted trajectories to the policy parameters. In a standard model-free setup, the policy learns only from sampled rewards. A differentiable model gives it a direct signal about how a small change in an action would change the outcome several steps later.

Here's the counter-intuitive part. Most people assume a differentiable model is automatically a better simulator. It isn't. A learned model is an approximation, and its errors compound across the rollout horizon. If the model is off by a small amount per step, a 50-step prediction can be far off by the end, and the gradient you backpropagate through it reflects that error. Gradients through a wrong model can push a policy confidently in the wrong direction. The payoff depends on how well the world model matches the real physics over the horizons you train on, and that's an empirical question that the README does not answer. Check the paper before assuming the differentiable setup transfers to hardware.

Generalized coordinates vs. maximal coordinates

The two task IDs are the most practical part of the README:

  • SuperMimic-GenCoord-G1 uses generalized coordinates, where the state is the minimal set of joint variables that fully describe the robot's configuration.
  • SuperMimic-MaxCoord-G1 uses maximal coordinates, where each rigid body carries its own pose, and constraints such as joints link them together.

This is a standard trade-off in multibody dynamics. Generalized coordinates keep the state small and enforce kinematic constraints automatically, which is why most robot controllers use them. Maximal coordinates are more redundant, but they make it easier to express contacts and body-level targets directly, since each link's pose is an explicit variable.

AspectGeneralized coordinates (GenCoord-G1)Maximal coordinates (MaxCoord-G1)
State definitionMinimal joint variablesFull pose for each rigid body
Kinematic constraintsEnforced by constructionEnforced through explicit constraint equations
Body-level targetsComputed through forward kinematicsRead directly from the state
Typical use caseCompact control and standard joint-space policiesBody-level tracking and contact-rich motion

The README doesn't say which representation performs better on which motions, so treat this table as a guide to what each one simplifies, not a verdict. If your reference motion depends heavily on individual limb poses, the maximal representation's direct access to body state is worth testing first. If you care about a compact state for a deployable policy, start with generalized coordinates.

Setting up the two repositories

SuperMimic depends on a sibling repository, so the layout matters. Following the README's setup steps:

  1. Install uv with the official install script. uv manages the Python environment and dependencies for the project.
  2. Create a workspace folder and clone both supermimic and mjlab_research into it as siblings. Putting them side by side is required, not optional.
  3. Change into the supermimic directory before running any scripts.

The most common failure here is cloning mjlab_research somewhere else in the tree and then wondering why imports fail. Keep the siblings together, and you avoid the problem before it starts.

Training and playback

Training runs through train.py. You pick a task ID, set the GPU, and give the run a name so you can find it later:

CUDA_VISIBLE_DEVICES=<GPU_ID> uv run src/supermimic/scripts/train.py \
  <TASK_ID> --runner.run_name <description>

Playback loads a trained run from Weights & Biases:

uv run src/supermimic/scripts/play.py \
  <TASK_ID> --wandb-path <RUN_PATH>

Two practical notes. First, the CUDA_VISIBLE_DEVICES prefix pins training to one GPU, which helps when several jobs share a workstation. Second, the --runner.run_name flag is worth using every time. Without distinct names, runs blur together in the logs, and you lose track of which representation or hyperparameter produced which behavior.

The README lists only two task IDs: SuperMimic-GenCoord-G1 and SuperMimic-MaxCoord-G1. Both target the G1 humanoid. The repo does not document support for other robots, so don't assume the same commands will work on a different platform.

Where SuperMimic sits in the research lineage

The README acknowledges three projects. ControlVAE (BSD-3-Clause) is a variational approach to controlling simulated characters. robotic_world_model (Apache-2.0) comes from ETH Zurich's Robotic Systems Lab and learns dynamics models for legged robots. DreamerV3 (MIT) contributes the general recipe of training policies inside a learned world model.

SuperMimic's README says it "contains code inspired by" these projects. That wording means borrowed ideas or code patterns, not a claim that SuperMimic is a drop-in version of any of them. When you read the source, check the license headers in each file if you plan to reuse code, since the README alone doesn't map files to licenses.

Here's where the practical picture gets murkier. The project is released under Apache-2.0, and the copyright line reads 2026. Research code like this often changes quickly after release, so pin a commit before you build anything on it.

Risks and caveats you should know about

The README is a usage guide, not a results document. It doesn't report tracking error, success rates, or hardware trials. Everything about how well SuperMimic performs has to come from the linked paper and project page, and you should verify those numbers against your own motions.

Three specific caveats apply:

  • Model error compounds. A differentiable world model is only as good as its fit to real dynamics over the rollout length you train on. If you change horizon length or the motion distribution, re-check the model before trusting the policy.
  • Simulation is not hardware. The stack is built on mjlab, a MuJoCo-based framework. Contact modeling, actuator delays, and friction rarely match a real robot exactly. Expect a sim-to-real gap and budget time to close it.
  • Representation choice has consequences. Switching from generalized to maximal coordinates changes what the policy sees and how it can be deployed. Don't treat the two task IDs as interchangeable configurations.

Frequently Asked Questions

What is humanoid whole-body motion tracking?

Humanoid whole-body motion tracking is the task of making a humanoid robot reproduce a reference motion with all of its joints and body parts together while staying balanced. Unlike end-effector tracking, it requires the torso, legs, and arms to match the target at the same time. SuperMimic addresses this with a learned world model and a policy trained on top of it.

How do I train a SuperMimic policy?

Clone supermimic and mjlab_research as sibling folders, install uv, and run train.py from the supermimic directory. Pass a task ID such as SuperMimic-GenCoord-G1 and a --runner.run_name so you can identify the run later. Use CUDA_VISIBLE_DEVICES to choose the GPU.

Is a differentiable world model better than a standard simulator?

Not automatically. A differentiable world model lets gradients flow through predicted rollouts, which gives the policy a richer training signal. The trade-off is that a learned model's errors compound over long horizons, so the gradients can point the wrong way if the model is inaccurate. Validate the model against real dynamics before relying on it.

When should I use generalized coordinates instead of maximal coordinates?

Start with generalized coordinates if you want a compact joint-space state and automatic enforcement of kinematic constraints. Test maximal coordinates when your motion depends heavily on individual body poses, since each rigid body's pose is an explicit state variable there. The README doesn't benchmark the two, so compare them on your own reference motions.

Humanoid whole-body motion tracking gets much easier to reason about once you separate the representation, the world model, and the training loop, and SuperMimic keeps those pieces visible in a small codebase. Clone both repos, run one training job with SuperMimic-GenCoord-G1 and a clear run name, then compare its playback against the MaxCoord variant on the same motion. Share this with someone who's stuck on whole-body balance and wondering whether a differentiable model is worth the complexity.