Scene-Aware Humanoid Locomotion via Unreal Engine Data Generation

Motivation

Humanoid locomotion on uneven terrain (stairs, ledges, slopes) is mostly tackled by one of the following families, each with its own limitation at the time we started:

Approach	Limitation
Reward-based RL	Rewards must be redesigned and tuned per terrain; the gaits are unnatural (over-bent knees, dragging feet); tall steps and ledges remain hard.
AMP	Imitating a human motion style makes the gait more natural, but the other issues remain and it tends to handle hard terrain worse.
Motion tracking (BeyondMimic, SONIC)	Natural motion with a simple reward, but the training data is almost all flat-ground motion capture with no scene.
Video real-to-sim (VideoMimic, CRISP)	Scene reconstruction and scene-motion alignment from monocular video are inaccurate (feet float or penetrate), retargeting from a different body adds errors, and every terrain needs full-body footage plus heavy processing.

In other words, what is missing is natural human-like motion that is correctly aligned with uneven terrain. Game engines, on the other hand, seem to have exactly this: Unreal Engine has its own physics, and game characters walk over uneven terrain all the time. So if we could put a G1 into Unreal, generate maps automatically and build a controller that walks the G1 around by itself, we could generate unlimited uneven-terrain humanoid locomotion data. That was the starting point of this project.

Preliminary study: what does human motion buy us?

Before building anything, we first wanted to know how much a human motion prior actually helps on uneven terrain. To find out, we compared two RL-based policies:

Pure RL: only a "don't fall" reward and regularization rewards.
AMP: the same, plus a human motion prior learned from the reference motions below.

Human reference motions used as the AMP motion prior.

Next, to compare the two fairly, we built the following terrain benchmark. Both policies are trained on the same terrains, with a curriculum whose difficulty increases gradually from 0 to 1:

Terrain map at difficulty 0 — Terrains at difficulty 0.

Terrain map at difficulty 1 — Terrains at difficulty 1.

After training on this benchmark, the two policies behave quite differently.

Pure RLAMP

Pure RL and AMP policies on the terrain benchmark.

In the video above, pure RL does traverse the terrain, but it keeps its center of mass very low to avoid falling, which looks unnatural. AMP, in contrast, walks much more naturally. However, it does not lift its feet very high, so it looks less stable when stepping over tall obstacles or high steps. The natural question, then, is how these differences show up in the numbers.

Success rate: pure RL vs AMP — Success rate of pure RL and AMP per terrain type.

Surprisingly, the success rates are almost the same. As the chart above shows, both policies are near 100% on almost every terrain type, from slopes and rough ground to stairs. The one clear exception is gaps, where AMP almost never succeeds (3.7% vs. 72.9%). If the motion prior does not improve success, where does it help?

Energy: pure RL vs AMP — Average energy consumption of pure RL and AMP per terrain type.

The answer is energy. As the chart above shows, AMP spends less energy than pure RL on 10 of the 11 terrains, for example 1.84 vs. 2.82 on flat ground and 2.73 vs. 4.65 on boxes. While pure RL gets across by keeping its center of mass low, AMP walks more like a person, and this turns out to be cheaper.

In short, human motion is a good reference for using energy efficiently. So what if we collected lots of human motion on uneven terrain and trained a tracking policy on it, like SONIC?

Getting uneven-terrain human motion from video

To do that, however, we first need human motion on uneven terrain, which motion-capture datasets rarely contain. The most practical source is video, and two existing works extract human motion together with the terrain from it: VideoMimic and CRISP. We started with VideoMimic as the baseline. Given a monocular video, it reconstructs the surrounding scene and the person's motion, and then retargets that motion onto the humanoid, so that each video yields a (terrain, motion) pair.

Input videoReconstructed scene + motionReconstructed terrain + G1

VideoMimic on stair videos.

As the video above shows, however, the terrain is not reconstructed accurately and the feet dig into the steps, so the result is physically unstable. This is a serious problem for motion tracking: if the reference foot sits inside a step, the policy is asked to imitate something that cannot happen physically, and it either ignores the reference or learns strange behavior to satisfy it.

Could a stronger pipeline fix this? CRISP is a natural candidate, since it replaces the key components with better ones: a stronger open-source video-to-motion model (GVHMR) and a better depth model (MoGe). We therefore expected noticeably higher quality.

Input videoCRISP reconstruction

CRISP on a stair video.

Indeed, thanks to the better depth, the stairs are represented much better than with VideoMimic. Even so, the alignment is still poor, and the toes still sink into the steps.

Even with better models, extracting terrain and motion from video leaves them misaligned. Do we have to reconstruct them from video at all? What if we started from a world where terrain and motion are aligned by construction?

Idea: put the humanoid into the game engine

Think about video games: characters walk up stairs and over rough ground with ease, and their feet land on the terrain because the engine knows exactly where the terrain is. So instead of extracting terrain and human motion from video, why not put the humanoid directly into a game engine and use its motion as the reference for motion tracking? As a bonus, this also removes retargeting issues: when human and humanoid bodies differ, retargeted motions become physically odd or hard to follow, but a humanoid animated inside the engine has no such gap.

To make this work, we need two things to animate a humanoid in Unreal Engine:

Base motions that the character plays repeatedly;
A controller that picks the right motion for the terrain at the right moment.

Building both from scratch would be a big project, so we looked for an existing one to start from and found Advanced Locomotion System V4 (ALS v4), a free Unreal asset that provides both.

Advanced Locomotion System V4 running in Unreal Engine.

As the video shows, its foot contacts are quite clean, even on uneven ground. This made ALS v4 a good starting point, so our first goal was to turn it into a Unitree G1 version by reusing its motions and controller. If the same system could drive a G1 instead of the mannequin, every frame it plays would already be a (terrain, G1 motion) pair, with no scene reconstruction and no human-to-robot retargeting in between. That is exactly the data the video-based pipelines were struggling to produce.

Base motions: why we retargeted ALS instead of using MoCap

For the base motions, our first attempt was to find motions similar to the ALS v4 ones in public MoCap (AMASS). Finding matching motions was hard enough on its own, but there was a more basic problem: Unreal plays motion clips in a loop. A walk cycle that starts on the left foot must return exactly to the same pose, or the transition is visibly jerky.

ALS v4 base motions — ALS v4 base motion assets.

To see why looping matters, note that ALS drives its character with a set of short clips like these: walk, run and sprint cycles in several directions, plus poses that are blended together. Each clip is played over and over, and the controller switches or blends between them depending on speed and direction.

An Unreal motion clip playing in an infinite loop.

The clip above loops forever and still looks natural, but only because its first and last frames match exactly. Such seamlessly looping clips, however, are almost impossible to find in public datasets. We therefore decided to retarget the ALS v4 motion assets themselves to the G1.

Retargeting ALS v4 to the G1

Retargeting itself is not trivial either. Mapping mannequin joints directly onto G1 joints does not work well, because the two bodies have different proportions. Instead, we used SMPL as a bridge: it is a parametric human body whose shape can be changed while the motion stays the same, so we can first match the mannequin, then morph the body into the G1's proportions, and only then transfer the motion to the robot.

Convert the ALS v4 mannequin motion into an SMPL body with the mannequin's shape.
Find the SMPL shape parameters of the G1 and swap them into the motion from step 1.
Retarget the G1-shaped SMPL motion directly to the G1.

**Retargeting overview.** ALS mannequin → mannequin-shaped SMPL → G1-shaped SMPL → Unitree G1.

WalkRunKneel

Retargeted G1 motions.

Moreover, because the source clips loop seamlessly, the retargeted G1 clips do too. As a result, they can be plugged straight back into the ALS controller in place of the mannequin's animations.

The retargeted motions driven by the ALS controller: the first G1 version of ALS.

At first glance, this works: the G1 walks and runs around the scene under the ALS controller. Looking closer at the legs on uneven ground, however, reveals a problem.

Problem: Unreal's IK does not fit the G1

The problem lies in foot IK, which adjusts the legs so that the feet land on the terrain. Unreal's built-in IK is not accurate for the G1. It worked for the ALS mannequin because the mannequin's hip and ankle are spherical joints, so each leg has only three joints and the optimized two-bone IK applies. The G1, in contrast, has one joint per degree of freedom, not spherical joints, so using the two-bone IK as-is gives wrong solutions.

Two-bone IK problem on the G1 — Two-bone IK on the ALS mannequin (left) and on the G1 (middle, right).

As the figure shows, the two-bone IK cannot place the G1's leg correctly. We therefore replaced it with our own general IK that solves over the G1's actual leg joints, so the foot can reach the target on the terrain with a pose the G1 can actually take.

Results with the custom G1 IK in Unreal.

With the G1 IK in place, the G1 could finally walk around Unreal scenes with its feet on the terrain. This let us test the whole idea end to end: we let it walk over ramps and climb-up obstacles in Unreal, recorded its motion together with the terrain, and used these recordings as reference motions to train tracking policies in simulation with BeyondMimic. As shown below, the trained policies walk on ramps of various slopes:

10° ramp20° ramp30° ramp

Policies trained on motion data collected in Unreal, tracking ramp motions in simulation.

Stairs: abrupt height changes

Since ramps worked, we next moved on to stairs, where the height changes abruptly, still using the same walk motion and IK.

The G1 walking up stairs with the walk motion and our IK.

As the video above shows, unlike on ramps, the IK breaks down in two ways:

joint angles change very sharply as the height changes;
the feet do not land where they should on the steps.

This happens because IK only corrects a walk motion that was made for flat ground. On a ramp the height changes gradually, so small corrections every frame are enough. On a staircase the height jumps at every step edge, so the IK target jumps too, and the leg has to change pose abruptly while the underlying walk cycle knows nothing about where the step is.

To fix this, we first tried detecting height changes in front of the G1 and planning a foot trajectory that never passes through the steps. However, between our limited Unreal knowledge and a task that was turning into pure engineering rather than research, this took too long. Instead, we took a simpler route and recorded dedicated stair-climbing motions so that the G1 climbs naturally.

Captured stair motion.

A single recorded motion, though, only covers one step size, while real staircases come in many heights and depths. To cover them, we used Unreal's blend space: given anchor motions, it fills in the space between them by linear interpolation, so we can generate motions beyond the recorded ones:

Stair up (blend space)Stair down (blend space)

Stair motions generated with a blend space.

This way, the recorded motions act as anchors and the blend space interpolates between them for the step sizes in between, so a handful of recordings covers a whole range of staircases.

The G1 climbing 30 cm-deep stairs with the stair motions.

As the video shows, however, the stair motions still do not remove the choppy feel when the step height changes.

Before vs. after adding stair motions

Despite this remaining issue, does the change actually help the tracking policy? To check, we generated reference motions on the same 30 cm-deep stairs twice, once with the walk motion plus IK and once with the new stair motions, and trained a tracking policy on each.

before walk motion + IK

Unreal referenceTracking in simulation

after dedicated stair motions

Unreal referenceTracking in simulation

Comparing the two, the movement with the dedicated stair motions is clearly more natural, both in the Unreal reference and in the tracked result. With this, we had a way to produce reasonable stair data in Unreal.

Stair benchmark

The remaining question was whether this data is actually better than the alternatives. To answer it, we focused on stairs and built a full pipeline in four steps:

Terrain generation: randomized spiral and linear staircases with random step width and height.
Automatic data collection with Unreal's auto navigation.
Terrain-aware multi-motion tracking (teacher policy).
Terrain-aware locomotion (student policy, distilled from 3 with DAgger).

Steps 1–2: procedural stair generation and automatic navigation in Unreal.

Once set up, steps 1 and 2 need no human in the loop at all: a new staircase is generated, the G1 navigates it on its own, and its motion is recorded together with the terrain. As a result, it collects about six hours of motion data per hour of wall-clock time, far faster than recording and processing videos. The numbers below summarize the results of the following steps.

~6 h / hmotion data collected per wall-clock hour

94%teacher success (training data)

87.5%student success (training data)

~50%student success on unseen stairs

Teacher: terrain-aware multi-motion tracking

For step 3, we trained the teacher, a multi-motion tracking policy that sees the reference motion and the terrain around it. As training data, we collected 200 motions on linear staircases with varied step count, width and height. On this training data, the teacher reaches a 94% success rate.

success

failure

Looking at the failures more closely, most of them are ascending motions where the blend space output drifts far from the anchor motions.

Student: terrain-aware locomotion

For step 4, the student is distilled from the teacher with DAgger. Unlike the teacher, it only sees the root of the reference motion and the terrain map, so given just a root trajectory it can follow it. On the training data, the student reaches 87.5% success. This is the policy we would actually deploy, because at test time there is no full-body reference motion, only where the robot should go and what the terrain looks like.

success

failure

How well does the student generalize?

However, success on the training data only shows that the student can reproduce the staircases it was trained on. What really matters is whether it handles staircases it has never seen, so we built a benchmark with a grid of step heights and depths:

Step height	15–35 cm, in 5 cm increments
Step depth	height + 5 cm – 70 cm, in 5 cm increments
Number of steps	6
Success	reaching the final goal

Student generalization results — Student success rate per step height on the benchmark.

As the chart shows, overall success is only about 50% (43 / 80 episodes), far below the 87.5% on the training data, and it drops as steps get taller (ascending 73% → 14%, descending 80% → 0%). In other words, the student works on staircases similar to its training data, but not much beyond them.

Baseline: Hiking in the Wild

How does this compare to a method without any Unreal data? For comparison, we took the publicly released checkpoint of Hiking in the Wild, an advanced form of AMP, and evaluated it on the same benchmark without any retraining. It reaches about 94% success.

The public Hiking in the Wild checkpoint on the stair benchmark.

This was the deciding result. Hiking in the Wild needs none of the Unreal pipeline, no retargeting, custom IK, recorded stair motions or data generation, and still clearly outperforms our student on the same staircases.

Scene-Aware Humanoid Locomotion