MoMo: Dial Motion Mode in Robot Manipulation with Spatiotemporal Action Tokenization

Robots have gotten decent at picking things up or turning knobs, but they've mostly done it one way: whatever way the training data showed them. Apple's ML research team, led by Yuhan Hu and colleagues, is trying to separate the 'what' from the 'how' — teaching a robot to grab a cup gently or grab it briskly, using the same underlying skill but a different execution style.

Their system, called MoMo, works in two stages. First, a spatiotemporal action tokenizer breaks down robot motion into chunks that capture timing and movement patterns, not just end positions. Then a behavior-cloning transformer takes in the task plus a continuous 'motion-mode' knob and generates actions that match both. Turn that knob one way and you get slow, steady, careful movements. Turn it the other way and the robot moves fast and dynamic, with measurable differences in joint speed, acceleration, and even the angle its gripper approaches an object from.

The team tested this across six real-robot manipulation tasks, and human raters could reliably tell the difference between the modes just by watching. That's a meaningful result on its own — it means the style differences aren't cosmetic, they're perceptible and consistent. But the more interesting finding is what happened when they only showed the robot one style of demonstration for a given task, then asked it to perform that same task in a mode it had never seen. MoMo pulled it off, adapting its execution style while mostly keeping the task successful.

That transfer is the real point here. It suggests motion mode isn't tangled up with any specific task — it's something closer to a reusable behavioral dial that can be applied across different jobs a robot might do. Instead of collecting separate training data for 'careful pouring' and 'careful stacking' and 'careful screwing,' you could in theory teach 'careful' once and let it generalize. Apple frames this as evidence of compositional generalization, which is a fancy way of saying the robot learned to mix and match things it wasn't explicitly taught to mix.

It's a narrow study — six tasks, real-robot hardware, one research group's framework — but the idea has legs. Robots operating in homes or warehouses will need to shift from delicate to forceful depending on context, without needing a fresh dataset every time the situation changes. MoMo is a small step toward robots that adjust their demeanor the way people do, without anyone reprogramming them from scratch.