mjlab manipulation benchmark — task previews

37 clips · 3.0s · 30fps · random agent · seed 0
Random agent: this shows scene layout and that the env steps — NOT task solvability.

arm_gripper 29

Mjlab-Axial-Extract-Franka
articulationtier 3 · precision_graspcontact-rich
obs 60 · act 8 · episode 1e+09s
notes

Motion profile: FRICTION-BREAKAWAY EXTRACTION — pinch the plug head top-down, exceed the ~4 N static-friction breakaway, then guide the plug straight up out of the bore (metaworld peg-unplug / disassemble lineage). Inverse of the drawer's low-friction hooked glide. Structural (smoke) gate only; reward shaping not yet train-validated.

Mjlab-Cage-Drag-Franka
non_prehensiletier 2 · mild_contactcontact-rich
obs 60 · act 8 · episode 1e+09s
notes

Motion profile: FORM-CLOSURE (CAGING) TRANSPORT — straddle the cube with OPEN fingers and translate; success carries the benchmark's first episode-long NEGATIVE constraint (min aperture over the episode must never dip below the cage threshold), so a pinch-and-drag solution scores zero. First NON_PREHENSILE task. Structural (smoke) gate only; reward shaping not yet train-validated.

Mjlab-Drag-Pull-Franka
planar_pushtier 2 · mild_contact
obs 60 · act 8 · episode 1e+09s
notes

Motion profile: PLANAR DRAG-PULL — engagement inverted vs push: the object spawns FAR and the goal sits NEAR, so the fingertips reach over and behind the object and the arm retracts while keeping far-face contact (metaworld pull/push-back lineage). Structural (smoke) gate only; reward shaping not yet train-validated.

Mjlab-Edge-Grasp-Franka
pick_placetier 3 · precision_graspcontact-rich
obs 60 · act 8 · episode 1e+09s
notes

Motion profile: EXTRINSIC-DEXTERITY EDGE GRASP — the plate is unspannable and, flat, unpinchable; slide it over the ledge edge until it overhangs, then pinch the exposed 16 mm thickness and lift clear (edge-grasp literature, Chavan-Dafle lineage). Two-stage: a slide phase whose only purpose is to manufacture graspability. Structural (smoke) gate only; reward shaping not yet train-validated.

Mjlab-Flip-Switch-Franka
articulationtier 2 · mild_contactcontact-rich
obs 60 · act 8 · episode 1e+09s
notes

Motion profile: BALLISTIC COMMIT past a detent. A spring pushes the toggle back until it crosses centre, so quasi-static servoing fails and success is a state flip, not a displacement threshold (contrast Push-Button, a compliant press). Structural (smoke) gate only; reward shaping not yet train-validated.

Mjlab-Lift-Cube-Franka
pick_placetier 3 · precision_graspcontact-rich
obs 60 · act 8 · episode 1e+09s
notes

Grasp-and-lift; the canonical fragile task in FINDINGS.md (catastrophic forgetting under regularization-only CL).

Mjlab-Lift-Cylinder-Franka
pick_placetier 3 · precision_graspcontact-rich
obs 60 · act 8 · episode 1e+09s
notes

Grasp-and-lift a cylinder; rolls and lacks flat faces, so grasp alignment is less forgiving than the cube (grasp-geometry generalization).

Mjlab-Lift-Ellipsoid-Franka
pick_placetier 3 · precision_graspcontact-rich
obs 60 · act 8 · episode 1e+09s
notes

Grasp-and-lift an elongated ellipsoid; grasp success depends on approach orientation (grasp across the short axis).

Mjlab-Lift-Sphere-Franka
pick_placetier 3 · precision_graspcontact-rich
obs 60 · act 8 · episode 1e+09s
notes

Grasp-and-lift a sphere; rolls, no flat faces — hardest free-object grasp geometry (grasp-geometry generalization).

Mjlab-Open-Door-Franka
articulationtier 2 · mild_contactcontact-rich
obs 60 · act 8 · episode 1e+09s
Mjlab-Open-Drawer-Franka
articulationtier 2 · mild_contactcontact-rich
obs 60 · act 8 · episode 1e+09s
Mjlab-Open-Lid-Franka
articulationtier 2 · mild_contactcontact-rich
obs 60 · act 8 · episode 1e+09s
notes

Motion profile: VERTICAL ARC UNDER GRAVITY. Gravity opposes the motion throughout and the lid falls shut if released, unlike the door's gravity-neutral vertical-axis swing. NOTE: runs with gravity ENABLED, unlike the other articulation tasks. Structural (smoke) gate only; reward shaping not yet train-validated.

Mjlab-Peg-Insertion-Franka
insertiontier 4 · dexterouscontact-rich
obs 51 · act 8 · episode 1e+09s
notes

Insert a peg into a hole board: grasp + tight alignment into a ~3cm hole. Reuses the Stack MDP with tighter xy tolerance. New 'insertion' skill; most fragile tier (multi-contact precision).

Mjlab-Pivot-Lift-Franka
pick_placetier 3 · precision_graspcontact-rich
obs 60 · act 8 · episode 1e+09s
notes

Motion profile: PIVOT-AGAINST-WALL GRASP — the flat board is ungraspable on open ground; push it INTO the wall to pivot it up onto an edge, then pinch the exposed thickness and carry it to an airborne goal (Zhou & Held extrinsic-dexterity lineage). The wall is a per-env static fixture owned by the command. Structural (smoke) gate only; reward shaping not yet train-validated.

Mjlab-Place-In-Container-Franka
pick_placetier 3 · precision_graspcontact-rich
obs 60 · act 8 · episode 1e+09s
notes

Success shape: CONTAINMENT + RELEASE (inside the footprint, below the rim, settled) rather than the stable-contact predicate used by Stack. The bin walls also make the approach non-monotonic — a straight-line reach to the goal hits a wall. Structural (smoke) gate only; reward shaping not yet train-validated.

Mjlab-Push-Button-Franka
articulationtier 1 · planar
obs 60 · act 8 · episode 1e+09s
notes

Press a button; forgiving, retained near-1.0 through sequences.

Mjlab-Push-Cuboid-Franka
planar_pushtier 2 · mild_contact
obs 60 · act 8 · episode 1e+09s
notes

Task-0 fragility case in FINDINGS.md (retains ~0.6-0.84).

Mjlab-Push-Disc-Franka
planar_pushtier 2 · mild_contact
obs 60 · act 8 · episode 1e+09s
Mjlab-Push-Flap-Franka
articulationtier 2 · mild_contact
obs 60 · act 8 · episode 1e+09s
notes

Motion profile: NON-PREHENSILE HINGE-ARC PUSH — same vertical hinge as Open-Door but the panel has NO handle: fingertips face-push and must re-orient along the arc as the contact normal rotates (metaworld faucet/door-close lineage, ~15 external tasks collapse onto this one profile). Structural (smoke) gate only; reward shaping not yet train-validated.

Mjlab-Reach-Target-Franka
reachtier 1 · planar
obs 38 · act 8 · episode 1e+09s
notes

Move the gripper to a sampled 3D target; dense monotonic reward, no contact. Anchors the low-fragility end of the axis (new 'reach' skill family).

Mjlab-Reorient-Object-Franka
pick_placetier 3 · precision_graspcontact-rich
obs 60 · act 8 · episode 1e+09s
notes

Success shape: ORIENTATION, not position — the only Class A task scored on angular alignment (stand a lying cylinder upright). Position enters only as a loose anti-fling drift bound. Structural (smoke) gate only; reward shaping not yet train-validated.

Mjlab-Rotate-Valve-Franka
articulationtier 4 · dexterouscontact-rich
obs 60 · act 8 · episode 1e+09s
notes

Motion profile: MULTI-CYCLE ROTATION WITH REGRASP. The 270deg target exceeds the wrist range from any single grasp, so the policy must turn, release, re-grasp the opposite spoke and continue — the only Class A task needing a regrasp cycle. Structural (smoke) gate only; reward shaping not yet train-validated.

Mjlab-Slide-Window-Franka
articulationtier 2 · mild_contact
obs 60 · act 8 · episode 1e+09s
notes

Motion profile: LATERAL FACE-PUSH. Same joint type as Open-Drawer but along the robot's lateral axis, and the drawer's proven fingertip-hook strategy does not transfer (you push a face, not hook a bar). Structural (smoke) gate only; reward shaping not yet train-validated.

Mjlab-Stack-Cube-Franka
pick_placetier 3 · precision_graspcontact-rich
obs 51 · act 8 · episode 1e+09s
notes

Stack a cube on a cuboid base: grasp + precise place + release. Dynamic goal (tracks the base object). Higher-fragility pick_place than lift.

Mjlab-Strike-Slide-Franka
planar_pushtier 3 · precision_grasp
obs 60 · act 8 · episode 1e+09s
notes

Motion profile: IMPULSIVE STRIKE-TO-SLIDE — the goal band (x 0.88-1.05) sits beyond the arm's ~0.85 m stretch, so the puck must be struck and released; contact ends before arrival (FetchSlide lineage). Tier 3 by the instant-criticality criterion: no grasp is involved, but the impulse is all-or-nothing at release with no recovery. Structural (smoke) gate only; reward shaping not yet train-validated.

Mjlab-Throw-To-Bin-Franka
pick_placetier 3 · precision_graspcontact-rich
obs 60 · act 8 · episode 1e+09s
notes

Motion profile: THROW-TO-BIN — grasp the cube, accelerate, and RELEASE ON TIME so the ballistic arc lands in a bin beyond the arm's stretch (TossingBot lineage). Reuses the place-in-container base: same containment+settled predicate, and the bin band beyond reach (x 0.78-0.90) is what turns placing into throwing. Structural (smoke) gate only; reward shaping not yet train-validated.

Mjlab-Tool-Pull-Franka
tool_usetier 4 · dexterouscontact-rich
obs 69 · act 8 · episode 1e+09s
notes

Motion profile: TWO-STAGE TOOL USE — grasp a stick, then drag a puck that spawns beyond direct reach back into the near zone. Contact that matters happens at the tool tip, not the fingertips. First TOOL_USE task in the benchmark. Structural (smoke) gate only; reward shaping not yet train-validated.

Mjlab-Topple-Block-Franka
non_prehensiletier 2 · mild_contactcontact-rich
obs 60 · act 8 · episode 1e+09s
notes

Motion profile: NON-PREHENSILE TOPPLE — every block face exceeds the 0.08 m aperture so force closure is impossible by construction; poke above the centre of mass to tip it over an edge until the body x-axis lands vertical (either sign: symmetric predicate). Orientation-scored like Reorient-Object but grasp-free and ballistic past the tipping point. Structural (smoke) gate only; reward shaping not yet train-validated.

Mjlab-Turn-Lever-Franka
articulationtier 2 · mild_contactcontact-rich
obs 60 · act 8 · episode 1e+09s
notes

Motion profile: WRIST ROTATION about the approach axis. The lever hinge points at the robot, so turning it rotates the wrist rather than pulling with the arm (contrast Open-Door, a whole-arm hinge arc). Structural (smoke) gate only; reward shaping not yet train-validated.

arm_hand 3

Mjlab-Lift-Cube-Franka-Leap
pick_placetier 3 · precision_graspcontact-rich
obs 88 · act 23 · episode 1e+09s
notes

Franka arm + LEAP hand grasping and lifting a cube (dexterous grasp on a full arm).

Mjlab-Reach-Target-Franka-Leap
reachtier 1 · planar
obs 66 · act 23 · episode 1e+09s
notes

Franka arm + LEAP hand reaching to a 3D target. First Class-B (arm + dexterous hand) task; reuses the arm reach base.

Mjlab-Stack-Cube-Franka-Leap
pick_placetier 3 · precision_graspcontact-rich
obs 79 · act 23 · episode 1e+09s
notes

Franka arm + LEAP hand stacking a cube on a cuboid base.

floating_hand 5

Mjlab-Lift-Cube-Leap
pick_placetier 3 · precision_graspcontact-rich
obs 86 · act 22 · episode 1e+09s
notes

Floating LEAP hand grasping and lifting a cube with its fingers (dexterous grasp). Reuses the arm lift base with the hand embodiment.

Mjlab-Lift-Sphere-Leap
pick_placetier 3 · precision_graspcontact-rich
obs 86 · act 22 · episode 1e+09s
notes

Floating LEAP hand grasping and lifting a sphere (hardest grasp geometry).

Mjlab-Peg-Insertion-Leap
insertiontier 4 · dexterouscontact-rich
obs 77 · act 22 · episode 1e+09s
notes

Floating LEAP hand inserting a peg into a hole board.

Mjlab-Reach-Target-Leap
reachtier 1 · planar
obs 64 · act 22 · episode 1e+09s
notes

Floating LEAP hand reaching its palm to a 3D target (actuated 6-DoF base). First Class-C task; reuses the arm reach base with the hand embodiment.

Mjlab-Stack-Cube-Leap
pick_placetier 3 · precision_graspcontact-rich
obs 77 · act 22 · episode 1e+09s
notes

Floating LEAP hand stacking a cube on a cuboid base.