Add strict spatial generalization suites for task skills

[Yuan-Xinyi]

Description

Summary

• add strict spatial-generalization coverage for 13 task-level Atomic Skills:
  • Franka: PickUp, MoveHeldObject, Place, Press, Slide, Twist, OpenDoor, Pour, AxisAlign, and PushObject
  • dual UR5: HandOver, CoordinatedPickment, and CoordinatedPlacement
• evaluate isolated spatial domains with a matching nominal population
• cover planar and selected 3D object/goal poses through YAML configuration
• keep assets, physics, robot, task semantics, seeds, and success criteria fixed within each track
• use deterministic B=8 perturbation rows and B=1 nominal rows for seeds 11, 23, and 47
• freeze generated poses in the case manifest and restore them before every planner replay

Motivation and context

This establishes a strict spatial-generalization test loop on top of #766 using #712's formal benchmark suite/config entry and atomic-skill-v1 outcome schema.

Refs #766.

Suites

• atomic_franka_pgi_spatial_generalization_v5
• atomic_dual_ur5_pgi_hand_over_spatial_generalization_v1
• atomic_dual_ur5_pgi_coordinated_pickment_spatial_generalization_v3
• atomic_dual_ur5_pgi_coordinated_placement_spatial_generalization_v3

Per-task spatial-generalization coverage

Every task includes a fixed-scene nominal baseline. Each robustness population varies one spatial domain at a time.

Task / Atomic Skill	Subject XY ±3 cm	Subject Z ±2 cm	Subject yaw ±30°	Subject roll/pitch ±10°	Relative goal XY ±2 cm	Target Z ±2 cm	Target yaw ±30°	Target roll/pitch ±10°	Support XY ±3 cm	Support yaw ±30°	Task-frame XY ±3 cm	Task-frame yaw ±10°
PickUp	✓		✓
Pour	✓		✓
AxisAlign	✓		✓
MoveHeldObject	✓		✓		✓	✓	✓	✓
Place	✓		✓		✓		✓
PushObject	✓		✓		✓
Press	✓	✓	✓	✓
Twist	✓	✓	✓	✓
Slide	✓	✓	✓	✓
OpenDoor	✓	✓	✓	✓
HandOver	✓		✓		✓
CoordinatedPickment	✓		✓		✓	✓	✓	✓
CoordinatedPlacement	✓		✓		✓	✓	✓		✓	✓	✓	✓

Dimension semantics:

• Subject pose changes the initial rigid object or fixed-base articulation root. For CoordinatedPlacement, subject denotes the placing object.
• Relative goal / target pose changes the Cartesian object goal relative to its nominal location and orientation.
• Support pose changes the support object independently in CoordinatedPlacement.
• Task-frame pose applies one shared planar rigid transform to both coordinated-placement objects and their goals, preserving the relative object layout.
• Subject Z and roll/pitch are enabled for fixed-base articulations. Target Z and roll/pitch are enabled when the commanded object remains held or moves together with its support at that target.

Config interface

Every domain is configured under tracks[].domains[].overrides.randomization:

randomization:
  enabled: true
  mode: spatial_pose
  pose_batch_size: 8
  object_translation_jitter_m: [0.03, 0.03, 0.02]
  object_yaw_jitter_rad: 0.523599
  object_roll_pitch_jitter_rad: [0.174533, 0.174533]
  target_translation_jitter_m: [0.02, 0.02, 0.02]
  target_yaw_jitter_rad: 0.523599
  target_roll_pitch_jitter_rad: [0.174533, 0.174533]
  support_object_translation_jitter_m: [0.03, 0.03, 0.0]
  support_object_yaw_jitter_rad: 0.523599
  support_object_roll_pitch_jitter_rad: [0.0, 0.0]
  task_frame_translation_jitter_m: [0.03, 0.03, 0.0]
  task_frame_yaw_jitter_rad: 0.174533
  task_frame_roll_pitch_jitter_rad: [0.0, 0.0]
  seed_stride: 1

translation, planar_pose, and spatial_pose progressively enable translation, yaw, and roll/pitch controls. Each shipped domain sets only its active range; the other ranges resolve to zero. Resolved ranges and per-row poses are recorded in benchmark artifacts.

The sample run-atomic-action CLI uses direct arguments with internal scene defaults, following Yueci's review on #768.

Dependencies

This PR targets xinyi/pr712-with-main, synchronized with #712's evaluation-standard commit. It adds no package dependencies.

Type of change

• Enhancement (non-breaking change which improves existing functionality)

Validation

• black . with Black 26.3.1: 1,233 files unchanged after formatting
• motion-generation benchmark tests: 306 passed, 1 skipped
• all four spatial suites loaded through the production load_suite() path
• python docs/scripts/check_api_docs.py: 2487/2487 exports documented
• project-context affected check against the PR base: no affected topics
• no live simulator/model evaluation or result generation was performed

The focused tests used DexSim 0.5.0 with a process-local RTRenderMode compatibility enum.

Screenshots

Not applicable.

Checklist

• I have run the black . command to format the code base.
• I reviewed affected documentation and agent context; README was updated and the context checker reported no affected topics.
• Public API documentation coverage is aligned.
• I have added tests that prove the feature works.
• Dependencies have been reviewed.

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The reviewed changes appear safe to merge; no blocking issue remains.

Summary

Adds spatial-generalization suites for 13 task skills, with fixed nominal comparisons, deterministic pose batches, and saved poses for planner replay.

• Spatial suites compare fixed nominal cases with isolated pose changes.
• Benchmark pose settings can vary object, goal, support, and task-frame rotation.
• Skill scores can use measurements from named action phases and settling.
• Shared scene and geometry tools support reusable scene building and checks.
• A shared robot catalog powers robot discovery and the list command.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Measured object poses] --> B[Build grasps]
  A --> C[Remove shared task frame]
  C --> D[Build placement targets]
  D --> E[Restore shared task frame]
  E --> F[Save targets for replay]

Loading

_{Reviews (11) · Last reviewed commit: "fix(benchmark): transform placement targ..." · Reviewed by Greptile}