Add composed spatial-generalization benchmarks

[Yuan-Xinyi]

Description

Extends the strict spatial-generalization benchmark introduced by #772 with deterministic multi-factor, distance-shell, and bilateral-role domains.

• adds sampling_pattern: stratified for deterministic multi-factor coverage and sampling_pattern: workspace_shell for ordered near/mid/far XY-offset radii
• records shell radii and row-level spatial buckets in the case manifest
• aggregates bucket success rates, sample counts, and failure stages without adding report tables
• adds planar task-frame and skill-appropriate composed object/target/support domains for all 13 evaluated Atomic Skills
• adds dual-arm bilateral role-mirror domains by swapping configured left/right control-part bindings
• keeps uniform sampling as the default and preserves the YAML-config benchmark entry point

Atomic Skill	Workspace/relative shell	Shared workspace domain	Composed domain	Bilateral mirror	Simultaneously varied dimensions
PickUp, Pour, AxisAlign	object_workspace_shell	task_frame_planar	object_planar_pose		subject XY + subject yaw
MoveHeldObject	target_relative_shell	task_frame_planar	object_target_spatial_pose		subject XY/yaw + target XYZ/yaw/roll/pitch
Place	target_relative_shell	task_frame_planar	object_target_planar_pose		subject XY/yaw + target XY/yaw
PushObject	target_relative_shell	task_frame_planar	object_target_position		subject XY/yaw + target XY
Press, Twist, Slide, OpenDoor	articulation_workspace_shell	task_frame_planar	articulation_spatial_pose		articulation XYZ/yaw/roll/pitch
HandOver	target_relative_shell	task_frame_planar	object_target_position	✓	subject XY/yaw + delivery XYZ
CoordinatedPickment	target_relative_shell	task_frame_planar	object_target_spatial_pose	✓	subject XY/yaw + target XYZ/yaw/roll/pitch
CoordinatedPlacement	support_relative_shell	task_frame_planar	object_support_target_task_frame_pose	✓	subject XY/yaw + support XY/yaw + target XYZ/yaw + task-frame XY/yaw

This is a stacked PR and depends on #772. It adds evaluation definitions, manifest/report metadata, and validation only; no benchmark results were run or committed.

Type of change

• Enhancement (non-breaking change which improves an existing functionality)

Screenshots

Not applicable.

Validation

• black . (black==26.3.1)
• black --check --diff --color ./
• python docs/scripts/check_api_docs.py — 2613/2613 exports documented
• pytest -q tests/benchmark/motion_generation — 326 passed, 1 skipped

Checklist

• I have run the black . command to format the code base.
• I reviewed affected documentation and agent context. The benchmark README was updated; the context router reports no affected topics because this change is benchmark-only.
• Public API changes are reflected in the API docs; no public package API was added.
• I have added tests that prove the feature works.
• Dependencies have been updated, if applicable; no dependency changes were required.

[greptile-apps]

Confidence Score: 4/5

[Medium impact] Adds spatial-generalization benchmarks and motion-planning features.

Restore success and shell summaries for the five contact skills before merging.

Findings

1.  Five skills lose their results ▶

Fix with agent prompt

### Issue 1
scripts/benchmark/motion_generation/generalization.py:216-223
`Press`, `Twist`, `Slide`, `OpenDoor`, and `PushObject` now set `selected_planner_backend_used=False`. This filter removes their measured runs from the generalization summaries, leaving success rates as `None` and shell bucket rates, counts, and failure stages empty-even after successful runs. The other report datasets exclude them too, with no separate summary preserving their results.

Keep these runs out of planner rankings, but report their skill and shell results under `effective_motion_algorithm`.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Adds composed pose domains, distance shells, and mirrored arm roles to the spatial benchmarks. It also changes motion timing, grasp commands, and planner attribution.

• Benchmark suites combine pose shifts and add workspace shells.
• Reports group shell results and separate planner-attributed cases.
• Joint and Cartesian paths expand to meet motion limits.
• Contact skills keep their exact paths through interpolation.
• Composite actions keep planned durations and observed hand positions.
• Runtime commands keep one duration on the simulator time grid.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Atomic skill case] --> B{Motion strategy}
  B -->|motion_gen| C[Selected planner]
  B -->|ik_interp| D[Exact Cartesian path]
  C --> E[Measured trial records]
  D --> E
  E --> F{Planner attribution}
  F -->|Included| G[Success and shell summaries]
  F -->|Excluded| H[Counts only; measured summaries lost]

Loading

_{Reviews (6) · Last reviewed commit: "Fix Atomic Skill trajectory timing and c..." · Reviewed by Greptile}

[greptile-apps]

Five skills lose their results

Press, Twist, Slide, OpenDoor, and PushObject now set selected_planner_backend_used=False. This filter removes their measured runs from the generalization summaries, leaving success rates as None and shell bucket rates, counts, and failure stages empty—even after successful runs. The other report datasets exclude them too, with no separate summary preserving their results.

Keep these runs out of planner rankings, but report their skill and shell results under effective_motion_algorithm.

Prompt To Fix With AI

This is a comment left during a code review.
Path: scripts/benchmark/motion_generation/generalization.py
Line: 216-223

Comment:
**Five skills lose their results**

`Press`, `Twist`, `Slide`, `OpenDoor`, and `PushObject` now set `selected_planner_backend_used=False`. This filter removes their measured runs from the generalization summaries, leaving success rates as `None` and shell bucket rates, counts, and failure stages empty—even after successful runs. The other report datasets exclude them too, with no separate summary preserving their results.

Keep these runs out of planner rankings, but report their skill and shell results under `effective_motion_algorithm`.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.