Skip to content

feat(datasets): move episodes out of a local dataset into a new one - #142

Merged
nicolas-rabault merged 2 commits into
huggingface:mainfrom
ravediamond:feat/move-episodes
Oct 29, 2026
Merged

nicolas-rabault merged 2 commits into
huggingface:mainfrom
ravediamond:feat/move-episodes

Conversation

@ravediamond

Copy link
Copy Markdown
Contributor

Follow-up to the note on #95: "removing episodes by moving them into a separate dataset" as its own small PR.

Problem

After a recording session some takes are bad (failed grasp, arm out of frame, forgotten reset), and there is no way to drop them from the browser. Today that means leaving LeLab for LeRobot's lerobot-edit-dataset.

Approach

A "Move episodes" button in the dataset browser, shown for local datasets with more than one episode. The dialog is prefilled with the episode being viewed and takes a list like 3, 7, 10-12 plus a name. Those episodes go to a new dataset local/<name>, and the episodes left in the source are renumbered. Nothing is deleted, so the moved takes can be merged back or kept apart.

  • POST /datasets/move-episodes wraps LeRobot's delete_episodes, run twice: once for the moved copy, once for the copy that stays. Both are written to a staging dir next to the output, and the source is swapped only once both exist, so a failure leaves the source untouched.
  • The route returns 409 while a recording is running. Empty or negative indices are a 422 at the request model; out-of-range indices, moving every episode, an invalid name or an existing output are a 400. ensure_episode_index runs before the dataset is opened, and the output path is contained under the cache root like merge's.
  • /dataset-frame and /dataset-video are now no-cache (videos revalidate against the FileResponse ETag). Both assumed a frame never changes for a given dataset/episode/index, which a move breaks.

If the dataset is also on the Hub, the local copy no longer matches it after a move.

Testing

  • pytest: new tests cover a real move on a LeRobot-written dataset, invalid requests, an existing output name, a failure mid-move that leaves the source untouched, the 409 while recording, and request validation.
  • vitest (dialog + episode-list parser), eslint, npm run build, pre-commit run --all-files.
  • Moved episodes through the HTTP route on a small recorded video dataset and decoded frames from both results: each kept its own footage, no staging dir left behind.
  • Drove the dialog end to end in headless Chromium.

Tested on Linux (Jetson Orin, aarch64) only. Not tested on macOS, Windows, or a real SO-101 dataset.

ravediamond and others added 2 commits October 29, 2026 15:59
The move dialog picks its destination with the shared dataset picker, so
episodes can go to a new local dataset or be merged into an existing one.
An existing target is merged with the shared merge, which keeps its fps,
features and robot type checks, and is swapped in with the source only after
both copies are built. The Move episodes button uses an arrow leaving a box.

@nicolas-rabault nicolas-rabault left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this! I added a commit on top so episodes can also move into an existing dataset.

@nicolas-rabault
nicolas-rabault merged commit c0a4292 into huggingface:main Oct 29, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants