The refinement pipeline

The refinement pipeline

The ActiScale pipeline turns raw collector footage into training-ready action data in six steps: ingest, filter, segment, pose & contact, narrate & label, and QC & package to LeRobot / RLDS.

Every lab can hire people to film. Almost none can turn what they film into something a model can train on. This is where that happens.

Last updated:

How does the pipeline work? Key facts

  • ✓6 steps: Ingest, Filter, Segment, Pose & contact, Narrate & label, QC & package.
  • ✓Automatic rejection: blur, drift, wrong-task and duplicate footage is rejected at the filter step.
  • ✓Per-frame extraction: hand and body keypoints and object contact events; natural-language narration and action labels on each clip.
  • ✓Output: tiered peer review, then packaged to LeRobot / RLDS and shipped.

Ingest

Video + sensor streams land with provenance and consent attached.

Filter

Blur, drift, wrong-task and duplicate footage is rejected automatically.

Segment

Continuous footage is cut into discrete, task-aligned clips.

Pose & contact

Hand and body keypoints, object contact events, extracted per frame.

Narrate & label

Natural-language narration and action labels applied to each clip.

QC & package

Tiered peer review, then packaged to LeRobot / RLDS and shipped.

See the data

What does one hour of refined data look like?

Real egocentric captures, refined into per-hand mesh, narration and action labels — the structure a model can train on.

Raw egocentric capture

Wide-FOV head-mounted video exactly as recorded — the raw input, before any refinement.

Hand mesh & narration

Per-hand mesh (blue = right, orange = left) tracked on the video, with natural-language narration and action labels.

Tool use & action labels

The same refinement on tool-use tasks across a whole room — hand mesh, narration and per-clip action labels.

Why is the refinement pipeline the product?

Automation absorbs more of the human QC as we scale, so the cost per delivered hour falls over time. The refinement — not the footage — is the defensible, valuable part.

Need demonstration data?

Book a 30-minute data review. We'll scope your task taxonomy and send a free sample slice.

Request a sample slice