DiverseUMI

The most task-diverse and environment-diverse UMI dataset in robotics, collected across hundreds of real tables and fully annotated with dense task labels from Argus.

We collect DiverseUMI in several ways at once, each tuned for maximum diversity and efficiency.

Stack

Balancing a clip on a cradle

Left wrist
Right wrist

Freeform

Operators get around 15 objects on a table and perform any tasks they like with no script, and Argus labels the tasks afterwards.

Missed grasp

Retrying a cylinder pickup

Left wrist
Right wrist

Freeform adversarial

The same freeform setup, with episodes full of missed grasps, drops and failed tasks, along with the recoveries that follow.

99% of tasks never repeat. Failures are collected on purpose, and 88% end in a recovery.

Tasks by skill
  1. Pass & place19%
  2. Pick & place17%
  3. Turn & place7.6%
  4. Stack7.3%
  5. Hand over4.3%
  6. Stand up4.0%
  7. Regrasp & place4.0%
  8. Slide3.5%
  9. Insert3.1%
  10. Assemble2.2%
  11. Press1.8%
  12. Flip1.8%
  13. Other24%

Pick and place split by what else happens in the task.

Failures by type
  1. Repeated attempts13%
  2. Missed grasp · passing12%
  3. Missed grasp · placing11%
  4. Dropped · passing11%
  5. Missed grasp · other tasks9.9%
  6. Knocked object6.6%
  7. Dropped · stacking5.5%
  8. Missed grasp · stacking4.9%
  9. Dropped · other tasks4.3%
  10. Dropped · placing2.5%
  11. Failed placement2.3%
  12. Hesitation2.0%
  13. Other16%

Found in 83% of recordings.

Objects per table
  1. <1%0
  2. 15%5
  3. 25%10
  4. 22%15
  5. 17%20
  6. 10%25
  7. 10%30+

From a handful of objects on a table to more than sixty.

Tabletop colors

Each square is 1% of tables.

Every episode is densely annotated by Argus, down to each gripper's opening and 6-DoF pose.

DiverseUMI-100

A 100-hour sample of DiverseUMI, free to download.

Download on Hugging Face

The full dataset

For a larger portion of DiverseUMI: we work with commercial and academic labs on joint research and data collaborations.

Contact us

Related research