Introducing EgoVerse: an ecosystem for robot learning from egocentric human data. Built and tested by 4 research labs + 3 industry partners, EgoVerse enables both science and scaling 1300+ hrs, 240 scenes, 2000+ tasks, and growing Dataset design, findings, and ecosystem 🧵 https://t.co/qagpce0vxl
EgoVerse Launches Ecosystem for Robot Learning from Egocentric Data
· Updated
EgoVerse, an ecosystem for robot learning from egocentric human data, has been released by Georgia Tech, Stanford, UC San Diego, ETH Zurich, Meta Reality Labs, Scale AI, and Mecka AI (physical AI data infrastructure). The dataset spans 1,300+ hours of egocentric video with hand and camera tracking across 240 scenes and 2,000+ tasks, with cloud infrastructure and transfer algorithms.
Across four independent academic labs, a consistent finding emerged: a small set of aligned human and robot data in the same task acts as a bridge between large-scale diverse human data and downstream robot performance, echoing EgoScale and Data Analogies. This reshapes data collection strategy for robot learning.
You can capture egocentric data using Project Aria glasses (Meta's egocentric capture device) or an iPhone app from Mecka AI and contribute it to the ecosystem. Focus on tasks where aligned human-robot demonstration is feasible — even modest matched data improves robot performance.
Danfei Xu
@danfei_xu
129retweets
View on X



