NVIDIA Outlines Technical Roadmap for Scaling Robot Dexterity and Physical AGI
· Updated
NVIDIA's Jim Fan delivered Robotics: Endgame, a 20-minute talk at Sequoia AI Ascent that lays out a roadmap for Physical AGI as a deliberate parallel to the LLM success story. The talk is positioned as the sequel to his Physical Turing Test talk from a year earlier at the same conference.
Talk title
Robotics: Endgame
Speaker
Jim Fan (NVIDIA)
Venue
Sequoia AI Ascent
Length
~20 minutes
Key concepts introduced
World Action Models, EgoScale, Dexterity Scaling Law, DreamDojo, Physical RL
The roadmap walks through several pieces in order: why current Vision-Language-Action models (VLAs) fall short, video world models as a second pretraining paradigm, World Action Models (WAM), strategies for robot data collection and an FSD-style physical data flywheel for robot manipulation, EgoScale and a newly discovered Dexterity Scaling Law, Physical RL as the last-mile step, and DreamDojo — an end-to-end neural physics engine for scaling reinforcement learning in silico.
You can watch the full talk on YouTube via the link in Fan's announcement tweet. The chapter markers in the tweet map directly to the segments above, so viewers can jump straight to a specific argument — DreamDojo at 15:39, World Action Models at 06:09, or the Civilizational Technology Tree predictions near the end of the talk.
I promise this will be the best 20 min you spend today! Robotics: Endgame, the sequel to my last year's Sequoia AI Ascent talk, "Physical Turing Test". I laid out the roadmap for solving Physical AGI as a simple parallel to the LLM success story. Be a good scientist, copy homework ;)
And stay till the end, more easter eggs and predictions for your polymarket!
00:30 DGX-1 origin story at OpenAI, I was there in 2016 signing with Jensen and Elon. Heading to the Computer History Museum!
01:42 The Great Parallel
03:31 Robotics, the Endgame
03:39 Why VLAs fall short
04:32 Video world models as the 2nd pretraining paradigm
06:09 World Action Models (WAM)
07:46 Strategies for robot data collection and the FSD equivalent to physical data flywheel for robot manipulation
11:06 EgoScale and the Dexterity Scaling Law we discovered recently
14:00 Physical RL: bridging the last mile
15:39 DreamDojo: an end-to-end neural physics engine for scaling RL in silico
17:00 Civilizational Technology Tree and my predictions for the near future. Spoiler: it's closer than you think.
Thanks to my friends at Sequoia for inviting me back to AI Ascent this year! I had a blast! Last year's talk is attached in the thread if you missed it.
Robotics: Endgame is a roadmap talk delivered by NVIDIA's Jim Fan at Sequoia AI Ascent. It lays out his view of how Physical AGI gets solved, structured as a parallel to the LLM success story. The talk is the sequel to his earlier Physical Turing Test talk at the same conference a year prior, and runs about 20 minutes.
World Action Models (WAM) is the paradigm Jim Fan introduces in Robotics: Endgame as the next step beyond current Vision-Language-Action models for robotics. The chapter on WAM appears around the 6-minute mark of the talk, after a section explaining why current VLAs fall short and a section on video world models as a second pretraining paradigm.
DreamDojo is described in the talk as an end-to-end neural physics engine for scaling reinforcement learning in silico — that is, in simulation rather than on physical robots. Jim Fan covers DreamDojo around the 15:39 mark of the Robotics: Endgame talk, framing it as the infrastructure layer that makes Physical RL scaling tractable.
Jim Fan describes EgoScale and a Dexterity Scaling Law as a recently discovered result, presented around the 11-minute mark. The framing in the tweet is that dexterity itself can be scaled like other capabilities once you have the right data collection strategy and a physical data flywheel comparable to what FSD uses for autonomous driving.
