HeadsUpAI

NVIDIA Open Sources Physical AI Agent Skills for Robotics and Manufacturing

· Updated

NVIDIA builds on the Agent Skills launch by releasing open-source instructions that bring agentic automation to physical systems. These skills allow agents to directly invoke NVIDIA libraries—including Omniverse and Isaac—to execute engineering tasks. The release joins the Alpamayo 2 Super release to provide agents with the reasoning models and tools needed to automate simulation and training.
Agent Skills
Neural Reconstruction, Video Augmentation, Defect Image Generation
Core Frameworks
Omniverse, Cosmos, Isaac, Metropolis, Alpamayo, Jetson
Security Tools
NemoClaw, OpenShell
Integrations
Hermes Skills Hub, Microsoft, CoreWeave, Nebius
Industry Partners
TSMC, Foxconn, SK hynix, and others

This shift moves AI agents beyond digital tasks into physical-world orchestration. It turns manual workflows into agent-executable instructions, a transition highlighted by Foxconn's hospital agent launch. These tools help realize the roadmap for scaling physical intelligence through simulation and synthetic data (AI-generated information used for training) across factories and labs.

Skills are available on GitHub for use with agents like Claude Code. The collection includes specialized skills for neural reconstruction and defect generation. For secure deployment, the NVIDIA NemoClaw blueprint and NVIDIA OpenShell runtime provide policy-based governance for agents on local or cloud hardware.

View the full update on nvidianews.nvidia.com
NVIDIA Newsroom
NVIDIA Newsroom
@nvidianewsroom
X

At #NVIDIAGTC, Jensen announced a major open source collection of physical AI agent skills and tools, available now on GitHub. The new collection helps agents tap into NVIDIA technologies including Omniverse libraries, Cosmos world foundation models, Isaac simulation frameworks, Metropolis, Alpamayo and Jetson, with skills spanning autonomous vehicles, robotics, vision AI, industrial digital twins and healthcare. Read the release ➡️ https://t.co/TGnxEGt5TU

95retweets509likes
View on X

Still wondering? A few quick answers below.

They are open-source instruction sets that allow AI agents to use NVIDIA software libraries for physical-world tasks. By turning complex workflows into agent-callable tools, they enable agents to autonomously manage the data generation, simulation, and training pipelines required for robotics, autonomous vehicles, and industrial digital twins.

The skills are provided as repeatable instructions that agents like Claude Code or Cursor can follow. They define which NVIDIA tools to call, what outputs to generate, and how to validate results. This allows developers to describe a physical AI task in natural language and let the agent execute it.

The collection includes skills and tools for Omniverse, Cosmos, Isaac, Metropolis, Alpamayo, and Jetson. These integrations allow agents to operate across the entire NVIDIA physical AI stack, from neural scene reconstruction in simulation to fine-tuning edge AI systems for real-world deployment.

NVIDIA provides the NemoClaw blueprint and OpenShell runtime to ensure agents operate safely. These tools implement policy-based security and privacy governance, allowing developers to set guardrails on agent actions whether they are running on local hardware or in the cloud.

Every HeadsUpAI update is written based on its original source and reviewed before it's published. Read our editorial standards →

Share this update