NVIDIA Releases Asset Harvester to Turn Driving Video Into 3D Objects

NVIDIANVIDIA

· Updated

NVIDIA researchers open-sourced Asset Harvester, an end-to-end pipeline that extracts manipulable 3D assets from standard autonomous driving video logs. By turning sparse real-world footage into simulation-ready objects, it removes the manual bottleneck of creating diverse training environments for physical AI.

NVIDIA released Asset Harvester, an open-source pipeline that converts raw autonomous driving video into complete 3D assets. It handles "in-the-wild" logs—footage from normal driving—to reconstruct vehicles from sparse views (limited camera angles). This follows the Physical AI Data Factory blueprint for automating synthetic training data generation.

This update addresses data scarcity in world foundation models like Cosmos 3. While those models simulate environments, they need massive libraries of 3D objects for realistic edge cases. Asset Harvester lets developers harvest these assets from existing driving logs rather than building them manually.

You can access the code on GitHub to integrate real-world objects into dynamic simulations. The tool works with NVIDIA NuRec, letting you insert or replace 3D assets within reconstructed scenes. This is essential for testing autonomous agents in diverse virtual environments that mirror the complexity of real-world driving.

View the full update on research.nvidia.com
NVIDIA AI
NVIDIA AI
@NVIDIAAI
X

NVIDIA researchers just released Asset Harvester — an end-to-end pipeline that turns autonomous driving video into manipulable 3D object assets. A key building block for dynamic scene simulation in AV development. Code is open. Check it out:

18retweets123likes
View on X

Still wondering? A few quick answers below.

Asset Harvester is an end-to-end pipeline and image-to-3D model designed to extract 3D object assets from autonomous driving video logs. It allows researchers to turn real-world driving footage into complete, simulation-ready 3D objects, such as vehicles, which can then be manipulated or placed into new virtual environments for testing and development.

The pipeline is specifically designed to work with in-the-wild object views captured during standard driving tests. Even when an object is only seen from a few angles—known as sparse views—the model can reconstruct a full 3D asset. This capability makes it possible to use existing driving logs rather than requiring expensive, controlled 3D scanning environments.

Yes, NVIDIA has released the code for Asset Harvester as an open-source project. Developers and researchers can access the repository on GitHub to implement the pipeline in their own workflows. This open release is intended to support the development of dynamic scene simulations and world models for autonomous vehicle research and synthetic data generation.

The tool is primarily used as a building block for dynamic scene simulation in autonomous vehicle development. By extracting real-world objects from video, developers can populate virtual training environments with diverse, realistic assets. It is also integrated with NVIDIA NuRec, allowing users to remove, insert, or replace 3D objects within reconstructed driving scenes for testing.

Every HeadsUpAI update is written based on its original source and reviewed before it's published. Read our editorial standards →

NVIDIA

See all AI news & updates from NVIDIA →

Share this update