Hao AI Lab Open Sources Dreamverse for Real Time Video Directing

Hao AI LabHao AI Lab

· Updated

Hao AI Lab released Dreamverse, an open-source reference application that generates 30-second 1080p videos in 7 seconds on a single NVIDIA B200 GPU. The system introduces vibe directing, a workflow where creators steer video generation through natural language in a real-time interactive loop.

Hao AI Lab, a UCSD research group, open-sourced Dreamverse as a full-stack reference application for real-time video generation. Built on the FastVideo framework and LTX-2 model, it enables vibe directing—steering video via natural-language iteration—to provide creators with immediate visual feedback.
Video resolution
1080p
Generation speed
30 seconds of video in 7 seconds
Inference hardware
Single NVIDIA B200 GPU
Base model
LTX-2 (open weights)
Inference format
NVFP4

This release joins the performance tier established by Runway's Characters API latency benchmarks. The implementation utilizes NVFP4 to maximize NVIDIA B200 efficiency, adopting the LongLive-2.0 4-bit video infrastructure recently introduced by NVIDIA. It demonstrates that high-fidelity, 1080p video can be generated significantly faster than real-time playback.

You can self-host the Dreamverse workspace using Docker or deploy it to serverless platforms like Modal. The architecture includes a streaming layer that delivers fragmented MP4 chunks over websockets for instant playback. The lab is also exploring methods to support consumer hardware like the NVIDIA RTX 5090 in future updates.

View the full update on haoailab.com
Hao AI Lab
Hao AI Lab
@haoailab
X

🚀Generate a 30-second 1080p video in just 7 seconds! We’re open-sourcing FastVideo Dreamverse: real-time vibe directing for video generation on a single NVIDIA B200 GPU with LTX-2 model @ltx_model Repo: https://t.co/xTXsfCB6pF Blog: https://t.co/kA19cQhOJo https://t.co/cV5sX7t6Ve

115retweets727likes
View on X

Still wondering? A few quick answers below.

Dreamverse is an open-source reference application for real-time generative video. It introduces a workflow called vibe directing, which allows creators to steer video generation through natural language instructions in an interactive loop. Users can generate a scene, rewrite the direction, and continue the video with immediate visual feedback rather than waiting for batch processing.

Dreamverse can generate a 30-second video at 1080p resolution in approximately 7 seconds. This performance is achieved by running the system on a single NVIDIA B200 GPU. The architecture uses a streaming layer that delivers fragmented MP4 chunks over websockets, allowing the browser to begin playback while the rest of the video is still being generated.

The real-time generation path in Dreamverse is specifically optimized for the NVIDIA B200 GPU, which uses the Blackwell architecture. Each worker process requires one dedicated GPU to handle the workload. While currently optimized for high-end hardware, the developers are researching methods to support consumer-grade cards like the NVIDIA RTX 5090, 4090, and 3090 in the future.

Yes, Hao AI Lab has open-sourced the full stack for Dreamverse, including the frontend workspace and the backend runtime. The release includes Docker images, deployment scripts for local or serverless environments, and the optimized generation path built on the LTX-2 open-weights model. Developers can access the code on GitHub to build their own real-time video applications.

The system uses several technical optimizations to achieve real-time speeds, including NVFP4 inference, which is NVIDIA's block-scaled 4-bit format for efficient Tensor Core computation. It also utilizes FlashAttention 4 and torch.compile across the pipeline stages. For low-latency prompt rewriting, the system connects to external language model endpoints powered by first-generation LPUs from providers like Groq.

Every HeadsUpAI update is written based on its original source and reviewed before it's published. Read our editorial standards →

Share this update