NVIDIA announced Dynamo Snapshot, a framework that reduces Kubernetes inference startup times from minutes to under five seconds. By decoupling model weights from process state, the system enables near-instant elastic scaling for large language models without maintaining expensive, idle GPU capacity.
NVIDIA announced Dynamo Snapshot, a framework using checkpointing (saving a running process's state) to eliminate cold-start delays in Kubernetes. It combines CRIU for host-side state with a cuda-checkpoint tool to serialize GPU memory. This allows a fully warmed inference worker to resume execution nearly instantly on any node in a cluster.
- Startup time
- Under 5 seconds
- Startup speedup (120B model)
- 21x
- Snapshot size (Qwen3-0.6B)
- 6.2 GiB
- Snapshot size (gpt-oss-120b)
- 129 GiB
- Supported engines
- vLLM, SGLang
AI demand is elastic, but the minutes required to load weights often cause SLA violations during spikes. This update extends the NVIDIA Dynamo 1.0 inference OS to address the infrastructure bottleneck that forced providers to over-provision hardware. By unmapping the KV cache, it makes rapid, serverless-style scaling economically viable.
The experimental release supports single-GPU workloads running vLLM or SGLang. Future updates will add support for TensorRT-LLM and multi-node clusters. The snapshot agent is available via Helm to help you implement sub-5-second auto-scaling, with core CRIU optimizations currently pending an upstream merge.
