Google Releases Gemma 4 QAT Checkpoints for Efficient On-Device AI

GoogleGoogle

· Updated

Google released new Gemma 4 Quantization-Aware Training (QAT) checkpoints, including GGUF (Q4_0) and a custom mobile schema under 1GB. These enable running Gemma 4 models locally on consumer GPUs and mobile devices with reduced memory footprint and accelerated decode speeds, while preserving reasoning quality.

Google released new Gemma 4 Quantization-Aware Training (QAT) checkpoints, optimizing its open models for local and mobile deployment. These checkpoints include GGUF (a file format for running LLMs locally) Q4_0 formats for all model sizes and Gemma 4 drafter models, alongside a custom mobile-specialized schema. This mobile format reduces the memory footprint of Gemma 4 E2B to under 1GB by integrating quantization (simulating compression during training) directly into the training process.
Mobile memory footprint (Gemma 4 E2B)
Less than 1 GB
Quantization method
Quantization-Aware Training (QAT)
Key mobile optimizations
Static activations, Channel-wise quantization, Targeted 2-bit decoding layers
Supported local runtimes
llama.cpp, Ollama, LM Studio
Supported formats
GGUF, compressed tensors for vLLM

This approach minimizes quality loss during model compression, a common challenge with standard Post-Training Quantization. By preserving reasoning quality while drastically reducing memory requirements and accelerating decode speeds, Google makes powerful AI more accessible for local execution. This enables running capable Gemma 4 models on everyday edge devices and consumer GPUs.

The Q4_0 and mobile weights are on Hugging Face now: GGUF for llama.cpp, compressed tensors for vLLM. On a laptop they drop into Ollama or LM Studio; on a phone, LiteRT-LM or Transformers.js runs them on-device. The mobile schema shrinks Gemma 4 E2B under 1GB, so a capable model runs where a server was once required.

View the full update on blog.google
Google AI Developers
Google AI Developers
@googleaidevs
X

New @GoogleGemma 4 QAT (Quantization-Aware Training) checkpoints are here, so you can run models locally on consumer GPUs and mobile devices with minimal quality loss. What’s new: 🔹 GGUF (Q4_0): Checkpoints: Max local performance across all sizes and drafter models 🔹 Custom Mobile Schema: We shrunk Gemma 4 down to less than 1GB for mobile devices by using a custom mixed precision schema designed for edge hardware (featuring targeted 2-bit decoding layers, optimized KV caches, and static activations) By simulating compression during training rather than after (Post-Training Quantization), we've drastically reduced the memory footprint and accelerated decode speeds while preserving reasoning quality. https://t.co/k2OfUkxWPF

9retweets53likes
View on X

Still wondering? A few quick answers below.

These are optimized versions of Google's Gemma 4 models that have undergone Quantization-Aware Training. This process simulates model compression during the training phase, which helps reduce memory requirements and accelerate inference speeds while maintaining the model's original quality.

For mobile devices, Google developed a custom mixed precision schema. This includes targeted 2-bit decoding layers, optimized KV caches, and static activations, allowing Gemma 4 E2B to run with less than 1GB of memory on edge hardware.

Developers can download the Q4_0 and mobile model weights from Hugging Face. The models are provided in formats like GGUF for llama.cpp and compressed tensors for vLLM, supporting various local and on-device deployment workflows.

Every HeadsUpAI update is written based on its original source and reviewed before it's published. Read our editorial standards →

Google

See all AI news & updates from Google →

Share this update