Kimi Open Sources FlashKDA to Double Long Context Prefill Speeds

KimiKimi

· Updated

Kimi open-sourced FlashKDA, a high-performance implementation of its Kimi Delta Attention kernels built on the CUTLASS library. The release achieves up to a 2.22x speedup in prefill processing on H20 GPUs compared to standard linear attention baselines. This allows developers to significantly reduce latency for long-context applications by swapping in a more efficient backend.

Kimi, an AI company building models with long-context capabilities, open-sourced FlashKDA. It is a high-performance implementation of Kimi Delta Attention (KDA) kernels—a linear attention mechanism (an architecture that processes text more efficiently) built using the CUTLASS library.

Building on its shift toward linear attention, hardware-level optimization is the primary bottleneck for speed. FlashKDA achieves a 1.72x to 2.22x speedup in the prefill phase on NVIDIA H20 GPUs. This follows Kimi's recent efforts to slash long context costs.

You can use FlashKDA as a drop-in backend for the flash-linear-attention library to improve performance. The code is available on GitHub, providing the CUDA kernels needed to accelerate inference. This enables faster execution for massive parallel agent tasks and mirrors the integration of Kimi models into agentic coding environments.

View the full update on github.com
Kimi.ai
Kimi.ai
@Kimi_Moonshot
X

We're open-sourcing FlashKDA — our high-performance CUTLASS-based implementation of Kimi Delta Attention kernels. Achieves 1.72×–2.22× prefill speedup over the flash-linear-attention baseline on H20, and works as a drop-in backend for flash-linear-attention. Explore on github: https://t.co/sf4UohXDWY

111retweets1.2klikes
View on X

Still wondering? A few quick answers below.

FlashKDA is an open-source implementation of high-performance kernels for Kimi Delta Attention, a linear attention mechanism developed by Kimi. It is built using the CUTLASS library to optimize how AI models process long sequences of text. By improving the efficiency of attention calculations, it helps models handle massive context windows more effectively.

FlashKDA achieves a significant performance boost during the prefill phase, which is the initial processing of input data. According to Kimi, it delivers a 1.72x to 2.22x speedup compared to the standard flash-linear-attention baseline. These performance gains were specifically measured on NVIDIA H20 GPUs, which are specialized hardware for AI workloads.

Yes, Kimi has officially open-sourced FlashKDA. The implementation is available for public exploration and use on GitHub under the MoonshotAI organization. Developers and researchers can access the repository to integrate these high-performance kernels into their own AI projects or to study the underlying implementation of Kimi Delta Attention.

FlashKDA is designed to work as a drop-in backend for the flash-linear-attention library. This means developers already using that library can swap in FlashKDA to gain performance improvements without needing to rewrite their entire codebase. It specifically targets the Kimi Delta Attention kernels to provide a more efficient way to execute linear attention operations.

Every HeadsUpAI update is written based on its original source and reviewed before it's published. Read our editorial standards →

See all AI news & updates from Kimi →

Share this update