GPU kernel languages can produce correct but drastically slow code, study finds

Modern GPU programming languages designed for specialized tasks—particularly Triton and TileLang—are becoming the standard for writing deep-learning kernels and generating optimized code automatically. However, a new analysis reveals a significant blind spot: code that passes correctness validation can still perform far worse than the optimized library functions it is meant to replace.

The Gap Between Correctness and Performance#

Researchers at Carnegie Mellon University evaluated 22 production kernels written in Triton and TileLang across five operator types, running them on NVIDIA A100 and GH200 GPUs. Their findings show that passing numerical correctness checks—the current gold standard for validating kernels—does not guarantee acceptable performance. One striking example: an idiomatic TileLang implementation of LayerNorm passed correctness validation despite running more than 300 times slower than PyTorch's baseline version.

This "correctness-performance gap" matters because these domain-specific languages are increasingly used as both targets for automated code generation and as primary tools for implementing critical operations in deep-learning systems. When a generated or hand-written kernel is functionally correct but impractical in speed, it becomes unsuitable as a library replacement—yet existing evaluation practices cannot reliably flag this problem.

Root Causes Vary by Kernel Type#

The study identifies distinct patterns of failure depending on the kernel family. In normalization and reduction operations, slowdowns typically stem from repairable authoring defects: sequential reductions instead of parallel ones, redundant data-type conversions, and other suboptimal coding patterns that developers can learn to avoid. In contrast, convolution and large general matrix multiplication (GEMM) kernels retain significant gaps even after optimization, pointing to limits in code generation and autotuning coverage rather than author mistakes. Vendor library algorithm choices play only a minor role.

Detecting Problems Without Exhaustive Testing#

Because comprehensive benchmarking is impractical for every deployed kernel, the researchers propose two lightweight screening techniques that together identify every slow-but-correct kernel in their test suite. Measuring library-relative efficiency—how a kernel's throughput compares to the function it replaces—and —whether hardware resources are well-used—provides complementary signals. These checks separate authoring defects (which developers can address) from structural limitations in the language or compiler (which require deeper changes).