Chinese Researchers Open-Source CUDA Agent, a Model That Writes GPU Kernels Faster Than PyTorch’s Compiler
By OFOKI TECH | August 20, 2026
A team from ByteDance Seed and Tsinghua University’s Institute for AI Industry Research has released CUDA Agent, an open-source system that trains large language models to write high-performance GPU kernels. The model outperforms PyTorch’s torch.compile on 96.8% of benchmark tasks and exceeds the performance of Claude Opus 4.5 and Gemini 3 Pro by approximately 40% on the most difficult tasks.
The Problem
GPU kernel optimization is foundational to modern deep learning but remains a specialized task. Existing approaches to automated CUDA generation have hit a ceiling. General-purpose language models can write syntactically correct CUDA code, but the resulting kernels often run slower than compiler-optimized baselines.
The ByteDance/Tsinghua team quantified this gap. On the KernelBench benchmark, their base model (Seed1.6) achieved a 74% task success rate, but only 27.2% of its generated kernels ran faster than torch.compile. The average geometric speedup was actually slower than the compiler.
The Solution
CUDA Agent uses a three-component approach. First, a scalable data synthesis pipeline generates training problems by crawling seed operators from PyTorch and Transformers libraries, then using an LLM to compose them into fused tasks. The final curated dataset, called CUDA-Agent-Ops-6K, contains 6,000 training samples.
Second, a skill-augmented CUDA development environment equips the model with a structured workflow for writing, validating, and optimizing kernels. The environment includes automated correctness verification, performance profiling, and system-level permission isolation to prevent reward hacking.
Third, the team uses Proximal Policy Optimization (PPO) reinforcement learning trained over 150 steps. A multi-stage warm-up strategy stabilizes training for long-context, multi-turn agentic workflows. The system supports up to 128,000 tokens of context and 200 interaction turns.
The Results
On KernelBench, CUDA Agent achieved:
- 98.8% overall pass rate
- 96.8% of kernels faster than torch.compile
- 2.11x geometric mean speedup over torch.compile
The benchmark splits tasks into three difficulty levels. On Level 1 (single operators), CUDA Agent achieved 100% faster rate over torch.compile. On Level 2 (operator sequences), it also achieved 100% faster rate with a 2.80x speedup. On Level 3 (full neural network blocks like ResNet BasicBlock), it achieved 92% faster rate.
For comparison, Claude Opus 4.5 achieved a 66.4% faster rate and 1.46x speedup. Gemini 3 Pro achieved 69.6% faster rate and 1.42x speedup. CUDA Agent outperformed both by roughly 40% on the Level 3 split.
What Is Released
The research team has open-sourced the training dataset, the SKILL.md specification instructions, reward and warm-up training recipes, and the agent workdir environment. The complete model weights are not yet publicly available, though the GitHub repository indicates they may be released in the future.
The project is hosted under the BytedTsinghua-SIA GitHub organization, which also maintains related projects including Direct-OPD, MemAgent, and DAPO.
Implications
If the results hold up to independent verification, CUDA Agent represents a shift in how GPU kernels are written. Rather than relying on human experts or static compiler heuristics, the system discovers hardware-specific memory access patterns and tiling strategies through iterative optimization. This is particularly relevant for operator fusion, where traditional compilers struggle with non-trivial combinations.
The open-source release of the dataset and training recipes means other research groups can replicate and extend the work. The applications cited by the team include AI infrastructure, large model inference services, autonomous driving, and quantitative trading.
Sources and further reading