Wednesday, 18 November 2026 · 7 min read
AscendGYM benchmarks kernel optimization on Ascend NPUs with cost-aware metrics
AscendGYM evaluates agents' ability to optimize existing Ascend NPU kernels from real LLM workloads, using a new Speedup-Token Score metric that balances performance gains with token cost under budget constraints.
Based on the paper AscendGYM: A Cost-Aware Kernel Optimization Benchmark for Real-World Workloads on Ascend NPUs by AI Training Platform Team, Bingwei Zhang, Jing Li, Amir Abdullin and 23 others
Challenges in optimizing Ascend NPU kernels for real workloads
Efficient training and inference of large language models (LLMs) on Ascend NPUs depend heavily on the performance of underlying kernel implementations. Optimizing these kernels is complex because it requires understanding performance bottlenecks in the context of specific input configurations and hardware characteristics, then iteratively modifying and testing code to improve speed without breaking correctness.
Existing benchmarks for Ascend kernels focus mainly on generating new kernels from specifications rather than improving existing implementations under realistic input shapes and data types drawn from actual workloads. This leaves a gap in evaluating how well agents can optimize kernels in practical scenarios.
Moreover, assessing optimization quality is difficult because final speedup alone does not capture the cost or effort required to find improvements. Different agents may achieve similar speedups but with vastly different resource consumption, making a cost-aware evaluation necessary.
Sign in to read the whole paper
You have read the opening. The rest is free with a Bezen account, and no card is needed.
In the rest of this paper
- How the method works, step by step, with a flow chart
- Three key formulas from the paper, explained symbol by symbol
- Three figures from the paper, and how to read them
- How it was tested, what was found, and where it falls short