Daily papers

Wednesday, 18 November 2026 · 5 min read

LLMs design sparse table models outperforming Transformers in weight-space programming

Large language models (LLMs) writing weights directly without gradient training prefer sparse associative-memory tables over Transformers, achieving up to 54.4% accuracy on ARC with 1,800× fewer parameters and up to 17× faster inference on GPUs.

Based on the paper Weight-Space Programming Doesn't Need Transformers by Hyunjong Ok, Jaeho Lee

Challenges of weight-space programming without gradient training

Transformers excel in gradient-based training and GPU efficiency, but weight-space programming (WSP) tasks a large language model (LLM) with directly writing target model weights without gradient descent or data fitting.

This shift removes the usual reasons for choosing Transformers, raising the question of whether Transformers remain the best architecture for WSP.

WSP requires models whose weights can be written and debugged line-by-line, with each weight having a clear meaning and edits affecting only local predictions, unlike dense, overlapping Transformer weights.

Sign in to read the whole paper

You have read the opening. The rest is free with a Bezen account, and no card is needed.

In the rest of this paper

  • How the method works, step by step, with a flow chart
  • Three key formulas from the paper, explained symbol by symbol
  • How it was tested, what was found, and where it falls short