Wednesday, 11 November 2026 · 7 min read
Tree-Ising optimizes sparse bit-level edits to fix reasoning loops in language models
Tree-Ising formulates repairing reasoning loops in LLMs as a discrete optimization over BF16 weights, outperforming a prior LLM agent repair on Qwen3.5-0.8B with fewer weight changes, less compute, and no LLM tokens, and transfers to Qwen3.5-2B without reference edits.
Based on the paper Tree-Ising: Repairing Language Models with Bit-Level Ising Edits by Hadi Salloum, Ammar Ali
Reasoning loops in LLMs and the challenge of weight-space repair
Large language models sometimes fail by entering reasoning loops, repeatedly generating similar tokens until hitting a token limit, which drastically reduces accuracy on tasks like math problem solving. This failure mode is tied to the model's weights, specifically in certain MLP layers, making direct weight edits a promising repair approach.
Prior work by Takehi et al. demonstrated that an LLM agent can read a model's weights, diagnose failures, and write a dense fix without gradients, but this process is costly, requiring many forward passes, millions of tokens, and hours of compute, with no guarantee of optimality.
The challenge is to find a more efficient, principled method to repair such failures by formulating the problem as a discrete optimization over the model's stored weights, leveraging forward pass statistics and a representation that respects the BF16 weight format.
Sign in to read the whole paper
You have read the opening. The rest is free with a Bezen account, and no card is needed.
In the rest of this paper
- How the method works, step by step, with a flow chart
- Three key formulas from the paper, explained symbol by symbol
- How it was tested, what was found, and where it falls short