Daily papers

Monday, 9 November 2026 · 7 min read

P²-MOPD improves multi-teacher distillation by jointly training student weights and expert parameter mixes

P²-MOPD treats the student model as a linear combination of trainable weights and frozen expert parameters, jointly optimizing both under on-policy distillation. It achieves faster and more balanced recovery than standard MOPD, reaching 94.7% average recovery at 1.7B scale versus 80.9% for MOPD.

Based on the paper P^2-MOPD: Accelerating Multi-Teacher On-Policy Distillation via Parameter Priors by Sinuo Wang, Jiacheng Lu, Zeyu Zhang, Maojia Song and 7 others

Challenges in consolidating multiple domain experts into one model

Large language models often consolidate separately trained domain experts into a single generalist model. Multi-teacher on-policy distillation (MOPD) supervises the student model using feedback from each expert on the student's own generated trajectories, enabling dense reward signals for reinforcement learning tasks.

However, MOPD is computationally heavy since it requires sampling rollouts and teacher scoring at every step, and it often fails to recover all domains evenly. Some domains lag behind, and continued distillation can even degrade capabilities present in the initial merged model.

An alternative is model merging via task arithmetic, which linearly combines expert parameters without training. While efficient, this approach is limited to the linear span of the frozen expert parameters and often recovers only part of the experts' capabilities, lacking guarantees on quality.

Sign in to read the whole paper

You have read the opening. The rest is free with a Bezen account, and no card is needed.

In the rest of this paper

  • How the method works, step by step, with a flow chart
  • Three key formulas from the paper, explained symbol by symbol
  • Three figures from the paper, and how to read them
  • How it was tested, what was found, and where it falls short
P²-MOPD improves multi-teacher distillation by jointly training student weights and expert parameter mixes · Bezen