Tuesday, 20 October 2026 · 6 min read
Parameter-efficient fine-tuning scales large multimodal models for brain encoding
This paper shows that low-rank adapters (LoRA) enable efficient fine-tuning of very large pretrained multimodal models (up to 104B parameters) for predicting fMRI brain responses, improving performance and interpretability while reducing training cost by 350×.
Based on the paper Scaling Parameter-Efficient Fine-Tuning for Brain Encoding by Zhuofan Josh Ying, Guanglong Sun, Nolan Ho, Kieran Liu and 5 others
Challenges in adapting large pretrained models for brain encoding
Brain encoding models predict neural responses to stimuli, often using a pretrained backbone model followed by a readout trained on neural data. The dominant approach freezes the backbone and trains only the readout, leveraging pretrained priors but missing specialization opportunities.
Training large models from scratch or fully fine-tuning them on neural data is difficult because neural data is scarce and costly, limiting the ability to adapt large backbones effectively.
Parameter-efficient fine-tuning (PEFT) offers a middle ground by updating only a small number of parameters, allowing adaptation of large pretrained models to brain encoding tasks without requiring extensive neural data or compute.
Sign in to read the whole paper
You have read the opening. The rest is free with a Bezen account, and no card is needed.
In the rest of this paper
- How the method works, step by step, with a flow chart
- Two key formulas from the paper, explained symbol by symbol
- Three figures from the paper, and how to read them
- How it was tested, what was found, and where it falls short