BrainLoRA: Scaling Brain Encoding with Parameter-Efficient Fine-Tuning (PEFT)
Summary:BrainLoRA is a novel approach to brain encoding that leverages Parameter-Efficient Fine-Tuning (PEFT) to adapt large pretrained models with a minimal number of trainable parameters. By using a Low-Rank Adapter (LoRA) and a lightweight readout layer, BrainLoRA achieves state-of-the-art (SOTA) performance on the Algonauts 2025 Challenge using the Qwen3-Omni backbone. The method also demonstrates scalability across nine diverse multimodal backbones and introduces a multi-LoRA kernel for accelerated
1. Background and Challenges
Brain encoding models aim to predict neural responses to stimuli. Traditional approaches rely on frozen pretrained backbones for feature extraction and use a small number of trainable parameters for output adaptation. However, training large models from scratch or fully fine-tuning them with neural data is challenging due to the high cost and limited availability of such data.
2. Core Innovations of BrainLoRA
BrainLoRA addresses these challenges through Parameter-Efficient Fine-Tuning (PEFT). Its key innovations include:
- Low-Rank Adapter (LoRA): By injecting a small number of trainable parameters into the pretrained model, it significantly reduces training costs while maintaining performance.
- Single-Pass Multi-Timestep Prediction: Leveraging the causal structure of the backbone, it predicts multiple timesteps in a single forward pass, avoiding the computational overhead of sliding windows.
- Multimodal Backbone Support: The method is applicable to multimodal backbones with up to 104 billion parameters, making it the largest adapted model for brain encoding to date.
3. Experiments and Results
BrainLoRA demonstrates consistent performance improvements across nine diverse multimodal backbones. On the Qwen3-Omni backbone, it achieves the best published performance on the Algonauts 2025 Challenge. Additionally, BrainLoRA shows robustness in modality attribution, recovering the canonical sensory topography that traditional methods often construct from separate unimodal encoders, and enhances the contribution of audio inputs and cross-modal mixing within the backbone.
4. Scaling and Optimization
To scale the number of adapted models, the research team developed a multi-LoRA kernel that accelerates the joint training of multiple LoRAs on a shared backbone. This enables the ensemble of over 200 adapted models, which ranks first on the public post-challenge leaderboard.
5. Conclusion and Future Work
BrainLoRA highlights the potential of PEFT for brain encoding. Its efficient training process and excellent performance make it a valuable tool for neuroscience and neural engineering. The team plans to further optimize the multi-LoRA kernel and explore its applications in larger models and more complex tasks.
— END —Source: Hugging Face Daily Papers (2026-10-19)
Tags: #BrainLoRA #PEFT #Brain Encoding #Multimodal Models #Hugging Face
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments