Domenic Bersch

Vision models are sensitive to object–background context, yet it remains unclear how recognition changes as the semantic distance between objects and backgrounds increases. We introduce ImageNet-OOC1k, a benchmark enabling continuous control of object–background relationships via a consensus semantic dissimilarity axis. This formulation reveals a highly consistent monotonic decline in recognition accuracy across 35 models spanning convolutional neural networks, Vision Transformers, and vision–language models. Internal analyses of Vision Transformers reveal aggregation failure, where object evidence is present at the patch level but not consolidated into the final prediction. Leveraging this characterization, a simple spatial re-ranking strategy recovers a subset of these errors and improves accuracy by up to 5.4 percentage points. These results establish context sensitivity as a continuous and predictable property of modern vision systems, with Vision Transformer analyses showing that failures can arise from limitations in aggregating competing signals rather than from missing object representations.

How the relationship between image-text aligned and vision-only models changes across development remains unknown. To address this, we applied time-resolved representational similarity analysis to EEG data from 8-year-olds, 12-year-olds, and adults. Commonality analysis showed complementary structure for image-text aligned versus vision-only models at 8 years, near-zero shared variance at 12 years, and greater overlap in adults, while both retained unique variance at all ages. This pattern generalized across model pairings, whereas vision-only model pairs showed overlapping variance at every age, suggesting a double dissociation. Within-category structure appeared complementary even in adults, suggesting that image-text aligned training may capture fine-grained structure not recovered by vision-only training. Together, these findings suggest that image-text aligned and vision-only models capture distinct aspects of visual representations that become more aligned in adulthood, and that vision-only models alone may miss structure relevant for developmental model-brain alignment.

In silico neural responses generated from encoding models increasingly resemble in vivo responses recorded from real brains, enabling the novel research paradigm of in silico neuroscience. In silico neuroscience scales beyond what is possible with in vivo data, allowing to explore and test scientific hypotheses across vastly larger solution spaces. To catalyze this emerging research paradigm, here we introduce the Brain Encoding Response Generator (BERG), a resource consisting of multiple pre-trained encoding models of the brain and a Python package to generate accurate in silico neural responses to massive amounts of arbitrary stimuli with a few lines of code. We show that BERG's encoding models accurately predict neural responses to visual stimuli, and that these in silico responses reproduce key neural signatures of visual processing in the brain. Together, this opens the doors to using in silico neural responses for scientific discovery, which we envision will lead to a more efficient and reproducible science.

There is growing symbiosis between artificial and biological intelligence sciences: neural principles inspire new intelligent machines, which are in turn used to advance our theoretical understanding of the brain. To promote further collaboration between biological and artificial intelligence researchers, we introduce the 2025 edition of the Algonauts Project challenge: How the Human Brain Makes Sense of Multimodal Movies. In collaboration with the Courtois Project on Neuronal Modelling (CNeuroMod), this edition aims to bring forth a new generation of brain encoding models that are multimodal and that generalize well beyond their training distribution, by training them on the largest dataset of fMRI responses to movie watching available to date.

We introduce Net2Brain, a graphical and command-line user interface toolbox for comparing the representational spaces of artificial deep neural networks (DNNs) and human brain recordings. Net2Brain supports activations from over 600 DNNs trained on diverse vision-related tasks (e.g., semantic segmentation, depth estimation, action recognition), for both image and video datasets. It computes representational dissimilarity matrices (RDMs) from activations and compares them to brain recordings using representational similarity analysis (RSA), weighted RSA, and searchlight search. The toolbox also allows integration of new stimuli and brain recording datasets. An example demonstrates its utility for testing cognitive computational neuroscience hypotheses.