Lecture: "Beyond single neurons:Holistic neural representations and structure-preserving cognition"

Professor Christoph von der Malsburg is an IAS Distinguished Scholar of the Institute of Advanced Studies for the academic year 2024/2025.

Abstract

Perception arises from vast arrays of neural signals that, in isolation, lack meaning. Only when integrated into structured ensembles (Gestalts) do they become interpretable by linking to memory representations. A key question in neuroscience is how these higher-level patterns are neurally represented.

Current AI models assume that single neurons encode whole mental objects, but this approach fails to capture the structure of sensory patterns, requiring extensive supervised learning. I propose an alternative: cooperative nets, where neurons form small connectivity structures (“net fragments”) that assemble like jigsaw pieces under a consistency constraint. This enables structure-preserving mappings, allowing transformed variants of a pattern to be recognized from minimal exposure.

At higher cognitive levels, schematic nets can generalize across scene types, vastly reducing learning demands. This holistic representation model offers a biologically plausible alternative to token-based AI and advances our understanding of perception, memory, and cognition.