From Exploring Questions to Exploiting Answers

I have a host of nagging opinions in my head that are vestiges of the 10 or so years I spent in the field of computational neuroscience. Here's one I can't seem to shed that speaks to a perennial question in the sociology of science: what's a 'good' answer?

Over the past decade, two research traditions in systems neuroscience have been on a collision course. On one side, deep learning engineers build models trained on large datasets of neural activity, achieving unprecedented predictive accuracy. On the other hand, theoretical physicists study idealized recurrent neural networks, pursuing mechanistic understanding through mathematical analysis. The engineers optimize for performance; the physicists optimize for insight. And the two communities don’t talk much to each other.

This collision is fueled by data: deep convolutional network models have achieved remarkable accuracy in predicting neural responses—a situation reminiscent of Rich Sutton’s Bitter Lesson, that performance optimization will, in the end, select for larger, more versatile models unencumbered by hand-crafted constraints. But these feed-forward models are now running up against neural recordings from deeper brain areas whose features they struggle to explain. One way to overcome those limitations, as Kar & DiCarlo (2021) have shown, is to incorporate feedback and recurrence.

There is something that strikes me as odd about this trajectory given the 40-plus years of physics-inspired work on recurrent dynamics in neural networks. Shouldn’t recurrence have been in the engineers’ models from the start—or its omission carefully justified? At the very least, one might expect it to be flagged as a tenuous approximation. Yet a widely cited perspective paper (Yamins & DiCarlo, 2016) makes only cursory reference to recurrence, and there is remarkably little overlap in citations between the two communities. The engineers’ recent “discovery” of recurrence is not really a discovery at all.

I am not complaining about citation practices, but rather trying to highlight a larger question: how should a scientific field manage the diversity of methods used by practitioners as it matures? I want to propose that the explore-exploit dilemma of reinforcement learning offers a useful lens for thinking about this.

The explore-exploit problem in scientific methodology

The ideas here germinated in discussions with Jessica Thompson, who has written a thoughtful piece on scientific explanation in neuroscience (Thompson. J Neurophys 2021). Towards the end, Thompson highlights how feminist philosophy of science emphasizes the social milieu in which science is practised, and how we judge what constitutes a good explanation largely through the community we keep. One of her takeaways is that diversity—of methods and of colleagues—is a strength.

I share this value. But I think the relationship between method diversity and lasting knowledge is more nuanced. For knowledge to persist, it needs to achieve consensus, and consensus-making is frustrated if practitioners cannot convince each other on account of their incompatible backgrounds. A refined version of the principle might be: we should maintain diversity in methods to hedge against overfitting to a particular kind of answer, while also ensuring we share a language through which competing results can be adjudicated.

There are many scientists that believe the best candidate for that shared language is mathematics. Bill Bialek captures the spirit well in Searching for Principles: “Not only is the book of Nature written in mathematics, but also there is only one book.” He goes on to clarify his view that this applies to answers, not questions—at the question-forming stage, insisting on unanimity would amount to one group claiming the sole right to determine what is relevant.

This distinction between question-forming and answer-giving maps naturally onto the explore-exploit tradeoff in reinforcement learning. Decision-making under uncertain rewards has been treated mathematically by incorporating Bayesian ideas into RL theory, and the resulting algorithms are good precisely because they manage the transition from exploration to exploitation intelligently.

The analogy to scientific practice is straightforward. In nascent fields where the right questions are uncertain, a diverse set of approaches makes sense—the community should explore. But once a field’s objects of study and phenomena are well-delimited, it becomes more valuable to exploit the methods that most efficiently produce lasting explanations. Platt noted something similar in his “strong inference” paper (Platt, 1964), highlighting molecular biology as a field that achieved rapid progress through systematic convergence on method. The question is: what should systems neuroscience converge toward?

What should the exploit target look like?

I believe the right target is the kind of explanation used in the physics of complex systems: quantitatively faithful mechanisms working within an effective description accurate over some limited range of spatiotemporal scales.

The core intuition is simple. Effective descriptions of natural phenomena typically have a geometry and mechanics to them—stuff moving around in a space. This space can be physical (a synapse; a circuit) or abstract (a low-dimensional manifold in population activity space). These systems are open: we observe only part of the full dynamics, driven by outside forces we can at best summarize statistically. The mathematics of transport and transformation—centuries of well-worn mathematical theory—applies directly, and it imposes strong constraints on how such systems can behave. Normal forms and bifurcation theory characterize the qualitatively distinct ways that dynamics can unfold; the same equations appear across minimal models in chemistry, ecology, neuroscience, and epidemiology. This universality is a feature that reflects a similar underlying logic of interaction.

Crucially, any truly quantitative approach also quantifies its own uncertainty—from measurement imprecision, finite data, and model idealizations. This is a data science engineering problem, that brings many aspects of the deep learning approach with it. The best we can hope for in inferring a mechanism is a distribution of belief over which mechanism is at play, given our data and priors. Falsifiability, as a methodological principle, is largely inadequate for complex systems. But the ingenuity of experimental design can still concentrate the posterior distribution over a tractable set of candidate mechanisms.

So, in my view, the narrowing of focus in methodology called for in the later "exploit" stages of a maturing field is really about bringing these two traditions to work in tandem.

Back to the engineers and the physicists

So where does this leave the contemporary debate? On the engineering side, deep learning models have given neuroscience something the physics approach was not providing on its own: modelling with genuine experimental contact at the level of function. We should absolutely give DiCarlo and colleagues credit and praise for that. Science isn't all prediction, however. Higher accuracy from more complex models likely comes with greater difficulty in identifying underlying mechanisms. The success the engineering approach has enjoyed with feed-forward architectures will likely not continue as it confronts tasks involving more cognition (even in visual cortex, c.f. Tzalavras et al. (2026)). At the same time, the physics approach alone faces its own problem: the zoo of dynamical mechanisms and jungle of biological detail are too vast for a diffusive search biased toward mathematical tractability to find the essential ingredients on its own.

The RL perspective suggests the field should now be collapsing method diversity onto approaches that reveal mechanisms of the kind just described—dynamical systems, optimization theory, stochastic processes, and statistical inference. A practical obstacle is that not enough practitioners, including those in the engineering camp, are trained in the mathematics and perspective of complex systems physics. It is encouraging, however, that current work at this intersection is beginning to delineate what an integrated framework should look like.

Should all systems neuroscience programs demand a joint degree in statistical physics? No—but functional fluency in the methodology of complex systems could be gained through specialized courses at various levels of mathematical competence. An integrated program would emphasize that brains can be coarse-grained at many scales, but that descriptions at each scale need to capture the statistics of external drive and internal interactions and remain self-consistent with valid explanations at other scales.

I agree with Thompson that explanations are operational in character: they pertain to phenomena, not systems. But the deeper truth is that explanations for distinct phenomena from the same system are correlated for the simple reason that they describe the same system. We should strive to explain this larger web coherently, using a shared mathematical language that unifies disparate phenomena.

I am left reflecting on why the way people make models work often is not by using the math that explains why the systems work. I saw a talk by Jeff Pennington some years ago in which he presented results on a toy model of high-dimensional random feature learning that produces the infamous double-descent phenomenon (Adlam & Pennington, 2020). His conclusion: the phenomenon is baked into the structure of the high-dimensional inference problem and has nothing to do with deep learning per se. Do the engineers care? Are they paying attention to these retrospective theories explaining their success? Probably not. They are elsewhere, now making larger language models and doing pretty well. But “doing well” and “understanding why” remain different achievements—and a field that wants to build lasting knowledge needs both.