AI scientific models need more than a good fit, researchers argue

AI can write a scientific model that matches experimental measurements and still leave researchers unsure which explanation to trust. In a November 6 guest essay published by Anthropic, researchers Jakob Macke, Daniel Gedon and Stefan Wahl argue for systems that preserve competing explanations and show where the evidence remains uncertain.

Their scientific modeling perspective separates three jobs: constructing a model, fitting its parameters and interpreting what it means. The immediate news is that research argument, rather than a new Claude release. For scientists assessing AI tools, the practical question is whether the output helps design a better experiment, beyond reproducing an existing dataset.

Why one convincing fit can mislead

Consider a simulator of a nerve cell. Different settings can produce similar electrical activity, so a good fit need not identify a unique underlying mechanism. A 2020 eLife study addressed this problem using neural density estimators trained on simulations. Instead of returning just one parameter setting, the method estimates a distribution of settings compatible with observations and prior assumptions.

The researchers demonstrated the approach on receptive fields, ion channels, Hodgkin-Huxley models and a crustacean neural circuit. This is established background to the new essay, not a November breakthrough. Its useful distinction is between uncertainty about a model’s settings and uncertainty about the model’s structure. Better parameter fitting alone cannot settle whether the chosen equations describe the right mechanism.

ModelSMC explores competing explanations

The authors’ ModelSMC paper takes the search up a level, treating executable simulator programs as candidates in a probabilistic inference process. An LLM proposes and revises programs, likelihood-based evaluations weight them, and resampling concentrates the search on promising candidates. The work first appeared as a preprint in February 2026, was revised in June and appeared at ICML in July.

There is an important qualification in the paper’s theoretical analysis. Its consistency result concerns an idealized sampler, with assumptions including coverage of the target model space and exact likelihood evaluation. The implemented workflow uses approximations. A finite collection of generated programs should therefore not be described as an exhaustive catalogue of nature’s possible explanations.

That distinction matters when a research team interprets agreement among candidates. Similar predictions may be informative, but the search procedure and its assumptions still determine which alternatives had a chance to appear.

What researchers can inspect today

The official ModelSMC repository provides implementation code, benchmark tasks, recorded results and scripts for reproducing figures and tables. It also spells out material limitations: the potassium and aldosterone task itself is absent, although its recorded results are included. Most runs require LLM API keys. A minimal validation example runs without an LLM.

Reproduction is not a one-click guarantee. The published experiment configurations target the authors’ Slurm cluster and need suitable local or cluster settings elsewhere. Licensing also deserves a check: most code is MIT-licensed, but designated components and dependencies use other terms, including the Prior Labs License.

For a first evaluation, our recommendation is to start with a released task, record the exact configuration and computational budget, and compare predictions across repeated runs. Keep experimental measurements reserved for evaluation separate from data used to guide the search. Ask a domain expert to inspect the proposed mechanisms before treating the output as an explanation.

The next experiment is the real test

The guest authors acknowledge that evidence for these systems on problems without known answers remains sparse. Their proposal is most useful as a standard for scrutiny: retain alternatives, identify disagreements and test predictions that separate them.

A useful pilot should end with a checkable prediction and a record of its assumptions. A polished equation or a plausible narrative is not enough to justify a scientific conclusion.

For a related approach to choosing informative experiments, see ByteForward’s earlier coverage of the Model Discovery Agent and Bayesian experimental design.