Model Discovery Agent combines AI proposals with Bayesian experiments
Curve fits answer “what happened.” They choke on “what if we intervene.” On arXiv:2608.09696, University of British Columbia’s Kevin Murphy posts the Model Discovery Agent (MDA): an LLM proposes candidate mechanistic structures, then classical Bayesian tools pick the next experiment so the system learns a causal world model from few interventions.
The stake is data efficiency in science loops where every assay, probe, or patch-clamp run costs real money. Passive data underdetermines mechanisms. Experiments break the tie — if you choose them well.
What the agent proposes and tests
MDA treats the world as an intervenable state-space model. Given a domain description and any initial observational data, an LLM proposes candidate structures — force laws, rate equations, ion-channel mechanisms — as hypotheses. The agent then runs a budgeted loop: update beliefs, design an experiment, observe, repeat.
Discovery and design reinforce. Designed experiments identify the mechanism the proposer just introduced. Once that mechanism is pinned down, residuals get sharper, so the next round can propose subtler structure. When a predictive check says the truth sits outside the current hypothesis class (the M-open setting), the LLM expands the pool with a new model, then Bayesian design identifies its parameters.
The LLM is a proposer with priors. Identification and experiment choice stay in the Bayesian stack.
Bayesian SMC/SBI role
MDA wires three standard pieces. Sequential Monte Carlo (SMC) maintains a posterior over structures and parameters, with evidence (marginal likelihood) supplying an automatic Occam penalty against bloated models. When likelihoods are intractable — stochastic latent dynamics — simulation-based inference (SBI) and synthetic likelihoods on summary statistics step in. Value-of-information (VoI) picks the next experiment as the design that most reduces uncertainty about which hypothesis is true; under simple noise models that reduces to maximizing cross-model disagreement in predicted outcomes.
Baselines include random designs, LLM-proposed designs, and pure LLM forecast agents that skip the SMC/VoI loop. Once the proposal lands, the recipe is particle posteriors, evidence, VoI. The novelty is keeping that machinery alive while an LLM keeps opening the hypothesis class.
Benchmarks named in the paper
Results span three interactive discovery benches. ForceBench wraps DiscoverPhysics-style particle worlds: recover an unknown force law from probe launches; MDA is substantially more sample-efficient than a budget-matched LLM agent within eight one-at-a-time experiments. ChemBench wraps ActiveSciBench-style enzyme kinetics: recover algebraic rate laws over a seven-dimensional design space; MDA hits high symbolic accuracy in about eight experiments while a prior AutoSciLab-style LLM baseline needs tens of trials to catch up. NeuronBench is new: six mystery Hodgkin–Huxley-style neurons with partial observability, channel blockers, and spike-count summaries; Bayes forecasting with VoI (or LLM) acquisition beats in-context forecasting, and a stochastic extension shows when particle-filter likelihoods are required.
Across domains, the paper claims new state-of-the-art data-efficient model learning and more reliable interventional prediction than pure LLM baselines.
What replications should hold fixed
Hold the protocol fixed before chasing bigger proposers. Same experiment budget per round (batch size one in the main plots). Same design menus or continuous boxes the paper enumerates. Same forecaster split: Bayes-forecast versus LLM-forecast versus in-context forecast. Same pass thresholds (ForceBench nMSE under 0.1, ChemBench RMSLE gates). And keep the M-open trigger honest: residual checks that actually expand the hypothesis pool, not a frozen shortlist that pretends the truth was always in-class.
Analysis: MDA bets that scientific agents need Bayesian experiment design as much as fluent proposers. The LLM invents structure; SMC and VoI stop the loop from wasting the lab. If replications hold the efficiency gaps on ForceBench, ChemBench, and NeuronBench, “LLM proposes, Bayes designs” becomes a default recipe for mechanistic discovery — not another prompt that hallucinates a pretty equation.
