Bengio urges safety researchers to leave frontier AI labs
Yoshua Bengio has urged researchers who put safety first to leave frontier AI companies and work at safety institutes or mission driven organizations, including his own nonprofit, LawZero. The appeal appeared in an October 8 essay for Transformer.
Bengio argues that competition and commercial pressures undermine efforts to make increasingly capable AI safe. He acknowledges that researchers inside the companies have access to information and resources that could help them influence development. His conclusion is nevertheless that safety focused staff should take their work elsewhere.
The essay is also a recruitment pitch from a founder with an interest in attracting that talent. Its argument about where researchers can do the most good is Bengio’s judgment, rather than evidence that reform inside every commercial lab is impossible.
Public funding announced in September
The financial backdrop predates the essay. On September 16, Canada and Germany announced plans to invest C$150 million and €100 million respectively in LawZero. The Canadian government’s notice says Germany’s funding is subject to European Commission notification. These are planned investments with a stated condition, rather than confirmation that all the money has been disbursed.
Canada said its contribution would support research and engineering staff and computing capacity for Scientist AI, LawZero’s proposed technical approach. It also projected 360 full time jobs in Canada and dedicated computing infrastructure involving Hypertec and 5C. Those are projected outcomes, not a count of positions already filled.
LawZero’s own announcement describes joint backing of up to C$300 million and plans for a Berlin office. It says the funding will expand its scientific team and provide infrastructure for research at scale. That announcement presents the recipient’s case for the investment. Public support gives the project resources to pursue its approach, but does not itself demonstrate that the approach works.
What Scientist AI proposes
The technical idea has a longer history. In a paper first submitted in February 2025, Bengio and colleagues proposed a system built to explain observations and answer questions without pursuing its own goals in the world. They describe two components, a model that develops explanations for data and an inference system that answers questions. Both would explicitly represent uncertainty to reduce overconfident predictions.
The authors suggest that such a system could help human researchers and assess actions proposed by other AI agents. In that guardrail role, the distinction matters. A system that evaluates a possible action has a different job from an agent that chooses and carries it out. The proposal seeks to make useful intelligence available without giving the same system an open ended mandate to act.
The paper sets out a research direction and rationale. It should not be read as proof that a deployed system can reliably prevent every harmful action or that removing an explicit goal eliminates every safety problem.
Outside research can still involve the labs
Government safety research offers another institutional route, with a different relationship to commercial developers. The UK AI Security Institute’s published research agenda describes three channels for its work, informing government decisions, developing international practices and protocols, and acting as an independent technical partner to labs.
That last role includes testing advanced models, sharing findings with their developers and working on targeted improvements. The institute also describes partnerships with academic researchers, industry and government security specialists. This provides a concrete example of safety work outside a commercial employer that still depends on collaboration across organizations.
The agenda does not settle Bengio’s argument about where an individual researcher should work. It shows why employer choice and the practical relationship with model developers are separate questions. Researchers considering a move would still need to examine access to models, research independence and how findings can lead to changes.
For related policy coverage, see Canada creates national AI council to advise on adoption and safety.
Yoshua Bengio in 2019. Photo by Maryse Boyce via Wikimedia Commons, CC BY 4.0. Cropped by ByteForward.
