Ai2 releases ToyBlocks for controlled robot learning tests
Ai2 released ToyBlocks on November 6, 2026, giving robotics researchers a controlled tabletop world for studying how models learn and fail. The release combines simulation, generated demonstrations, training and evaluation tools, plus two trained models. It is aimed at repeatable research on block manipulation, where a team can change one condition and inspect the result.
The useful question is whether a robot learned a transferable skill or merely became good at familiar arrangements. A successful demonstration rarely answers that alone. ToyBlocks provides a way to separate changes in the training data, the planner and the movement policy, making a failed attempt more informative.
What the ToyBlocks release includes
The public repository describes an Isaac Sim and Isaac Lab framework that generates robot demonstrations automatically and records them in LeRobot format. A high-level policy chooses the next skill, while a lower-level policy or motion planner carries it out. Researchers can substitute an oracle at either level to help identify which part caused a failure.
The initial dataset includes tens of thousands of long-horizon demonstrations. Ai2 says more data, additional skills and fully integrated reinforcement learning training are planned. That integrated RL training capability is not included in the current package.
For teams already using the dataset format, our LeRobot 0.7.0 upgrade guide covers a separate set of compatibility and logging checks. A shared data format does not establish that every training environment or checkpoint can be exchanged unchanged.
Two models divide planning from movement
Molmo2-4B-tb is the high-level vision-language planner, with about 4.5 billion parameters. It reads a goal and a front-camera image, selects an action and points to its target. Its published weights use native Molmo2 format rather than a Transformers checkpoint. The card estimates about 20 GB of GPU memory for bfloat16 inference.
ManiFlow-tb is the 322.5-million-parameter action model. Its inputs include camera images, robot state, a task name and a target heatmap. It produces chunks of robot actions. The downloadable package includes weights, configuration and normalization statistics, so those supporting files are part of the integration, not optional documentation.
Evaluation settings can change the conclusion
ManiFlow’s model card reports four simulation runs on the same 128 held-out episodes, with success between 54 and 59 percent. Those runs use an oracle high-level plan and a 194-second time limit. The card warns that the default 30-second limit causes most episodes to time out. These are measurements of the movement policy under specified conditions, not complete Molmo2 planning results.
That distinction matters when comparing systems. Keep the task set, planner, allowed time and success definition fixed before attributing a difference to a model. Record collisions and incomplete attempts alongside successful episodes. A generous deadline can reveal eventual task completion while concealing a speed problem that matters in deployment.
For physical experiments, Ai2’s launch blog reports 15 successful spelling tasks out of 30, using the easiest setting with visible, upright characters. The Molmo2 card separately reports 60 percent across 10 real trials. The sources do not explain how those samples relate. Neither number establishes general robot reliability, and the smaller card result should not replace the blog’s stated sample.
Check hardware and licenses before a pilot
The installation guide specifies Linux, an NVIDIA GPU and a driver supporting CUDA 12.8, with driver 570 or newer. Its setup uses Python 3.11 for Isaac Sim 5.1. NVIDIA’s Omniverse EULA also applies. This is a research software environment to configure, not a browser-only robot service.
ToyBlocks code is Apache 2.0, but the repository explicitly separates licenses for submodules and downloaded assets. The dataset page lists ODC-BY. Molmo2’s card lists Apache 2.0 while warning that its base model uses third-party datasets with academic and noncommercial research conditions. Check the exact components for your intended use rather than treating the project as one unrestricted license.
Start with one controlled comparison
A useful first experiment is a small task whose failure is easy to inspect. Save the starting scenes, run a baseline, then alter only one factor, such as where the blocks begin. Compare completion, time and failure type before changing the training mixture. Keep early work in simulation and follow the hardware’s safety procedures before any physical trial.
ToyBlocks’ value is the ability to investigate why performance changes. Its narrow setting makes that investigation manageable. Evidence that a policy handles an unfamiliar block arrangement is useful research progress, while claims about broader real-world competence still need broader tests.
