OpenAI BioBasicsBench tests how AI reads laboratory evidence
OpenAI introduced BioBasicsBench on November 6, a benchmark built with biotech company Ligo to assess how AI agents interpret raw laboratory evidence. Its 100 tasks use material such as images, videos and instrument files. The evaluation concerns analysis, not physical laboratory execution.
How to read the BioBasicsBench scores
At xhigh, the highest reasoning setting shared across all seven tested models, GPT-6 Astra earned 66.7% of available rubric credit on average, compared with 62.0% for GPT-6.1 Sol and 43.8% for GPT-5.6 Sol. These are partial-credit averages, not experiment success rates. Each setting included five attempts per task.
OpenAI reports remaining errors involving unsupported assumptions, conflicting evidence and counting. The announcement does not specify Internet access or the full tool configuration, and does not link a public dataset download.
What a useful evaluation would measure
For a research team, the next question is whether an assistant can make a review process more useful without making its mistakes harder to find. Our recommendation is to begin with historical, non-sensitive cases that qualified reviewers already understand. Compare the assistant’s interpretation against a blinded expert assessment, then inspect disagreements rather than accepting one aggregate score.
The evaluation should distinguish the consequences of different mistakes. An unnecessary request for clarification and an unsupported confident conclusion impose different costs. Record whether an answer identifies the evidence it relies on, whether it acknowledges unresolved conflicts and whether a reviewer can trace the conclusion back to the original material.
That approach also creates a more useful acceptance test. A team could decide in advance which outputs require a second review, what evidence must accompany an answer and when the system should stop at an uncertainty statement. Those are workflow choices that a leaderboard cannot make for an individual organization.
Check the system around the model
Before comparing products, ask vendors for the exact model version, available tools, input preparation and grading procedure. A reproducible configuration makes it easier to investigate a later regression or determine whether an apparent improvement came from the model, the surrounding software or a changed test.
Measure total spend, elapsed time, reviewer effort and the frequency of corrections separately. A shorter response may be easier to read, but response length alone cannot tell a buyer whether a system saves money or reduces work. Compare the complete process on the same cases, with the same review requirements.
Readers evaluating physical automation can also consult our coverage of Google VERID and laboratory execution checks. That work addresses the interface between software and equipment, a separate layer from interpreting experimental evidence.
Keep the evidence beside the answer
A practical assistant interface should preserve the original input alongside its interpretation, make uncertainty easy to see and let a reviewer flag a disputed observation. It should also retain enough context to reconstruct how a conclusion was reached.
For buyers, that is a useful standard to take into a demonstration: ask for an inspectable decision trail and a realistic account of the human review still required. A polished explanation is only helpful when the person responsible for the work can check it.
