Ai2 announces Bwen and Blama byte model checkpoints
Ai2 announces Qwen and Llama based byte model checkpoints alongside its Nature paper, with Stage 1 models for research.
Ai2 announced Bwen 8B and Blama 8B checkpoints on October 7 alongside the Nature publication of its byteification research. The models adapt Qwen3 8B Base and Llama 3 8B to work with the bytes that encode text. The announcement expands on Bolmo, the Olmo based model family introduced last December, and includes intermediate checkpoints for researchers who want to experiment with the conversion process.
The Nature paper describes a learned way to group bytes into variable length patches. A local encoder prepares byte representations, a boundary predictor groups them, and the large transformer processes the resulting patches. A local decoder then produces predictions at the byte level.
Training happens in two stages. First, the original global model stays frozen while the added byte components learn. The second stage trains the entire system. The Stage 1 checkpoints expose that intermediate state, where the inner model parameters remain unchanged. Researchers can therefore explore changes after the initial conversion rather than repeating that first training stage for every experiment.
In the authors’ evaluations, Bwen 8B and Blama 8B came close to their respective source models, while Bwen outperformed Bolmo 7B. In the coding tests, the byteified models had a higher chance of at least one correct solution among sixteen sampled candidates, but lower success for a single candidate than their source models under the same sampling settings.
The licenses differ. The Bwen model card lists Apache 2.0, while Blama uses the Llama 3 Community License Agreement. The cards describe research and educational uses. Bwen’s card warns that it lacks safety filtering and can produce harmful or inaccurate text. Those limits matter when deciding whether to use these checkpoints in an application.
The announcement date should also be kept separate from artifact history. The Bwen repository and Blama repository record initial commits on August 26 and later updates on August 28. Those dates establish earlier files, but do not establish when the repositories became public. Ai2’s October 7 statement and the accessible model collection support the current announcement, without proving a first download date.
