Boole 1.7B compresses Qwen for local inference
Precision Numerics announced Boole 1.7B on October 11. The binary Qwen3 1.7B derivative targets English responses with thinking disabled.
The developer reports 247 MB of weights and an equal weight ten task average of 50.55 versus 58.83 for BF16. That retains 85.9 percent of the baseline average at about one fourteenth of its weight storage. ByteForward has not reproduced these results.
Which file you download matters
The model repository lists a 247.1 MB safetensors file and a 292.1 MB GGUF. The latter stores embedding and output matrices separately. The downloads therefore differ even though they represent the same model. The listed license for the weights and runtime code is Apache 2.0, with upstream Qwen attribution in the accompanying notice.
The headline counts weight tensors and quantization scales, excluding tokenizers, headers and metadata. It measures neither runtime memory nor latency.
Choosing a local runtime
The demo repository documents two routes. A pinned llama.cpp build loads GGUF on a CPU or CUDA GPU. The alternative uses vLLM with Humming on a single CUDA GPU and a loader configured for BF16. Developers should choose the route that matches their hardware before downloading the corresponding format.
The demo requires Python version 3.10 or later. Its vLLM path first exports the packed checkpoint into a local model directory and registers a plugin for the binary embedding and output head. The loader targets vLLM 0.31.0 and Humming kernels 0.1.16, so the project instructions matter more than a generic model loading command.
The examples use a context of 4,096 tokens and disable thinking. That is a demonstrated configuration in the instructions, not a measured maximum context or a performance result. The repository also describes CPU tests using a synthetic checkpoint. Their presence does not establish that the released model has been tested across consumer hardware.
What the quality average covers
Most tasks use greedy decoding, while GPQA Diamond averages ten sampled runs.
The card specifies no few shot examples and a standard seed of 42. GPQA Diamond uses temperature 0.6 and top p of 0.95 with seeds 42 through 51. Mathematics answers have a ten second scoring deadline, and timed out answers receive zero. Those details matter when attempting a comparable evaluation.
The evaluation uses full benchmark splits. It sets generation context to 16,384 tokens by default and 40,960 for function calling, capped when a checkpoint specifies a lower limit. Those evaluation settings should not be confused with the shorter context in the deployment examples.
Boole outperforms binary Bonsai 1.7B on seven tasks but trails on MuSR, IFEval and function calling, according to Precision Numerics.
A separate laptop comparison illustrates why memory fit and sustained performance need different checks.
