Kolibri in llama.cpp: storage and serving checks
Kolibri 1 gained a llama.cpp deployment route when pull request 30673 merged on October 17, 2026. This was runtime integration, not a new model launch: Aleph Alpha dates Kolibri’s original release to October 3.
The merged work adds architecture and checkpoint-conversion support, plus architecture tests. Applications bundling llama.cpp still need a revision that contains the implementation. An application’s installation date alone does not establish compatibility.
The smaller active count is not a memory budget
The contributor reports 156 GiB for a BF16 GGUF and 45 GiB for Q4_K_M, with testing on four A800 accelerators. These are reported artifacts and test conditions, not ByteForward measurements or minimum hardware requirements.
Aleph Alpha’s current model card lists roughly 78 billion total parameters and 3.46 billion active per token. The full model still needs storage. Selected experts reduce computation rather than making the remaining weights disappear. Runtime buffers and context storage add to the model file’s footprint.
Keep the serving routes separate
| Route | Current documented requirement | What still needs checking |
|---|---|---|
| llama.cpp | Build containing the Kolibri implementation and compatible GGUF | Bundled revision, backend and output parsing |
| Transformers | Native support from version 5.20 | Chat-template options and parsed output fields |
| vLLM | Aleph Alpha’s inference package and Kolibri reasoning/tool-call parsers | Supported package version and chosen context length |
The card documents German and English, reasoning controls and tool calling. It recommends no more than 262,144 tokens for serving efficiency and complex tasks, despite listing a 1,048,576-token maximum. Those vendor settings do not prove identical behavior across engines.
The published model uses Apache 2.0. That license is not a hosted-service price: local deployment still has hardware and operating costs. Our original Kolibri coverage covers the release and weights.
Loading is only the first check
The llama.cpp server guide separates chat templates, reasoning extraction and tool-call parsing. Its default template comes from model metadata. A client must also handle whichever response fields the selected parser produces.
A useful deployment check compares an ordinary reply, a reasoning reply and a tool call your application actually uses. Inspect the final answer and tool arguments rather than judging only whether the process starts. Fix the model revision, quantization and context size before comparing results.
The project’s build documentation provides separate CUDA and Metal paths. Choose the backend for the actual machine, then measure its memory use and response behavior. ByteForward has not run these tests or verified performance on consumer hardware.
AI-generated conceptual illustration by ByteForward. Kolibri wordmark: Aleph Alpha.
