Gewell 0.2.0 brings multimodal embeddings to Blackwell GPUs
Gewell published version 0.2.0 on October 12, adding Gemma 4 26B A4B generation and EmbeddingGemma 2 serving. The update gives developers another way to host these models, with new controls for tool calls and image processing.
The timing matters. Google’s release history dates EmbeddingGemma 2 to October 6 and the original Gemma 4 family to March 31. The news here is support in a separate inference engine.
Two different workloads
The developer says the 26B model supports HTTP serving, offline generation, image input and speculative decoding. EmbeddingGemma 2 gets a dedicated server with mixed text, image, video and audio inputs, batching and selectable output dimensions. The release also adds tighter tool selection, strict argument schemas and configurable image token budgets.
EmbeddingGemma 2 produces vectors for comparing and retrieving content. Google describes one shared space for its four input types, which can support applications such as searching a media collection with text. Its 740 million parameters include separate vision and audio encoders. That is a different job from generating conversational answers.
Our EmbeddingGemma 2 launch report explains the original model’s retrieval capabilities across text, images, video and audio.
Check the hardware first
Gewell’s build guide targets Linux and NVIDIA Blackwell GPUs with compute capability 12.0, including the RTX 5090 and RTX PRO 6000. The development toolchain uses CUDA 13.1. It also requires CMake 3.25 or newer, a compatible C++17 compiler and native CUDA, OpenSSL, PNG and JPEG libraries. Windows is described as untested.
The project can build CPU libraries and tests without CUDA, but that mode does not produce a CPU inference engine. Its Python artifact tests use small fixtures and do not load a full model on a GPU. A successful test build therefore would not establish usable local inference.
Memory planning needs similar care. Google’s model overview explains that Gemma 4 26B A4B activates about four billion parameters per token while retaining the full model in memory. The active count is not a four billion parameter storage requirement. Quantization, context and concurrency all affect the deployment budget.
Gewell’s bundle documentation describes a manifest, a .gwt weight file and tokenizer data. The runtime implements model behavior and chat rendering itself rather than treating a downloaded chat template as configuration. Developers should verify a bundle’s compatibility instead of assuming that any Gemma download can be substituted unchanged.
Familiar APIs have limits
The HTTP guide describes a subset of OpenAI compatible Chat Completions and Completions, with no Responses endpoint. Chat image requests must contain inline PNG or JPEG data. The server does not fetch remote image addresses or local file URLs.
The API guide describes reasoning effort as an on or off thinking switch. It does not select different reasoning budgets. Tool calls remain instructions for the client to execute and return, so an application still needs its own execution and permission controls.
Strict tool schemas accept a supported subset of JSON Schema. Unsupported assertions are rejected before generation, and reaching the token limit can still leave a constrained answer incomplete. Existing clients should test their actual request fields, schema shapes and termination handling before changing servers.
The launch guide binds the native listener to loopback by default and documents no authentication configuration. Exposing it beyond the local machine therefore requires appropriate network controls. Its examples also reserve memory separately for the cache, weights and execution workspace, so a cache setting alone does not describe total GPU use.
Release notes report an image attention mask correction. ByteForward has not built or benchmarked this version. The linked guides track the main branch and were not verified against the tagged source. The findings here come from documentation rather than reproduced tests.
