vLLM 0.32 adds decision serving and stricter browser access
The vLLM project has released version 0.32.0 of its model serving software, adding a structured decision endpoint, wider support for compact attention caches and stricter browser access defaults. The changes affect both what applications can ask a server to do and how operators deploy it.
The stable GitHub release was published on October 18, 2026 at 06.50 UTC. PyPI distribution files followed at 10.03 UTC. Several underlying changes had already merged into the source tree. The new milestone is their delivery in a versioned release.
A serving endpoint for defined decisions
The structured decisions documentation describes /v1/systemone, which accepts a state and typed questions, then returns probabilities over the allowed answers. Formats cover multiple choice, yes or no and ordered scores. It is available by default on supported generation models. Unsupported models or read strategies return an error rather than acquiring decision capabilities automatically.
The same documentation warns that identical requests can produce different probabilities depending on batching and cache use. A reproducibility setting trades throughput for repeatable reads. These scores still need evaluation against known outcomes before an application uses them to route work or trigger actions.
ByteForward’s Microsoft Decision 1 report explains the broader use of supplied choices for routing and evaluation. For teams serving their own models, a common decision interface could simplify the application layer while leaving model selection and error handling with the operator.
Another change concerns output parsing. The generic HF parser reads a checkpoint’s response template to extract tool calls and reasoning. It requires suitable metadata and cannot simply be combined with a different tool or reasoning parser. That gives model publishers a route to describe output formats without every deployment relying on a separate model specific parser.
Memory savings and throughput need separate measurements
For AMD serving, the UltraQuant integration stores attention cache data at four bits. Its author reports roughly twice the cache density of FP8 and stronger throughput as long requests fill the available cache. The benchmark notes also disclose different software bases across measurements, so the absolute throughput figures are indicative. This is a cache format change, with hardware and kernel requirements to check.
On the request handling side, connection balancing sends new connections to the least busy API worker when several workers are configured. The contributor reports 43 to 85 percent more requests per second in tests using Qwen3-0.6B with 32 input tokens and one output token. A single API worker is unchanged. These results describe a narrow serving workload, not a general increase in model generation speed.
A deployment test should therefore separate memory pressure from request handling overhead. Record the model, hardware, concurrency and input lengths. Compare time to the first token and slower requests alongside throughput. A result from tiny classification requests cannot establish the same benefit for lengthy generated answers.
Operators face compatibility and access changes
The published package requires Python 3.11 or newer, below Python 3.15. An environment still on Python 3.10 needs a runtime change before this version can be adopted.
The browser access change removes the default wildcard for allowed origins in the Python and Rust frontends. Operators serving browser applications must configure trusted origins explicitly. The change governs browser response access and preflight behavior. It does not replace API authentication or prevent every request from reaching a server.
The release also lists fixes for resource exhaustion through chat templates and peer sockets, plus a tokenizer boundary issue. Its migration notes include renamed reasoning stream events and changes to context window handling. Existing clients should be checked against those interface changes.
For an existing service, the useful next step is a controlled rollout using recorded requests and a rollback plan. Verify browser clients, tool parsing, decision outputs and latency before moving traffic. ByteForward has inspected the release records and source documentation but has not run the new version or reproduced the performance tests.
NVIDIA HGX B200 hardware photographed in 2025, used as context for AI model serving. Photo by Pokiiri via Wikimedia Commons under Creative Commons Attribution ShareAlike 4.0. Converted to WebP. The adapted photograph uses the same license.
