Infernix Windows release fixes cache exhaustion

Infernix published its first standalone Windows package on October 9. The Qwen engine grew from NInfer, with another release that day repairing cache exhaustion.

Hardware beyond GPU memory

Infernix specifies an RTX 50 series GPU, NVIDIA driver 580 or newer and Visual C++ runtime. No CUDA toolkit is required. Linux requires a source build.

For Qwen 3.8 Flash Next on an RTX 5090, the guide suggests around 96 GB RAM, roughly 80 GB of model files and a separate 52 GB table on NVMe. Experts occupy host memory with frequently used experts cached in VRAM.

NVIDIA lists 32 GB of GDDR7 memory for the RTX 5090. That figure describes memory on the graphics card, separate from the RAM installed on the motherboard. A hardware inventory should record both. Having the right GPU model does not establish how much system memory remains available once other applications are open.

There is a practical reason to leave room for the rest of the computer. NVIDIA’s CUDA guidance explains that pinned memory improves transfers between host and GPU, but excessive pinning can reduce overall system performance. The memory cannot be treated as an unlimited pool without consequences for other work.

For another perspective, ByteForward’s coverage of local model memory tradeoffs examines a separate laptop comparison.

Check the Windows runtime separately

Microsoft’s runtime download guidance says the redistributable architecture must match the application and the installed version must be at least as recent as the build tools used to create it. For an x64 application, follow the X64 entry. The runtime supplies libraries applications need to start and run. Its presence does not validate the GPU driver, model files or available memory.

What changes when upgrading

Release notes say recomputed cache blocks could consume duplicate pages and leave later requests returning HTTP 503. The repair also removes options and changes log and metric schemas. Logging moves to schema 27. A concurrency test still fails.

The notes also report small prefill slowdowns and lost reuse in some cache setups without host memory. The maintainer describes hobby development. Reported performance uses one machine and is not independently verified here.

NVIDIA’s verification guidance recommends comparing representative inputs with known good results after changes. Performance checking belongs alongside correctness checking. Its profiling guidance also emphasizes realistic workloads, since an artificial test can direct attention toward the wrong bottleneck.

For a local agent setup, a useful trial should answer questions the daily workflow will actually ask. Our suggested checks are deliberately small enough to repeat after each upgrade.

• Save the previous working launch configuration and record the model file version before changing the engine.
• Run a short conversation, then a longer conversation using the same client that will do the real work.
• Include a tool request, cancellation and another request afterward. Check the client receives a usable response rather than merely an open connection.
• Record time to the first visible token and total completion time separately. Repeat the same workload, noting whether the cache starts empty or already contains prior work.
• Inspect the logs and monitoring display after the trial. A successful reply alone does not establish that every monitoring field is still being read correctly.

Keep the first evaluation isolated from important work. These are suggested acceptance checks, not results from tests performed by ByteForward.

Featured photo shows a Palit GeForce RTX 5090 photographed in May 2025. Photo by PantheraLeo1359531 via Wikimedia Commons. CC BY 4.0. Rotated and resized with a white background.