Prime Inference opens serverless and reserved model serving
Prime Intellect launches a serving platform for open models, with serverless access, reserved capacity and important differences between hosted and gateway requests.
Prime Intellect released Prime Inference on October 2, offering serverless endpoints and reserved capacity for open models. The service targets variable demand and sustained workloads.
The company says its first public GLM 5.3 deployment reached OpenRouter on September 22. This announcement expands the public serving platform around an existing model.
For application teams, the practical question is what changes when they move traffic. Access to a familiar model is only part of that decision. Hosting arrangements, billing behavior and the quality of completed work matter too.
Check who actually runs the model
Prime’s inference documentation distinguishes two serving types behind the same API. Hosted models run on Prime infrastructure. Gateway models send requests to an external provider. Both use the same endpoint and API key, so a shared interface does not mean every request follows the same infrastructure path.
The Models API guide directs developers to the current catalog for model identifiers, serving types, pricing and specifications. Availability and rates can change. The returned identifier is the one to use when selecting a model.
That distinction should be part of a deployment review. A team choosing Prime specifically for its own infrastructure needs to verify the selected entry, rather than assuming the platform name settles the question.
For GLM 5.3 specifically, Prime documents a hosted text model with a context window of 1,048,576 tokens and maximum output of 131,072 tokens. Reasoning stays enabled, with low, high or max effort settings and max as the default. Other GLM variants can have different capabilities and serving providers.
Integration includes billing and measurement
The Chat Completions API accepts requests compatible with OpenAI’s format and supports streaming responses. Developers can request token counts and cost in the response through Prime’s usage extension. Supported parameters still depend on the selected model.
Billing deserves a separate check. Direct API requests charge the key owner’s personal account unless a team identifier is included in each request. Choosing a team in the CLI configuration does not automatically apply that choice to direct HTTP requests or SDK calls.
Before sending a larger workload, confirm the billed account and inspect a small test response. Keep a record of the model, reasoning setting and usage details so later comparisons measure the same setup.
Keep performance claims tied to the test
Prime describes separating prompt processing from token generation and reusing cached conversation state. In its GLM 5.3 tests on GB200 NVL72, the company reports roughly 50 percent more cache capacity with NVFP4 compression while maintaining comparable speed per user with decode prefix caching enabled.
That result concerns a particular serving configuration. It does not establish the cost or reliability of an application running a different mix of requests. ByteForward has not independently reproduced the tests.
Our coverage of DeepSeek’s Ascend kernels explores the related distinction between component benchmarks and the behavior of a complete service.
The launch post lists batch and asynchronous inference, plus dedicated deployments on reserved capacity, as roadmap work.
A useful pilot should include both returning conversations and new requests, record waiting time and completed task cost, and inspect tool failures. Compare those results with the current provider before committing steady traffic. The value of another serving option depends on whether it meets those practical requirements.
Featured image is an original AI generated conceptual illustration of inference traffic and computing capacity.
