Langfuse AI Gateway beta adds team keys and cost tracing

Langfuse opened its AI Gateway public beta on November 4, 2026, giving teams a central route for OpenAI and Anthropic calls with individual access keys and automatic usage records. The immediate value is knowing which person or service used a model without handing everyone the underlying provider credential. The beta still needs careful evaluation before its cost dashboard becomes a financial source of truth.

The launch announcement covers applications and coding agents, including Claude Code, Codex and OpenCode. Langfuse runs the cloud service in its EU, US, Japan and HIPAA regions. Model requests still go to the provider under the customer’s agreement, so a regional trace store does not mean the entire request stays there.

One gateway, native provider APIs

According to the architecture documentation, the gateway supports OpenAI Responses, OpenAI Chat Completions and Anthropic Messages. It preserves each API format rather than translating between them. An OpenAI client cannot use that route to call a Claude model.

Clients change their base URL and key. Provider credentials stay behind the gateway, which lets an organization rotate one without reconfiguring every client. Multiple credentials are supported. A prefixed model identifier selects a particular credential, while compatible unprefixed model names can use automatic matching. Teams with several provider accounts should make that choice explicit during testing.

Individual keys make access easier to unwind

The key documentation distinguishes a person’s key from a service account key. A user key stops working when its owner loses gateway access. A service key survives membership changes, making it more suitable for CI and shared applications. Both support expiry and revocation.

Keys can carry team or cost center metadata that is attached to recorded generations. That metadata is fixed when the key is created, so changing it means replacing the key. This makes a naming and ownership convention useful before a team distributes keys widely. A shared service should have an identifiable owner even when its access is not tied to that person’s membership.

Model controls also have a practical default to check. The model management guide says a credential with every model enabled automatically permits later models. Disabling one switches it to an explicit list, leaving new models unavailable until an administrator enables them. Organizations requiring review before adoption should verify that setting rather than assume new models arrive blocked.

Decide what the traces should contain

The default USAGE mode records operational information without storing prompts or responses in Langfuse. FULL mode adds input and output, including tool definitions and returned tool calls. The tracing guide describes organization-wide configuration, not a separate preference each developer chooses.

That creates a decision before rollout: whether the team needs usage attribution alone or the content required to debug and evaluate responses. Set access and retention accordingly. FULL collection can be valuable, but it expands the material available to people who can inspect traces.

The gateway sees model calls, not every retrieval or tool execution around them. Broader application tracing still requires instrumentation. ByteForward’s Dify 1.19.0 tracing coverage illustrates why the steps inside a workflow can matter when the model request itself succeeds.

Pricing includes Langfuse usage and provider charges

The billing explanation separates the two bills. Each recorded generation consumes a Langfuse billable unit. OpenAI or Anthropic charges model usage to the customer’s own provider account. Using the gateway does not replace that provider bill.

The current cloud pricing page lists a free Hobby plan with 50,000 monthly units, two users and 30 days of data access. Core costs $29 per month with 100,000 units, unlimited users and 90 days of access. Pro costs $199 per month with 100,000 units and three years of access. Paid plans list additional usage at $8 per 100,000 units before volume discounts. These are platform allowances, not a separate pool reserved for gateway requests.

The beta limits change the rollout decision

Langfuse’s known limitations list a 16 MiB request cap, 600-second call timeout and 120-second idle timeout. Inputs over 5 MiB are omitted from traces. Outputs over 1 MiB are recorded partially. Realtime and websocket APIs, retries, fallback and caching are absent.

Cloud quotas cover all organization traffic. Hobby allows 100 requests and 64 MiB per minute, Core 500 and 256 MiB, and Pro, Team or Enterprise 10,000 and 5 GiB. Either limit can trigger a 429.

Cost figures also need checking. OpenAI failures and cancellations use estimates, OpenAI cache writes are priced at zero, and Anthropic one-hour cache writes use the five-minute rate. Compare totals with provider billing before chargebacks.

Start with visibility, test the controls you need

The published roadmap places hard spending caps, provider fallbacks, PII redaction and automatic model selection in future work. A team looking mainly for attributed usage and centralized keys has a clearer reason to try this beta than one that needs those additional controls today.

A useful pilot would cover ordinary calls, a long coding session, cancellations and a rate-limit response. Compare recorded usage with the provider account, inspect the chosen collection mode and confirm how access ends when a person leaves. Keep the scope small enough to understand discrepancies before expanding it.

ByteForward reviewed the launch and documentation, but has not run the gateway or independently measured its latency, reliability or accounting accuracy. The beta offers a practical way to start collecting model-call evidence. Its value depends on understanding which evidence is complete enough for the decision being made.