LangChain Launches LLM Gateway for Agent Runtime Governance

LangChainLangChain

LangChain launched LangSmith LLM Gateway, a runtime governance layer for AI agents. This gateway enforces cost limits and redacts sensitive data before requests reach LLM providers, integrating policy violations as traceable events directly within LangSmith for unified management.

LangChain introduced LangSmith LLM Gateway, a new runtime governance layer that sits between AI agents and the LLM providers they call. It enforces spend limits and redacts sensitive data like PII (Personally Identifiable Information) and secrets before requests reach the model. When a policy is triggered, such as a blocked request or redacted information, these events appear as traceable records within LangSmith.
Spend limit levels
Organization, workspace, user, API key
PII redaction types
Names, places, nationality, religion, political affiliation, ages
Secrets redaction types
US phone numbers, US SSNs, API keys, tokens, credentials
Setup
One-line base_url change
Supported providers
Anthropic, AWS Bedrock, Google Gemini, and others
Availability
Private beta

Running autonomous agents in production often leads to challenges like unexpected costs from runaway loops or the exposure of sensitive data to LLM providers. Existing solutions typically require integrating separate gateways and guardrails platforms, making it difficult to correlate policy enforcement with agent behavior. This update unifies governance with the agent development lifecycle, similar to how Vercel AI Gateway and OpenRouter offer centralized controls.

LangSmith LLM Gateway is available in private beta, with setup involving a one-line base_url change and policy configuration in the LangSmith UI. This allows teams to manage agent behavior, update system prompts, and re-evaluate against existing test sets without switching tools, providing real-time cost visibility and audit logging.

LangSmith Gateway policy configuration interface for managing organization, workspace, user, and API key spend limits.
LangChain LLM Gateway architecture showing request flow, policy enforcement, and integration with the LangSmith observability ecosystem.
View the full update on langchain.com
LangChain
LangChain
@LangChain
X

Most governance tools live in their own consoles or dashboards. LLM Gateway lives in LangSmith. When requests are blocked or info is redacted, you get traceable events. See what your agents do, update system prompts or tool configs, re-evaluate against existing test sets, all in one place.

2retweets15likes
View on X

Still wondering? A few quick answers below.

LangSmith LLM Gateway is a runtime governance layer that sits between your AI agents and the LLM providers they use, enforcing policies before requests reach the models.

It offers spend limits at various organizational levels, PII and secrets redaction, real-time spend visibility, audit logging, and trace continuity within LangSmith.

The gateway detects and redacts sensitive data like PII and secrets from requests and responses before they reach the LLM provider or are written to a trace.

Policy violations, such as blocked requests or redacted data, surface as traceable events directly within the LangSmith platform, integrated with LangSmith Engine.

Setup involves changing your agent's base_url to the LangSmith Gateway endpoint, adding provider API keys to workspace secrets, and setting policies in the LangSmith UI.

Every HeadsUpAI update is written based on its original source and reviewed before it's published. Read our editorial standards →

Share this update