LangChain Curbs Runaway Coding Agent Costs With LangSmith LLM Gateway

LangChainLangChain

LangChain deployed its LangSmith LLM Gateway internally to manage rising coding agent costs. The system enforces budgets across organizations, workspaces, users, and API keys, routing calls from tools like Claude Code and Codex. This centralized control provides real-time spend visibility and prevents runaway usage, maintaining budget predictability while allowing continued agent-driven development.

Availability
Private beta
Setup
One-line `base_url` change
Enforcement levels
Organization, workspace, user, API key
Key controls
Spend limits, PII and secrets detection, trace continuity, audit logging
View the full update on langchain.com
LangChain
LangChain
@LangChain
X

One dev using coding agents can rack up thousands of dollars in weekly spend before anyone notices. This happened to us at LangChain, so we built a solution. How we use LangSmith LLM Gateway internally. https://t.co/znIGQptryN

4retweets16likes
View on X

Still wondering? A few quick answers below.

LangSmith LLM Gateway is a runtime governance layer for AI agents, launched by LangChain. It acts as an intermediary between your AI agents and the large language model providers they interact with.

It solves problems like unexpected high costs from AI agents stuck in retry loops and the risk of sensitive data exposure (PII, secrets) in LLM provider logs. It enforces policies proactively to prevent these issues.

The gateway redacts sensitive data, including personally identifiable information (PII) and secrets, from both requests and responses before they reach the large language model or are written to a trace.

Setup involves a one-line change to your agent's base_url to point to the LangSmith Gateway endpoint. You then add your LLM provider API keys to workspace secrets and set policies in the LangSmith UI.

Key features include spend limits at various organizational levels, real-time spend visibility, PII and secrets detection, trace continuity within LangSmith, and audit logging for administrative actions.

Every HeadsUpAI update is written based on its original source and reviewed before it's published. Read our editorial standards →

LangChain

See all AI news & updates from LangChain →

Share this update