LangSmith Releases Reusable Evaluators and 30+ Evaluator Template Library
Summary
- • LangSmith launches reusable evaluators — a centralized Evaluators tab to manage and apply evaluators across multiple tracing projects without duplicating them
- • 30+ evaluator templates now available covering safety/security, response quality, trajectory, user behavior, and multimodal evaluation
- • Templates work for both online production monitoring and offline experiment runs; include LLM-as-judge and rule-based code evaluators
- • openevals v0.2.0 also released simultaneously with new multimodal support for evaluating voice and image outputs
Details
Reusable Evaluators Tab
New Evaluators tab centralizes all evaluators in a workspace. Teams can attach any existing evaluator to a new tracing project in seconds without maintaining separate copies — keeping safety checks and quality metrics consistent across the organization.
30+ Evaluator Template Library
Templates cover 5 categories: safety/security (prompt injection detection, PII checks, bias and toxicity), response quality (correctness, helpfulness, tone), trajectory (did the agent take the right steps?), user behavior analysis (language distribution, satisfaction signals), and multimodal (voice and image review).
LLM-as-Judge and Code Evaluators Included
Templates include both LLM-as-judge evaluators with tuned prompts for subjective quality assessments, and deterministic rule-based code evaluators for objective checks. Use as-is or customize for the specific agent.
Online and Offline Evaluation Support
For online evaluation: templates categorize production traffic by detecting prompt injections, flagging unusual user behavior, or surfacing traces for human review. For offline evaluation: templates provide a starting point for running experiments across datasets — run, check scores, filter failures, understand what went wrong.
openevals v0.2.0 with Multimodal Support
Released simultaneously, openevals v0.2.0 adds multimodal evaluation support enabling assessment of voice and image outputs alongside text. The evaluator templates are also available in the openevals open-source framework.
Multi-Level Evaluation Required for Agents
A single evaluator checking the final answer won't catch whether a retrieval agent pulled the right documents or a planning agent delegated correctly. Effective agent evaluation requires testing individual steps, full trajectories, multi-turn conversations, and specific tool calls within a trace.
12+ Months of LangSmith Evaluation Tooling
Today's release builds on over a year of evaluation infrastructure: the openevals framework, Align Evals for evaluator calibration, and multimodal evaluator support. Reusable evaluators and the template library are the two most-requested features not yet addressed.
Sources: LangChain Blog, June 16 2026.
What This Means
For teams building AI agents, evaluation has historically been one of the highest-friction bottlenecks — requiring weeks to build evaluators from scratch with no way to reuse them across projects. LangSmith's update directly lowers that barrier: teams can now start with 30+ production-tested templates covering the hardest evaluation categories (safety, multi-step reasoning, multimodal) rather than building from scratch. The reusable evaluators tab ensures consistent quality and safety metrics across an organization without the maintenance overhead of duplicate copies. Combined with openevals v0.2.0's multimodal support, this positions LangSmith as a more complete evaluation platform as agents increasingly handle voice and image modalities in production.