LangChain Launches openevals and agentevals for LLM Evaluation

LangChain releases openevals and agentevals packages for LLM evaluation

The two new packages provide a set of pre-built evaluators and a common framework designed to lower the barrier for developers who need to assess LLM output quality before bringing applications to production.

LLM-as-a-judge is the most common eval type and requires no ground truth

openevals ships with pre-built starter prompts, few-shot examples, and scoring schemas. Because it is reference-free, it is well-suited for evaluating conversational quality, hallucination in summarization or QA, and writing coherence without needing labeled ground truth data.

openevals supports structured data evaluation via exact match or LLM-based validation

For applications that extract or generate structured output, openevals provides two validation modes: deterministic exact match for clear-cut cases and LLM-as-a-judge validation for more nuanced structured content.

agentevals introduces trajectory evaluation for agent step-and-tool-call sequences

Trajectory evaluation assesses the full sequence of steps and tool calls an agent executes, addressing evaluation challenges in agent systems where correctness depends not just on final output but on the path taken to reach it.

LangChain plans to expand coverage to RAG and multi-agent architectures

Current evaluator coverage targets LLM-as-a-judge, structured extraction, tool calls, and agent trajectories. RAG and multi-agent evaluation are listed as future expansion areas, signaling a roadmap toward comprehensive production-readiness tooling.

Data quality and diversity matter more than dataset size when starting evaluations

The guidance recommends curating a small set of high-quality, use-case-specific data points rather than amassing large generic datasets, noting that even a handful of well-chosen examples can produce evaluations that reflect real-world usage.