Human-in-the-loop AI agent evaluation: a complete guide
Why AI agent evaluation still needs humans in 2026, where to put them in the loop, and how to combine human review with automated evals on production traces.
Blog
Practical posts on tracing, evals, prompt changes, token cost, and the production habits that keep AI products explainable.
Highlights from the Currai blog: the posts worth reading first.
Why AI agent evaluation still needs humans in 2026, where to put them in the loop, and how to combine human review with automated evals on production traces.
A practical field guide to LLM evaluation tools — what each category is good at, where they break down, and how to pick one that survives contact with production traffic.
The best AI observability tools in 2026 compared on evaluation depth, quality-aware alerting, drift detection, cost tracking, and the production-to-eval loop.
Browse implementation notes, observability guides, product decisions, and workflow ideas by topic.
Point an existing Langfuse SDK at Currai, validate trace parity in a canary, and cut over with a tested rollback path.
Read more ›OpenTelemetry is the open standard for traces. Currai ingests OTLP spans, so you can use the collector and instrumentation you already run and still get LLM-aware views.
Read more ›Total latency hides the metric users actually feel — time to first token. Here's how to capture both on every generation and find what's making your LLM app feel slow.
Read more ›Agents loop, call tools, and call themselves — a single request can be dozens of model calls. Here's how to trace agent runs so you can see exactly where one went off the rails.
Read more ›Observability data is high-volume and append-heavy — the classic case for running ClickHouse yourself. Here's the real trade-off between self-hosting and a managed backend.
Read more ›