Quality metric scorers in Testing Center
Key Takeaway
Quality metric scorers use an LLM judge to grade the response itself. Most use a 0–5 scale, and a score of 3 or higher passes.
Testing Center grades each test case with scorers, so you can see where an agent or prompt template falls short. Quality metric scorers rate qualities of the response, such as completeness, coherence, and conciseness. You use the scores to decide what to fix in the agent or the prompt template.
Default evaluations and quality metric scorers
Agent tests in Testing Center include default evaluations, which run automatically on every test. Default checks such as Subagent Assertion and Action Assertion score 1 (pass) or 0 (fail). The score depends on whether the agent picked the right subagent and actions.
You can also add optional quality evaluations. Unlike the default checks, which look at the agent's choices, quality metric scorers judge the response itself.