Custom Testing Center scorers
Key Takeaway
A custom scorer is an LLM judge: an AI model you choose grades the agent's sessions against criteria you write in an evaluation prompt.
Custom scorers let you grade an agent's sessions against criteria you define. You build and test a custom scorer in Testing Center. Then you can activate it to score a live agent's sessions.
Where custom scorers fit in a Testing Center test
A Testing Center test can use three kinds of scorers. Custom scorers sit on top of the other two.
| What they measure | How a test uses them | |
|---|---|---|
| Default scorers | Response accuracy, subagent assertion, and action assertion | Every Testing Center test runs them. |
| Quality scorers | For example, completeness, coherence, conciseness, and latency | You can add them to a test. |
| Custom scorers | Criteria you define, such as tone, brand voice, policy adherence, resolution quality, sentiment, a business outcome, or a structured output like JSON | You add them on top of the other scorers to measure what your business cares about. |