Toxicity detection on generated responses
Key Takeaway
Toxicity detection on large language model (LLM) responses is always on. When a response comes back into the Salesforce Trust Boundary, the Einstein Trust Layer scores it for toxicity first, and only then demasks any masked data.
Toxicity detection is an Einstein Trust Layer feature that uses machine learning models to find and flag toxic content in LLM responses, and optionally in prompts. It matters because customer-facing AI output speaks in your company's brand and voice, so toxic output can harm your company's reputation.
When detection runs on a response
Under the zero data retention policy, external LLM partners don't keep prompts or responses, so Salesforce handles its own content and abuse moderation in the Einstein Trust Layer.
AI can sometimes generate toxic content, so the Einstein Trust Layer checks the output before it reaches users. The check happens in this order:
- The LLM response passes back into the Salesforce Trust Boundary.
- As it does, the Einstein Trust Layer scores the response for toxicity.
- Only then does the Einstein Trust Layer demask any masked data.
- The user sees the response.