Esta edición aún no está disponible en español. Mostramos la edición en inglés.

A grounded case study in shipping an LLM product where mistakes have real human cost. SonderMind's team shows how licensed clinicians — not engineers — define correctness: their annotations of real conversation failures automatically become typed evals that gate every release. The counterintuitive lesson is that off-the-shelf guardrails were too conservative for mental health, blocking the very people seeking help.

• They disabled built-in frontier model guardrails on day one because over-triggering on legitimate mental health conversations 'feels like a door slam to the face' and increases isolation.
• Guardrails run as separate modular LLM-as-judge calls on input and output — more robust to jailbreaks, at a deliberate latency and cost trade-off.
• Clinician annotations of traced conversations (expected observations, turn index, category) auto-generate typed evals in CI/CD, so domain experts, not vibes, define edge-case correctness.
• They open-sourced 300 clinically reviewed guardrail scenarios (200 input, 100 output) as a shared baseline for mental health AI safety.

Parte de Edición Nº 003: Evals Are the New PRDs: How Anthropic, Bridgewater, and SonderMind Actually Ship AI