The risks, safeguards, and responsibilities of building technology that interacts with human vulnerability.
AI is being hailed as the next frontier for early mental health detection, but its inherent 'sycophancy' creates a dangerous blind spot for complex risks like psychosis. We need a system designed to doubt its own intelligence.
AI is already outperforming clinicians in trials. But when something goes wrong, who is accountable? From a global regulatory vacuum to the startups quietly building the safety stack, this is what a responsible AI mental health architecture actually looks like.
If you're building in mental health, digital health, or regulated care and don't have a product function yet, or have one that's not working, I'd love to hear about it.