Show HN: Psychosis Guard – Safety for long LLM conversations (github.com)

The "Psychosis Guard" has been introduced as a trajectory-aware safety middleware designed to enhance the safety of long-running conversations with large language models (LLMs). Unlike traditional safety mechanisms that evaluate messages individually, Psychosis Guard continuously monitors the conversation's trajectory, assessing cumulative risk and intervening as needed with clinically-informed responses. This system allows it to effectively address and redirect discussions that may subtly steer towards delusional or harmful content, even when no single user message activates a conventional content filter. This innovation is significant for the AI/ML community as it offers a more nuanced approach to mental health safety in AI interactions, particularly in conversational AI. The open-source, model-agnostic design can be integrated seamlessly as an HTTP proxy or Python library, making it accessible for various applications. Key technical features include a graduated response system that can adjust interventions based on risk levels and a "Trajectory Rail" that captures gradual shifts in user sentiment, ensuring timely and appropriate responses to potentially harmful dialogues. Through rigorous evaluation, Psychosis Guard has demonstrated improved metrics for detecting delusions and enabling safety interventions, highlighting its potential as a critical tool for responsible AI deployment without compromising user experience.