I asked a psychologist what worries the people trying to make AI safer (www.techradar.com)

Genevieve Bartuski, a psychologist and AI risk advisor, emphasizes the need for responsible design in AI health and wellness applications as concerns grow over the emotional attachments users form with these tools. She warns that while AI can serve as a valuable supplement to therapy, it cannot replace the nuanced human connection that is crucial for therapeutic success. As AI tools increasingly position themselves as companions and support systems, the risks of misuse, misinterpretation, and emotional dependency become more pronounced, especially among vulnerable populations like children. Bartuski highlights the importance of understanding AI's limitations, such as its potential to “hallucinate” or provide misleading information. This poses a risk not only to individual mental health but also raises legal and ethical concerns as AI applications expand into wellness. As companies rush to innovate, she advocates for a deliberate pace in AI development, urging both creators and users to remain cautious about outsourcing critical thinking and emotional support to technology. The implications of failing to address these risks could lead to significant harm, underscoring the necessity of human oversight in AI's evolving role in mental health care.