Hugging Face is being used to easily undress women and children (www.theverge.com)

A recent report from the nonprofit AI Forensics has raised alarming concerns about Hugging Face, the well-known platform for open-source AI models, being exploited to create nonconsensual deepfakes. The investigation revealed that seven out of the top nine image editing models on Hugging Face could easily generate images showing women and children undressed using straightforward prompts. Unlike other mainstream AI models, which implement safeguards against sexualized content, Hugging Face's lack of comprehensive protections has allowed these harmful requests to thrive. Over the span of a week, AI Forensics tracked over 1,000 prompts to honeypot Spaces, discovering that a staggering 73 percent had sexual content, with a significant portion aimed at women and alarming instances directed at children. This revelation is significant for the AI/ML community as it underscores the urgent need for robust ethical standards and technical safeguards in AI model deployment. Hugging Face's current situation threatens to undermine trust in AI technologies, highlighting how open platforms can inadvertently facilitate abuse without adequate oversight. AI Forensics has recommended implementing both prompt-level filtering and output-level scanning to prevent the generation of harmful content. However, as it stands, the lack of safeguards on Hugging Face poses a serious risk not only to individuals but also to the ongoing discourse around accountability and ethics in AI development.