The manual check that saved them won't scale (hextrap.io)

Last month, a software consultancy named Softjourn narrowly avoided a malware incident after a developer, using an AI coding agent, was about to install a suspicious package. The package, recommended by the AI, showed few downloads and was very new, prompting a manual check that revealed its potential danger. This incident highlighted a significant concern within the AI/ML community: relying on human judgment in automated processes can be perilous, especially under time pressure. The problem arises from a phenomenon called "slopsquatting," where attackers exploit AI-generated package names that are plausible yet nonexistent. The story emphasizes the need for automated checks to ensure package integrity—specifically cooldown periods for newly created packages and reputation assessments based on downloads and maintainer history. Softjourn's experience serves as a reminder that while humans can provide valuable oversight, it's critical to implement robust automated systems that enforce security protocols in real-time. By doing so, developers can prevent security breaches while reducing the burden on individual engineers to perform repetitive checks that may be missed when under deadlines.