I Just Deleted All My Agent Skills (philippdubach.com)

In a recent reflection, Jakob Breu highlighted the striking findings from the SWE-Skills-Bench benchmark, where 92 out of 98 public software engineering skills showed no substantial improvements in coding task outcomes. Interestingly, web development skills not only diminished overall success rates but also significantly increased token costs by up to 394%. This analysis has led some in the AI community, including Breu, to reevaluate the utility of such skills, with some opting to delete them entirely from their agent models. These revelations are significant for the AI/ML community as they challenge the established belief in the necessity of extensive skill sets for enhancing model performance. The performance differences between public and curated skills are telling; curated skills lifted pass rates from 33.9% to 50.5%, suggesting that while not all skills are detrimental, a more selective approach could yield better results. As Anthropic demonstrated by trimming over 80% of Claude Code's system prompt without losing effectiveness, the trend indicates a movement towards simplification and refinement of AI capabilities to enhance operational efficiency while reducing costs. This shift calls for a critical reassessment of the role of skills in AI development, focusing on quality and contextual relevance over quantity.