Semantic Interaction Description – An accessibility standard for AI agents (sid-standard.github.io)

A new accessibility standard called Semantic Interaction Description (SID) has been announced, allowing web applications to provide structured metadata about their interactive elements. This metadata explains the functionality of each element and tracks results, enabling AI agents to understand applications similarly to a trained human user—efficiently and programmatically. Current methods, like DOM text extraction and using accessibility attributes, often lead to high costs and unreliable interactions due to ambiguities in element semantics and the need for vision processing. SID addresses these challenges by using a combination of HTML attributes and a JavaScript API. Developers can add specific attributes to elements, creating a clear description of their functions. Agents can then easily query this metadata, allowing for faster, more efficient automation without the drawbacks of traditional methods. Benefits of SID include universal compatibility for any agent that supports it, reduced token usage by avoiding vision processing, and reliable tracking of interactions. This development is significant for the AI/ML community as it promises to streamline web interactions, making AI agents more capable and reliable in navigating complex web applications.