Sherlock – See what's being sent to LLM APIs in real-time (github.com)

Sherlock has been launched as a real-time traffic inspector for Large Language Model (LLM) APIs. This transparent proxy intercepts HTTPS traffic, allowing developers to monitor token usage and context window limits through an intuitive terminal dashboard. Key features include real-time tracking of token consumption for each request, a visual progress display that alerts users to their limit status, and automatic saving of prompts in both markdown and JSON formats for easier debugging. Notably, Sherlock requires no code changes, making it accessible for developers using any tool that respects proxy environment variables. The significance of Sherlock lies in its ability to provide actionable insights into LLM API interactions, enabling developers to optimize prompt performance and manage associated costs effectively. By displaying cumulative token usage and offering visual indicators of usage levels, developers can prevent exceeding their token limits, thus avoiding unexpected charges. Additionally, the tool's compatibility with major LLM providers like Anthropic, with support for others like OpenAI and Google Gemini on the horizon, positions Sherlock as a vital utility for the AI/ML community looking to enhance the experience of working with LLM APIs.