Show HN: Log every LLM call with tokens, cost, latency, and time to first token (github.com)

A new self-hosted proxy tool, llmtee, has been unveiled to enhance the tracking of API calls for large language models (LLMs) from providers like OpenAI and Anthropic. By integrating llmtee into existing workflows, users can log vital information such as token usage, costs, latency, and time to first token without altering their code. This tool runs as a single Go binary with SQLite or PostgreSQL, allowing for effortless backups while providing a comprehensive web UI for data visualization, CSV export, and Prometheus metrics. The significance of llmtee lies in its ability for individuals and small teams to not only monitor their LLM expenditures but also set daily and monthly budgets for each API key, rejecting requests that exceed these limits. By facilitating an understanding of usage through a well-structured dashboard and offering detailed request logs, llmtee empowers users to manage their AI spending effectively. Notable features include the ability to log responses and costs with minimal added latency, making it a practical solution for real-time monitoring, although it requires careful implementation to ensure that system outages do not disrupt LLM calls.