Show HN: LLM-budget-cap – atomic Redis spend cap for LLM APIs (github.com)

A new tool, LLM-budget-cap, has been released to help developers manage expenses associated with large language model (LLM) APIs from providers such as OpenAI and Anthropic. This innovative solution uses a Redis counter to enforce a spending limit on API calls, running two crucial commands—INCR and PEXPIRE—atomically within a single Lua script. This atomic execution ensures that simultaneous requests do not inadvertently exceed the defined cost limits, preventing unexpected high bills caused by bugs, loops, or malicious activity. The significance of LLM-budget-cap lies in its ability to provide a fail-safe mechanism for projects heavily reliant on LLMs by capping API calls at a user-specified limit while maintaining flexible configurations, such as customizable time windows and individual counters per tenant, user, or IP. Unlike general-purpose rate limiters, this library targets cost control specifically. The tool also incorporates a fail-open feature, allowing operations to continue if Redis goes down, thereby keeping the application's functionality intact while alerting developers to potential service issues.