Dependencies in the LLM API Reseller Ecosystem (arxiv.org)

A recent study introduces CacheTracer, a groundbreaking tool that uncovers hidden dependencies in the Large Language Model (LLM) API reseller ecosystem. As LLM API resellers serve as crucial intermediaries to access modern AI services, they often operate in a complex, multi-tiered system that shrouds the supply chain in opacity. This can lead to significant risks to confidentiality and integrity, as user requests may pass through multiple resellers, each capable of altering content. CacheTracer uniquely leverages prefix-cache reuse as a side channel to reveal these hidden dependencies, conducting a comprehensive analysis of 39 reseller endpoints and 1.1 million API requests. The findings are striking: over 37% of the tested endpoint pairs exhibited shared cache reach, highlighting a dense structure of dependencies that can span multiple layers. This intricate web suggests that vulnerabilities in upstream services could have widespread repercussions, potentially jeopardizing users connected to various downstream resellers. By identifying these risks, CacheTracer not only enhances understanding of the LLM API landscape but also underscores the urgent need for improved transparency and security measures within the ecosystem, ensuring that the AI/ML community can navigate these complex interactions safely.