Show HN: `tc` like `wc` but for LLM tokens (github.com)

A new command-line interface tool called `tc` has been introduced to count tokens for large language models (LLMs), akin to the traditional `wc` command used for word counting. Users can leverage `tc` to analyze token counts in individual files, entire projects, or even as part of a pipeline with standard input. The tool is designed to assist developers by providing specific token metrics for LLMs, enabling them to gauge the input sizes before submitting content to language models. For instance, developers can quickly assess how their code or text compares in token count to well-known literary works. Significantly, `tc` supports different encoding formats, including both the default `o200k_base` and `cl100k_base`, tailored for various LLMs like GPT-4 and Claude. This functionality is crucial as it allows users to see how variations in encoding impact token count, which can influence model performance and costs associated with API usage. Additionally, the tool can output results in JSON format, making it easier to integrate into automated workflows. Overall, `tc` fills a critical gap for developers working with LLMs by simplifying the token management process, thus enhancing project efficiency and ensuring optimal interaction with LLM APIs.