RTK reports token savings, but our cost benchmarks disagree (quesma.com)

RTK (Rust Token Killer), a popular tool designed to filter and compress terminal output for AI coding, has garnered attention for its potential to reduce costs by streamlining data processing. With over 79,000 GitHub stars, it claims to cut terminal output by up to 90%. However, a recent analysis reveals that while RTK reported significant token savings—claiming to save 349.2 million tokens across various attempts—this did not translate to actual cost reductions across all tasks. In fact, testing showed that costs fell by only 5% for one AI model (Fable) and rose by 5% for another (DeepSeek), highlighting that reduced output does not equate to cheaper coding. The research emphasizes the complexity of optimizing AI interactions with terminal outputs. Key findings indicate that while RTK may enhance the efficiency of older models, its benefits are inconsistent and not universally applicable across different tasks or platforms. The study underscores a critical point for the AI/ML community: significant apparent savings in token counts can be misleading and might lead to increased overall costs due to additional agent turns needed for task completion. As such, RTK's utility appears limited and context-dependent rather than a go-to solution for cost-effective AI coding.