Logsum compresses log files to summary that matters, then ask your LLM (github.com)

Logsum, a new lightweight CLI tool, has been introduced to optimize raw application logs for consumption by large language models (LLMs). By transforming verbose logs filled with redundant details into a compact, token-efficient Markdown format, Logsum helps streamline the data that LLMs process. This tool eliminates unnecessary timestamps, repetitive messages, and boilerplate stack traces, reducing the noise and enhancing diagnostic accuracy while keeping costs low by minimizing token usage in API calls. The tool operates deterministically, detecting log formats and extracting key structured fields to create a concise Analysis Matrix highlighting significant anomalies. It effectively deduplicates and compresses log entries, preserving only essential error and warning messages along with critical context. This is especially valuable for developers and data scientists who need precise and actionable insights from numerous log files without the overhead of irrelevant data, ultimately improving both the efficiency and effectiveness of machine learning applications and diagnostic processes in software development.