Bobbin: Pentest with a local LLM, so target data never leaves your machine (github.com)

Bobbin, a new coding agent, has been introduced to enhance penetration testing using local language models (LLMs) without data egress concerns. Designed for seamless operation via Ollama, Bobbin functions independently of external libraries and is optimized for smaller models, ranging from 7B to 32B parameters. The agent's structured loop allows it to address coding tasks—such as refactoring and bug fixing—by executing model-generated tool calls, iterating until it either produces a prose answer or depletes its step budget. The project showcases extensive performance evaluations, noting that while it achieves parity with existing tools like Aider in some areas, it excels in multi-file refactors and exhibits a lower incidence of incorrect modifications. Significantly, Bobbin caters to scenarios where sensitive code cannot be exposed to hosted LLMs due to privacy regulations. It offers two evaluation suites targeting code review and security testing against local applications, allowing for structured vulnerability assessments without triggering false positives. The initial results indicate that while the tool’s recall for finding vulnerabilities is high, its effectiveness in exploiting certain classes of vulnerabilities varies based on the model used. Consequently, Bobbin positions itself as a useful first-pass reviewer that requires human intervention for final judgments, maintaining a commitment to transparency by enabling users to verify its performance through reproducible runs and comprehensive documentation.