Show HN: Speck – Cognitive architecture for small local LLMs (github.com)

Speck is a groundbreaking cognitive runtime designed to enhance the capabilities of small language models (LLMs) by transitioning cognitive functions such as memory, planning, and evidence tracking from the prompt directly into persistent software infrastructure. Unlike traditional approaches, where LLMs manage all cognitive tasks autonomously, Speck separates cognitive state from model inference, enabling a more efficient use of resources. This architecture allows for the unloading and swapping of models without losing the cognitive context of tasks, demonstrating significant potential for small local models, which often struggle with the complexity typically managed by larger counterparts. The significance of Speck lies in its innovative approach to cognition in AI systems, particularly for smaller models with limited capabilities. By offloading deterministic tasks to a specialized cognitive architecture, Speck enhances performance and efficiency while reducing the cognitive burden placed on LLMs. Key features include memory competition for attention, independent task validation, and the ability to use various models for different cognitive roles, allowing for more flexible and effective AI implementations. Ultimately, Speck aims to streamline cognitive processes to create powerful agents that leverage smaller models in a more sophisticated manner, leading to more capable and adaptable AI systems.