Lichen – A local, BOY model system1 (Jev) server with image support (github.com)

Lichen, a new local server developed as a drop-in replacement for TypeSafe's Jev model, has been announced. It operates on open-weight models, retains the same API endpoint as Jev, and even allows existing clients to transition seamlessly. Notably, Lichen can process images alongside text requests, enhancing its versatility by enabling questions based on the visual content. For instance, with the gemma-4-26B-A4B model running on a laptop GPU, Lichen achieved 204 out of 231 public decisions on the JevBench benchmark, outperforming Jev 1.13 significantly in terms of speed and accuracy. The significance of Lichen lies in its technical advancements, particularly in how it processes prompts and calculates probabilities. By employing unique prompt repetition and reordering techniques, Lichen enhances answer accuracy while reducing computational overhead. Its design allows it to generate response probabilities directly from a single model pass, thereby streamlining performance and improving confidence calibration. Lichen's successful integration of image analysis within its API holds substantial implications for AI/ML applications, making it a strong contender for tasks requiring multimodal understanding in local environments, and possibly paving the way for more efficient AI systems in real-world applications.