Show HN: Jeva.cpp – a llama.cpp fork with JEV-compatible API for all LLMs (github.com)

The newly announced jeva.cpp is a fork of the llama.cpp project, integrating a JEV-compatible decision API that allows for direct Choice, Score, and Noul evaluations from model logits. This enhancement preserves the standard autoregressive generation while extending the functionality to all models compatible with llama.cpp. By requiring models that output next-token vocabulary logits, jeva.cpp presents a versatile solution that maintains original capabilities for other model types. This development is significant for the AI/ML community as it streamlines LLM (Large Language Model) and VLM (Vision Language Model) inference with minimal setup while ensuring high performance across diverse hardware setups, including optimized support for various architectures like ARM, x86, and RISC-V. Key features include integer quantization for improved speed and reduced memory usage, custom CUDA kernels for NVIDIA GPUs, and hybrid inference capabilities that combine CPU and GPU resources. The project aims to simplify deployment and enhance accessibility for developers implementing LLM solutions in both local and cloud environments.