Stop Using Ollama (sleepingrobots.com)

Ollama, a widely used platform for running local large language models (LLMs), is facing criticism for its lack of transparency and significant drift from its original mission. Initially praised for simplifying access to llama.cpp, the foundational inference engine for local LLMs, Ollama has reportedly obscured its reliance on this technology. In a series of controversial design choices, the company avoided crediting llama.cpp in its documentation and has recently moved to a custom implementation, which is yielding performance issues and bugs that were previously resolved in llama.cpp. Benchmarks show that llama.cpp outperforms Ollama's implementation significantly, running models 1.8 times faster, leading to user frustrations over slower execution and broken model features. Additionally, Ollama's recent decisions to operate with a closed-source desktop application and impose a new configuration system (the Modelfile) have sparked backlash from the community. The Modelfile process complicates model management, requiring users to duplicate model files for minor changes—a stark contrast to llama.cpp's straightforward command-line parameter adjustments. The platform's limitations regarding model quantizations and delayed responses to new model releases further alienate its user base. As Ollama shifts towards cloud-hosted models, it risks losing the trust it initially built as a local-first solution for AI applications, prompting a call for the AI/ML community to reconsider its use.