Show HN: Osaurus – Ollama-Compatible Runtime for Apple Foundation Models (github.com)

Osaurus is a native, Apple Silicon–only local LLM server and desktop app from Dinoki Labs that runs on Apple’s MLX runtime to maximize performance on M-series Macs and (when available) routes requests to Apple Foundation Models accelerated by the Apple Neural Engine. It ships as a SwiftUI app with an embedded SwiftNIO HTTP server, exposes OpenAI‑compatible (/v1/models, /v1/chat/completions) and Ollama‑compatible (/chat with NDJSON) endpoints, and supports SSE and NDJSON streaming, OpenAI‑style function/tool calling (including streaming tool_call deltas), model management (download curated MLX models via Hugging Face), real‑time CPU/RAM monitoring, and a local UI/CLI. Requires macOS 15.5+ and M1 or newer; Apple Intelligence features need macOS 26. Significance: Osaurus lets macOS developers and power users run high-performance, private LLMs locally while remaining compatible with existing OpenAI and Ollama client libraries, reducing friction for app integration and offline/edge use cases. Technically notable are its MLX/MLXLLM optimization for ANE-backed inference, robust function‑call parsing that tolerates formatting noise, path normalization for common API prefixes, and reproducible benchmarks (example: llama-3.2-3b-instruct-4bit — TTFT 87 ms, total 1,237 ms, 554 chars/s across 20-run averages). It’s open to development (Xcode 16.4+), installable via Homebrew cask, and includes scripts to reproduce benchmarks — but be cautious exposing the server on LAN since there’s no built‑in HTTP auth.