Guide: Setting up Nvidia DGX Spark with ggml (github.com)

This guide shows how to turn an NVIDIA DGX™ Spark into a local, private AI assistant using the open-source ggml software stack. Aimed at developers and builders, the setup is executed with a single command (bash <(curl -s https://ggml.ai/dgx-spark.sh)) that downloads and builds the latest ggml/llama.cpp-based runtime and model weights; the first run may take a few minutes while weights are fetched. Once installed the system can operate offline, hosting multiple AI services on the same device thanks to the DGX Spark’s 128 GB of unified memory. Technically, the configuration exposes endpoints for chat, inline code completions (FIM), embeddings, multimodal document and image analysis (vision endpoint), and speech-to-text, and can be integrated into developer tools such as VSCode via the llama.vscode extension for completions, embeddings and tooling. Performance tuning and benchmarks reference llama.cpp on DGX Spark, highlighting that ggml’s CPU/GPU-optimized inference enables simultaneous services on-prem. The takeaway: a reproducible, developer-friendly path to run private, high-capacity LLM workloads locally, suitable for coding assistants, multimodal analysis and other edge/enterprise use cases without cloud dependency.