DGX Spark UMA can trick you (bartusiak.ai)

If you’re running LLMs on a DGX Spark and hit mysterious “out of memory” errors despite having plenty of RAM (e.g., 128 GB for a 7B model), the cause is often the machine’s Unified Memory Architecture (UMA) plus aggressive OS caching — not a buggy model. DGX Spark’s UMA means CPU and B10 GPU share the same physical memory pool. The Linux kernel aggressively uses free RAM for file/disk caches, and the GPU (and tools that query CUDA) can’t always distinguish genuinely used memory from reclaimable cache. That mismatch can trigger Hugging Face-style errors like “Some modules are dispatched on the CPU or the disk,” making it look as if the GPU lacks RAM even when the memory is only held by caches. The practical fix is simple: free the kernel caches so the shared memory is immediately available to GPU allocations by running as root: sudo sh -c 'sync; echo 3 > /proc/sys/vm/drop_caches'. Note this is a temporary, kernel-level operation (documented by NVIDIA) — the OS will rebuild caches as it runs — but it resolves many false OOMs and avoids unnecessary quantization/offload work. For teams using UMA systems, monitor both OS and CUDA-level memory, be aware of lazy cache reclaim behavior, and include cache-drop or controlled memory management in debugging workflows.