Can you self-host AI on Intel NPU or ARC (iGFX and proper card? (github.com)

A new local server solution allows users to self-host AI applications on Intel hardware, utilizing Intel's NPU and ARC architecture without relying on NVIDIA resources. This approach simplifies the process for running large language models (LLMs) and vision language models (VLMs) by seamlessly detecting the best available Intel device, whether it's a NPU, ARC iGPU, or CPU, and exposing APIs that are compatible with OpenAI and Ollama clients. The installation process is straightforward, requiring only a script to set up and manage operating models. This development is significant for the AI/ML community as it democratizes access to advanced AI capabilities, enabling users with Intel-based systems to run high-performance models locally. The server supports streaming chat features, dual device capabilities for text and image processing, and offers a built-in web interface for ease of use. Moreover, it foresees a future upgrade that could unify the handling of image and text models into a single pipeline, simplifying the user experience further. This innovation represents a notable shift towards more accessible AI infrastructure, which could lead to broader experimentation and deployment of AI applications across various sectors.