Running DeepSeek R1 32B with NPU
This article introduces how to accelerate the inference of the DeepSeek large language model using OpenVINO on a laptop equipped with a Lunar Lake CPU and NPU. The author first points out that running the 32B model using only the CPU results in low performance and lacks practical value. Subsequently, the article details the complete workflow for invoking the NPU for inference via OpenVINO. The key steps include: first, downloading and installing the NPU driver, firmware, and Level Zero components from the official Intel website, and configuring user permissions to access the acceleration device; second, installing dependency libraries such as OpenVINO, ONNX, and Optimum in a Python virtual environment; and finally, verifying that OpenVINO successfully recognizes the NPU device, and then loading the 4-bit DeepSeek model quantized specifically for the OpenVINO format. Testing indicates that although NPU inference speed is significantly better than that of the CPU, the improvement in actual user experience is limited. Overall, this solution provides a feasible path for efficiently deploying large models on edge devices.