Welcome to Intel® NPU Acceleration Library’s documentation! The Intel® NPU Acceleration Library is a Python library designed to boost the efficiency of your applications by leveraging the power of the Intel Neural Processing Unit (NPU) to perform high-speed computations on compatible hardware
intel_npu_acceleration_library package Submodules # intel_npu_acceleration_library bindings module # intel_npu_acceleration_library compiler module # class intel_npu_acceleration_library compiler CompilerConfig(use_to: bool = False, dtype: dtype | NPUDtype = torch float16, training: bool = False) # Bases: object Configuration class to store the compilation configuration of a model for the NPU intel_npu_acceleration_library
Basic usage — Intel® NPU Acceleration Library documentation Basic usage # For implemented examples, please check the examples folder Run a single MatMul in the NPU # from intel_npu_acceleration_library backend import MatMul import numpy as np inC, outC, batch =
Quick overview of Intel’s Neural Processing Unit (NPU) Quick overview of Intel’s Neural Processing Unit (NPU) # The Intel NPU is an AI accelerator integrated into Intel Core Ultra processors, characterized by a unique architecture comprising compute acceleration and data transfer capabilities
Decoding LLM performance — Intel® NPU Acceleration Library documentation Static shapes allows the NN graph compiler to improve memory management, schedule and overall network performance For a example implementation, you can refer to the intel_npu_acceleration_library nn llm generate_with_static_shape or transformers library StaticCache Conclusions #
intel_npu_acceleration_library. nn package Generate a NPU LlamaAttention layer from a transformer LlamaAttention one Parameters: layer (torch nn Linear) – the original LlamaAttention model to run on the NPU dtype (torch dtype) – the desired datatype Returns: A NPU LlamaAttention layer Return type: LlamaAttention class intel_npu_acceleration_library nn Module(profile: bool = False) #
Advanced Setup — Intel® NPU Acceleration Library documentation To build the package you need a compiler in your system (Visual Studio 2019 suggested for Windows build) MacOS is not yet supported For development packages use (after cloning the repo)
intel_npu_acceleration_library. backend package Returns: Return True if the NPU is available in the system Return type: bool intel_npu_acceleration_library backend run_factory(x: Tensor | List[Tensor], weights: List[Tensor], backend_cls: Any, op_id: str | None = None) → Tensor # Run a factory operation Depending on the datatype of the weights it runs a float or quantized operation