Native Backend guide for SystemDS
This guide covers the Native BLAS optimizations and software setup for using SystemDS
- Native BLAS setup
Native BLAS mode
SystemDS implements all the matrix operations in Java. This simplifies deployment especially in a distributed environment.
In Some cases (such as Deep Neural Networks), to take advantage of native BLAS instead of SystemDS internal Java library for performing single node operations such as matrix multiplication, convolution etc.
By default, SystemDS will first attempt to use Intel MKL (if installed), and then OpenBLAS (if installed). If none of the libraries are available, SystemDS falls back to its internal java library.
Note: Current SystemDS version supported on Linux, Windows x86-64 platform.
To know Intel MKL system requirements, see Intel® oneAPI Math Kernel Library System Requirements
Either of the following software is required to be installed in your system:
Download Intel MKL.
Supported MKL version is
Install OpenBLAS. Installation instructions on GitHub.
Note: In Ubuntu 20.04, remove
libopenblas0-pthread package and install
instead. So that OpenBLAS will be installed with OpenMP support.
Native BLAS setup
Add library to the System path
Scripts to set environmental variables
For example, in Linux to set Intel MKL libraries in the Path:
# path adjustments to that script according to your installation source /opt/intel/bin/compilervars.sh intel64
Note: This property can also be set with
Enable Native BLAS in SystemDS
sysds.native.blas property to
openblas as shown.
<!-- enables native blas for matrix multiplication and convolution, experimental feature (options: auto, mkl, openblas, none) --> <sysds.native.blas>mkl</sysds.native.blas>
If there are issues loading libs because the gcc ABI changed from gcc 4.x to 5 and above. In this case,
- fiddle with the patchelf utility or
- Compile a recent gcc and adjust your env vars (