SystemML Documentation

SystemML is a flexible, scalable machine learning system. SystemML’s distinguishing characteristics are:

1. Algorithm customizability via R-like and Python-like languages.
2. Multiple execution modes, including Spark MLContext, Spark Batch, Hadoop Batch, Standalone, and JMLC.
3. Automatic optimization based on data and cluster characteristics to ensure both efficiency and scalability.

The SystemML GitHub README describes building, testing, and running SystemML. Please read Contributing to SystemML to find out how to help make SystemML even better!

To download SystemML, visit the downloads page.

This version of SystemML supports: Java 8+, Scala 2.11+, Python 2.7/3.5+, Hadoop 2.6+, and Spark 2.1+.

Running SystemML

• Beginner’s Guide For Python Users - Beginner’s Guide for Python users.
• Spark MLContext - Spark MLContext is a programmatic API for running SystemML from Spark via Scala, Python, or Java.
  • Spark Shell Example (Scala)
  • Jupyter Notebook Example (PySpark)
• Spark Batch - Algorithms are automatically optimized to run across Spark clusters.
• Hadoop Batch - Algorithms are automatically optimized when distributed across Hadoop clusters.
• Standalone - Standalone mode allows data scientists to rapidly prototype algorithms on a single machine in R-like and Python-like declarative languages.
• JMLC - Java Machine Learning Connector.
• Experimental Caffe2DML API for Deep Learning.

Language Guides

• Python API Reference - API Reference Guide for Python users.
• DML Language Reference - DML is a high-level R-like declarative language for machine learning.
• PyDML Language Reference - PyDML is a high-level Python-like declarative language for machine learning.
• Beginner’s Guide to DML and PyDML - An introduction to the basics of DML and PyDML.

ML Algorithms

• Algorithms Reference - The Algorithms Reference describes the machine learning algorithms included with SystemML in detail.

Tools

• Debugger Guide - SystemML supports DML script-level debugging through a command-line interface.
• IDE Guide - Useful IDE Guide for Developing SystemML.

Other

• Contributing to SystemML - Describes ways to contribute to SystemML.
• Engine Developer Guide - Guide for internal SystemML engine development.
• Troubleshooting Guide - Troubleshoot various issues related to SystemML.
• Release Process - Description of the SystemML release process.