org.apache.sysds.runtime.controlprogram.parfor

Class DataPartitioner

java.lang.Object

org.apache.sysds.runtime.controlprogram.parfor.DataPartitioner

Direct Known Subclasses:

DataPartitionerLocal, DataPartitionerRemoteSpark

public abstract class DataPartitioner
extends Object

This is the base class for all data partitioner.

Method Summary

Modifier and Type	Method and Description
MatrixObject	createPartitionedMatrixObject(MatrixObject in, MatrixObject out, boolean force) Creates a partitioned matrix object based on the given input matrix object, according to the specified split format.
MatrixObject	createPartitionedMatrixObject(MatrixObject in, String fnameNew)
MatrixObject	createPartitionedMatrixObject(MatrixObject in, String fnameNew, boolean force)
static MatrixBlock	createReuseMatrixBlock(ParForProgramBlock.PDataPartitionFormat dpf, int rows, int cols)
void	disableBinaryCell()

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

createPartitionedMatrixObject

public MatrixObject createPartitionedMatrixObject(MatrixObject in,
                                                  String fnameNew)

createPartitionedMatrixObject

public MatrixObject createPartitionedMatrixObject(MatrixObject in,
                                                  String fnameNew,
                                                  boolean force)

createPartitionedMatrixObject

public MatrixObject createPartitionedMatrixObject(MatrixObject in,
                                                  MatrixObject out,
                                                  boolean force)

Creates a partitioned matrix object based on the given input matrix object, according to the specified split format. The input matrix can be in-memory or still on HDFS and the partitioned output matrix is written to HDFS. The created matrix object can be used transparently for obtaining the full matrix or reading 1 or multiple partitions based on given index ranges.

Parameters:

in - input matrix object

out - output matrix object

force - if false, try to optimize

Returns:

partitioned matrix object

disableBinaryCell

public void disableBinaryCell()

createReuseMatrixBlock

public static MatrixBlock createReuseMatrixBlock(ParForProgramBlock.PDataPartitionFormat dpf,
                                                 int rows,
                                                 int cols)