org.apache.sysds.runtime.matrix.data

Class LibMatrixReorg

java.lang.Object

org.apache.sysds.runtime.matrix.data.LibMatrixReorg

public class LibMatrixReorg
extends Object

MB: Library for selected matrix reorg operations including special cases and all combinations of dense and sparse representations. Current list of supported operations: - reshape, - r' (transpose), - rdiag (diagV2M/diagM2V), - rsort (sorting data/indexes) - rmempty (remove empty) - rexpand (outer/table-seq expansion)

Field Summary

Fields

Modifier and Type	Field and Description
static long	PAR_NUMCELL_THRESHOLD
static int	PAR_NUMCELL_THRESHOLD_SORT
static boolean	SHALLOW_COPY_REORG
static boolean	SPARSE_OUTPUTS_IN_CSR

Method Summary

Modifier and Type	Method and Description
static MatrixBlock	diag(MatrixBlock in, MatrixBlock out)
static boolean	isSupportedReorgOperator(ReorgOperator op)
static MatrixBlock	reorg(MatrixBlock in, MatrixBlock out, ReorgOperator op)
static List<IndexedMatrixValue>	reshape(IndexedMatrixValue in, DataCharacteristics mcIn, DataCharacteristics mcOut, boolean rowwise, boolean outputEmptyBlocks) MR/SPARK reshape interface - for reshape we cannot view blocks independently, and hence, there are different CP and MR interfaces.
static MatrixBlock	reshape(MatrixBlock in, MatrixBlock out, int rows, int cols, boolean rowwise) CP reshape operation (single input, single output matrix) NOTE: In contrast to R, the rowwise parameter specifies both the read and write order, with row-wise being the default, while R uses always a column-wise read, rowwise specifying the write order and column-wise being the default.
static void	rev(IndexedMatrixValue in, long rlen, int blen, ArrayList<IndexedMatrixValue> out)
static MatrixBlock	rev(MatrixBlock in, MatrixBlock out)
static void	rexpand(IndexedMatrixValue data, double max, boolean rows, boolean cast, boolean ignore, long blen, ArrayList<IndexedMatrixValue> outList) MR/Spark rexpand operation (single input, multiple outputs incl empty blocks)
static MatrixBlock	rexpand(MatrixBlock in, MatrixBlock ret, double max, boolean rows, boolean cast, boolean ignore, int k) CP rexpand operation (single input, single output)
static void	rmempty(IndexedMatrixValue data, IndexedMatrixValue offset, boolean rmRows, long len, long blen, ArrayList<IndexedMatrixValue> outList) MR rmempty interface - for rmempty we cannot view blocks independently, and hence, there are different CP and MR interfaces.
static MatrixBlock	rmempty(MatrixBlock in, MatrixBlock ret, boolean rows, boolean emptyReturn, MatrixBlock select) CP rmempty operation (single input, single output matrix)
static MatrixBlock	sort(MatrixBlock in, MatrixBlock out, int[] by, boolean desc, boolean ixret)
static MatrixBlock	sort(MatrixBlock in, MatrixBlock out, int[] by, boolean desc, boolean ixret, int k)
static MatrixBlock	transpose(MatrixBlock in, MatrixBlock out)
static MatrixBlock	transpose(MatrixBlock in, MatrixBlock out, int k)

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PAR_NUMCELL_THRESHOLD

public static final long PAR_NUMCELL_THRESHOLD

See Also:

Constant Field Values

PAR_NUMCELL_THRESHOLD_SORT

public static final int PAR_NUMCELL_THRESHOLD_SORT

See Also:

Constant Field Values

SHALLOW_COPY_REORG

public static final boolean SHALLOW_COPY_REORG

See Also:

Constant Field Values

SPARSE_OUTPUTS_IN_CSR

public static final boolean SPARSE_OUTPUTS_IN_CSR

See Also:

Constant Field Values

Method Detail

isSupportedReorgOperator

public static boolean isSupportedReorgOperator(ReorgOperator op)

reorg

public static MatrixBlock reorg(MatrixBlock in,
                                MatrixBlock out,
                                ReorgOperator op)

transpose

public static MatrixBlock transpose(MatrixBlock in,
                                    MatrixBlock out)

transpose

public static MatrixBlock transpose(MatrixBlock in,
                                    MatrixBlock out,
                                    int k)

rev

public static MatrixBlock rev(MatrixBlock in,
                              MatrixBlock out)

rev

public static void rev(IndexedMatrixValue in,
                       long rlen,
                       int blen,
                       ArrayList<IndexedMatrixValue> out)

diag

public static MatrixBlock diag(MatrixBlock in,
                               MatrixBlock out)

sort

public static MatrixBlock sort(MatrixBlock in,
                               MatrixBlock out,
                               int[] by,
                               boolean desc,
                               boolean ixret)

sort

public static MatrixBlock sort(MatrixBlock in,
                               MatrixBlock out,
                               int[] by,
                               boolean desc,
                               boolean ixret,
                               int k)

Parameters:

in - Input matrix to sort

out - Output matrix where the sorted input is inserted to

by - The Ordering parameter

desc - A boolean, specifying if it should be descending order.

ixret - A boolean, specifying if the return should be the sorted indexes.

k - Number of parallel threads

Returns:

The sorted out matrix.

reshape

public static MatrixBlock reshape(MatrixBlock in,
                                  MatrixBlock out,
                                  int rows,
                                  int cols,
                                  boolean rowwise)

CP reshape operation (single input, single output matrix) NOTE: In contrast to R, the rowwise parameter specifies both the read and write order, with row-wise being the default, while R uses always a column-wise read, rowwise specifying the write order and column-wise being the default.

Parameters:

in - input matrix

out - output matrix

rows - number of rows

cols - number of columns

rowwise - if true, reshape by row

Returns:

output matrix

reshape

public static List<IndexedMatrixValue> reshape(IndexedMatrixValue in,
                                               DataCharacteristics mcIn,
                                               DataCharacteristics mcOut,
                                               boolean rowwise,
                                               boolean outputEmptyBlocks)

MR/SPARK reshape interface - for reshape we cannot view blocks independently, and hence, there are different CP and MR interfaces.

Parameters:

in - indexed matrix value

mcIn - input matrix characteristics

mcOut - output matrix characteristics

rowwise - if true, reshape by row

outputEmptyBlocks - output blocks with nnz=0

Returns:

list of indexed matrix values

rmempty

public static MatrixBlock rmempty(MatrixBlock in,
                                  MatrixBlock ret,
                                  boolean rows,
                                  boolean emptyReturn,
                                  MatrixBlock select)

CP rmempty operation (single input, single output matrix)

Parameters:

in - input matrix

ret - output matrix

rows - ?

emptyReturn - return row/column of zeros for empty input

select - ?

Returns:

matrix block

rmempty

public static void rmempty(IndexedMatrixValue data,
                           IndexedMatrixValue offset,
                           boolean rmRows,
                           long len,
                           long blen,
                           ArrayList<IndexedMatrixValue> outList)

MR rmempty interface - for rmempty we cannot view blocks independently, and hence, there are different CP and MR interfaces.

Parameters:

data - ?

offset - ?

rmRows - ?

len - ?

blen - block length

outList - list of indexed matrix values

rexpand

public static MatrixBlock rexpand(MatrixBlock in,
                                  MatrixBlock ret,
                                  double max,
                                  boolean rows,
                                  boolean cast,
                                  boolean ignore,
                                  int k)

CP rexpand operation (single input, single output)

Parameters:

in - input matrix

ret - output matrix

max - ?

rows - ?

cast - ?

ignore - ?

k - degree of parallelism

Returns:

output matrix

rexpand

public static void rexpand(IndexedMatrixValue data,
                           double max,
                           boolean rows,
                           boolean cast,
                           boolean ignore,
                           long blen,
                           ArrayList<IndexedMatrixValue> outList)

MR/Spark rexpand operation (single input, multiple outputs incl empty blocks)

Parameters:

data - indexed matrix value

max - ?

rows - ?

cast - ?

ignore - ?

blen - block length

outList - list of indexed matrix values