org.apache.sysml.runtime.compress

Class ColGroup

java.lang.Object

org.apache.sysml.runtime.compress.ColGroup

All Implemented Interfaces:

Serializable

Direct Known Subclasses:

ColGroupUncompressed, ColGroupValue

public abstract class ColGroup
extends Object
implements Serializable

Class that stores information about a column group within a compressed matrix block. There are subclasses specific to each compression type.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected class	ColGroup.ColGroupRowIterator Base class for column group row iterators.
static class	ColGroup.CompressionType

Field Summary

Fields

Modifier and Type	Field and Description
protected int[]	_colIndexes Offsets of the columns that make up the column group.
protected int	_numRows Number of rows in the matrix, for use by child classes.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	ColGroup(int[] colIndices, int numRows) Main constructor.
protected	ColGroup(List<Integer> colIndicesList, int numRows) Convenience constructor for converting indices to a more compact format.

Method Summary

Modifier and Type	Method and Description
protected abstract void	countNonZerosPerRow(int[] rnnz, int rl, int ru) Count the number of non-zeros per row
abstract void	decompressToBlock(MatrixBlock target, int colpos) Decompress to block.
abstract void	decompressToBlock(MatrixBlock target, int[] colIndexTargets) Decompress the contents of this column group into uncompressed packed columns
abstract void	decompressToBlock(MatrixBlock target, int rl, int ru) Decompress the contents of this column group into the specified full matrix block.
long	estimateInMemorySize() Note: Must be overridden by child classes to account for additional data and metadata
abstract double	get(int r, int c) Get the value at a global row/column position.
int	getColIndex(int colNum) Obtain a column index value.
int[]	getColIndices() Obtain the offsets of the columns in the matrix block that make up the group
abstract ColGroup.CompressionType	getCompType() Obtain the compression type.
abstract long	getExactSizeOnDisk() Returns the exact serialized size of column group.
abstract Iterator<IJV>	getIterator(int rl, int ru, boolean inclZeros, boolean rowMajor) Create a column group iterator for a row index range.
int	getNumCols() Obtain the number of columns in this column group.
int	getNumRows()
abstract ColGroup.ColGroupRowIterator	getRowIterator(int rl, int ru) Create a dense row iterator for a row index range.
abstract void	leftMultByRowVector(MatrixBlock vector, MatrixBlock result) Multiply the slice of the matrix that this column group represents by a row vector on the left (the original column vector is assumed to be transposed already i.e.
abstract void	readFields(DataInput in) Deserializes column group from data input.
void	readFields(DataInput in, boolean skipDict) Deserializes column group from data input.
abstract void	rightMultByVector(MatrixBlock vector, MatrixBlock result, int rl, int ru) Multiply the slice of the matrix that this column group represents by a vector on the right.
abstract ColGroup	scalarOperation(org.apache.sysml.runtime.matrix.operators.ScalarOperator op) Perform the specified scalar operation directly on the compressed column group, without decompressing individual cells if possible.
void	shiftColIndices(int offset)
abstract void	unaryAggregateOperations(org.apache.sysml.runtime.matrix.operators.AggregateUnaryOperator op, MatrixBlock result)
abstract void	write(DataOutput out) Serializes column group to data output.
void	write(DataOutput out, boolean skipDict) Serializes column group to data output.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

_colIndexes

protected int[] _colIndexes

Offsets of the columns that make up the column group. Zero-based, and relative to the matrix block.

_numRows

protected int _numRows

Number of rows in the matrix, for use by child classes.

Constructor Detail

ColGroup

protected ColGroup(int[] colIndices,
                   int numRows)

Main constructor.

Parameters:

colIndices - offsets of the columns in the matrix block that make up the group

numRows - total number of rows in the parent block

ColGroup

protected ColGroup(List<Integer> colIndicesList,
                   int numRows)

Convenience constructor for converting indices to a more compact format.

Parameters:

colIndicesList - list of column indices

numRows - total number of rows in the parent block

Method Detail

getColIndices

public int[] getColIndices()

Obtain the offsets of the columns in the matrix block that make up the group

Returns:

offsets of the columns in the matrix block that make up the group

getColIndex

public int getColIndex(int colNum)

Obtain a column index value.

Parameters:

colNum - column number

Returns:

column index value

getNumRows

public int getNumRows()

getNumCols

public int getNumCols()

Obtain the number of columns in this column group.

Returns:

number of columns in this column group

getCompType

public abstract ColGroup.CompressionType getCompType()

Obtain the compression type.

Returns:

How the elements of the column group are compressed.

shiftColIndices

public void shiftColIndices(int offset)

estimateInMemorySize

public long estimateInMemorySize()

Note: Must be overridden by child classes to account for additional data and metadata

Returns:

an upper bound on the number of bytes used to store this ColGroup in memory.

decompressToBlock

public abstract void decompressToBlock(MatrixBlock target,
                                       int rl,
                                       int ru)

Decompress the contents of this column group into the specified full matrix block.

Parameters:

target - a matrix block where the columns covered by this column group have not yet been filled in.

rl - row lower

ru - row upper

decompressToBlock

public abstract void decompressToBlock(MatrixBlock target,
                                       int[] colIndexTargets)

Decompress the contents of this column group into uncompressed packed columns

Parameters:

target - a dense matrix block. The block must have enough space to hold the contents of this column group.

colIndexTargets - array that maps column indices in the original matrix block to columns of target.

decompressToBlock

public abstract void decompressToBlock(MatrixBlock target,
                                       int colpos)

Decompress to block.

Parameters:

target - dense output vector

colpos - column to decompress, error if larger or equal numCols

write

public abstract void write(DataOutput out)
                    throws IOException

Serializes column group to data output.

Parameters:

out - data output

Throws:

IOException - if IOException occurs

write

public void write(DataOutput out,
                  boolean skipDict)
           throws IOException

Serializes column group to data output.

Parameters:

out - data output

skipDict - skip shared dictionary

Throws:

IOException - if IOException occurs

readFields

public abstract void readFields(DataInput in)
                         throws IOException

Deserializes column group from data input.

Parameters:

in - data input

Throws:

IOException - if IOException occurs

readFields

public void readFields(DataInput in,
                       boolean skipDict)
                throws IOException

Deserializes column group from data input.

Parameters:

in - data input

skipDict - skip shared dictionary

Throws:

IOException - if IOException occurs

getExactSizeOnDisk

public abstract long getExactSizeOnDisk()

Returns the exact serialized size of column group. This can be used for example for buffer preallocation.

Returns:

exact serialized size for column group

get

public abstract double get(int r,
                           int c)

Get the value at a global row/column position.

Parameters:

r - row

c - column

Returns:

value at the row/column position

rightMultByVector

public abstract void rightMultByVector(MatrixBlock vector,
                                       MatrixBlock result,
                                       int rl,
                                       int ru)

Multiply the slice of the matrix that this column group represents by a vector on the right.

Parameters:

vector - vector to multiply by (tall vector)

result - accumulator for holding the result

rl - row lower

ru - row upper if the internal SystemML code that performs the multiplication experiences an error

leftMultByRowVector

public abstract void leftMultByRowVector(MatrixBlock vector,
                                         MatrixBlock result)

Multiply the slice of the matrix that this column group represents by a row vector on the left (the original column vector is assumed to be transposed already i.e. its size now is 1xn).

Parameters:

vector - row vector

result - matrix block result

scalarOperation

public abstract ColGroup scalarOperation(org.apache.sysml.runtime.matrix.operators.ScalarOperator op)

Perform the specified scalar operation directly on the compressed column group, without decompressing individual cells if possible.

Parameters:

op - operation to perform

Returns:

version of this column group with the operation applied

unaryAggregateOperations

public abstract void unaryAggregateOperations(org.apache.sysml.runtime.matrix.operators.AggregateUnaryOperator op,
                                              MatrixBlock result)

getIterator

public abstract Iterator<IJV> getIterator(int rl,
                                          int ru,
                                          boolean inclZeros,
                                          boolean rowMajor)

Create a column group iterator for a row index range.

Parameters:

rl - row lower index, inclusive

ru - row upper index, exclusive

inclZeros - include zero values into scope of iterator

rowMajor - use a row major iteration order

Returns:

an iterator instance

getRowIterator

public abstract ColGroup.ColGroupRowIterator getRowIterator(int rl,
                                                            int ru)

Create a dense row iterator for a row index range. This iterator implies the inclusion of zeros and row-major iteration order.

Parameters:

rl - row lower index, inclusive

ru - row upper index, exclusive

Returns:

an iterator instance

countNonZerosPerRow

protected abstract void countNonZerosPerRow(int[] rnnz,
                                            int rl,
                                            int ru)

Count the number of non-zeros per row

Parameters:

rnnz - non-zeros per row

rl - row lower bound, inclusive

ru - row upper bound, exclusive