org.apache.sysml.runtime.compress

Class ColGroupValue

java.lang.Object

org.apache.sysml.runtime.compress.ColGroup

org.apache.sysml.runtime.compress.ColGroupValue

All Implemented Interfaces:

Serializable

Direct Known Subclasses:

ColGroupDDC, ColGroupOffset

public abstract class ColGroupValue
extends ColGroup

Base class for column groups encoded with value dictionary.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.sysml.runtime.compress.ColGroup

ColGroup.ColGroupRowIterator, ColGroup.CompressionType

Field Summary

Fields

Modifier and Type	Field and Description
protected double[]	_values Distinct values associated with individual bitmaps.
static boolean	LOW_LEVEL_OPT
static boolean	SORT_VALUES_BY_LENGTH

Fields inherited from class org.apache.sysml.runtime.compress.ColGroup

_colIndexes, _numRows

Constructor Summary

Constructors

Modifier	Constructor and Description
	ColGroupValue()
protected	ColGroupValue(int[] colIndices, int numRows, double[] values) Constructor for subclass methods that need to create shallow copies
	ColGroupValue(int[] colIndices, int numRows, UncompressedBitmap ubm) Stores the headers for the individual bitmaps.

Method Summary

Modifier and Type	Method and Description
protected static double[]	allocDVector(int len, boolean reset)
protected static int[]	allocIVector(int len, boolean reset)
protected double[]	applyScalarOp(org.apache.sysml.runtime.matrix.operators.ScalarOperator op) Method for use by subclasses.
protected double[]	applyScalarOp(org.apache.sysml.runtime.matrix.operators.ScalarOperator op, double newVal, int numCols)
static void	cleanupThreadLocalMemory()
protected void	computeColMxx(MatrixBlock result, org.apache.sysml.runtime.functionobjects.Builtin builtin, boolean zeros) NOTE: Shared across OLE/RLE/DDC because value-only computation.
protected void	computeMxx(MatrixBlock result, org.apache.sysml.runtime.functionobjects.Builtin builtin, boolean zeros) NOTE: Shared across OLE/RLE/DDC because value-only computation.
protected int	containsAllZeroValue()
long	estimateInMemorySize() Note: Must be overridden by child classes to account for additional data and metadata
int[]	getCounts()
int[]	getCounts(boolean inclZeros)
abstract int[]	getCounts(int[] out)
int[]	getCounts(int rl, int ru)
abstract int[]	getCounts(int rl, int ru, int[] out)
MatrixBlock	getCountsAsBlock()
static MatrixBlock	getCountsAsBlock(int[] counts)
int	getNumValues() Obtain number of distinct sets of values associated with the bitmaps in this column group.
double	getValue(int k, int col)
double[]	getValues()
MatrixBlock	getValuesAsBlock()
long	getValuesSize()
abstract void	leftMultByRowVector(ColGroupDDC vector, MatrixBlock result)
protected double[]	preaggValues(int numVals, double[] b)
protected double[]	preaggValues(int numVals, double[] b, boolean allocNew)
static void	setupThreadLocalMemory(int len)
void	setValues(double[] values)
protected double[]	sumAllValues(org.apache.sysml.runtime.functionobjects.KahanFunction kplus, org.apache.sysml.runtime.instructions.cp.KahanObject kbuff)
protected double[]	sumAllValues(org.apache.sysml.runtime.functionobjects.KahanFunction kplus, org.apache.sysml.runtime.instructions.cp.KahanObject kbuff, boolean allocNew)
double	sumValues(int valIx)
protected double	sumValues(int valIx, double[] b)
double	sumValues(int valIx, org.apache.sysml.runtime.functionobjects.KahanFunction kplus)
double	sumValues(int valIx, org.apache.sysml.runtime.functionobjects.KahanFunction kplus, org.apache.sysml.runtime.instructions.cp.KahanObject kbuff)
void	unaryAggregateOperations(org.apache.sysml.runtime.matrix.operators.AggregateUnaryOperator op, MatrixBlock result)
abstract void	unaryAggregateOperations(org.apache.sysml.runtime.matrix.operators.AggregateUnaryOperator op, MatrixBlock result, int rl, int ru)

Methods inherited from class org.apache.sysml.runtime.compress.ColGroup

countNonZerosPerRow, decompressToBlock, decompressToBlock, decompressToBlock, get, getColIndex, getColIndices, getCompType, getExactSizeOnDisk, getIterator, getNumCols, getNumRows, getRowIterator, leftMultByRowVector, readFields, readFields, rightMultByVector, scalarOperation, shiftColIndices, write, write

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOW_LEVEL_OPT

public static boolean LOW_LEVEL_OPT

SORT_VALUES_BY_LENGTH

public static final boolean SORT_VALUES_BY_LENGTH

See Also:

Constant Field Values

_values

protected double[] _values

Distinct values associated with individual bitmaps.

Constructor Detail

ColGroupValue

public ColGroupValue()

ColGroupValue

public ColGroupValue(int[] colIndices,
                     int numRows,
                     UncompressedBitmap ubm)

Stores the headers for the individual bitmaps.

Parameters:

colIndices - indices (within the block) of the columns included in this column

numRows - total number of rows in the parent block

ubm - Uncompressed bitmap representation of the block

ColGroupValue

protected ColGroupValue(int[] colIndices,
                        int numRows,
                        double[] values)

Constructor for subclass methods that need to create shallow copies

Parameters:

colIndices - raw column index information

numRows - number of rows in the block

values - set of distinct values for the block (associated bitmaps are kept in the subclass)

Method Detail

estimateInMemorySize

public long estimateInMemorySize()

Description copied from class: ColGroup

Note: Must be overridden by child classes to account for additional data and metadata

Overrides:

estimateInMemorySize in class ColGroup

Returns:

an upper bound on the number of bytes used to store this ColGroup in memory.

getValuesSize

public long getValuesSize()

getNumValues

public int getNumValues()

Obtain number of distinct sets of values associated with the bitmaps in this column group.

Returns:

the number of distinct sets of values associated with the bitmaps in this column group

getValues

public double[] getValues()

setValues

public void setValues(double[] values)

getValue

public double getValue(int k,
                       int col)

getValuesAsBlock

public MatrixBlock getValuesAsBlock()

getCounts

public final int[] getCounts()

getCounts

public abstract int[] getCounts(int[] out)

getCounts

public final int[] getCounts(int rl,
                             int ru)

getCounts

public abstract int[] getCounts(int rl,
                                int ru,
                                int[] out)

getCounts

public int[] getCounts(boolean inclZeros)

getCountsAsBlock

public MatrixBlock getCountsAsBlock()

getCountsAsBlock

public static MatrixBlock getCountsAsBlock(int[] counts)

containsAllZeroValue

protected int containsAllZeroValue()

sumValues

public final double sumValues(int valIx)

sumValues

public final double sumValues(int valIx,
                              org.apache.sysml.runtime.functionobjects.KahanFunction kplus)

sumValues

public final double sumValues(int valIx,
                              org.apache.sysml.runtime.functionobjects.KahanFunction kplus,
                              org.apache.sysml.runtime.instructions.cp.KahanObject kbuff)

sumAllValues

protected final double[] sumAllValues(org.apache.sysml.runtime.functionobjects.KahanFunction kplus,
                                      org.apache.sysml.runtime.instructions.cp.KahanObject kbuff)

sumAllValues

protected final double[] sumAllValues(org.apache.sysml.runtime.functionobjects.KahanFunction kplus,
                                      org.apache.sysml.runtime.instructions.cp.KahanObject kbuff,
                                      boolean allocNew)

sumValues

protected final double sumValues(int valIx,
                                 double[] b)

preaggValues

protected final double[] preaggValues(int numVals,
                                      double[] b)

preaggValues

protected final double[] preaggValues(int numVals,
                                      double[] b,
                                      boolean allocNew)

computeMxx

protected void computeMxx(MatrixBlock result,
                          org.apache.sysml.runtime.functionobjects.Builtin builtin,
                          boolean zeros)

NOTE: Shared across OLE/RLE/DDC because value-only computation.

Parameters:

result - output matrix block

builtin - function object

zeros - indicator if column group contains zero values

computeColMxx

protected void computeColMxx(MatrixBlock result,
                             org.apache.sysml.runtime.functionobjects.Builtin builtin,
                             boolean zeros)

NOTE: Shared across OLE/RLE/DDC because value-only computation.

Parameters:

result - output matrix block

builtin - function object

zeros - indicator if column group contains zero values

leftMultByRowVector

public abstract void leftMultByRowVector(ColGroupDDC vector,
                                         MatrixBlock result)

applyScalarOp

protected double[] applyScalarOp(org.apache.sysml.runtime.matrix.operators.ScalarOperator op)

Method for use by subclasses. Applies a scalar operation to the value metadata stored in the superclass.

Parameters:

op - scalar operation to perform

Returns:

transformed copy of value metadata for this column group

applyScalarOp

protected double[] applyScalarOp(org.apache.sysml.runtime.matrix.operators.ScalarOperator op,
                                 double newVal,
                                 int numCols)

unaryAggregateOperations

public void unaryAggregateOperations(org.apache.sysml.runtime.matrix.operators.AggregateUnaryOperator op,
                                     MatrixBlock result)

Specified by:

unaryAggregateOperations in class ColGroup

unaryAggregateOperations

public abstract void unaryAggregateOperations(org.apache.sysml.runtime.matrix.operators.AggregateUnaryOperator op,
                                              MatrixBlock result,
                                              int rl,
                                              int ru)

Parameters:

op - aggregation operator

result - output matrix block

rl - row lower index, inclusive

ru - row upper index, exclusive

setupThreadLocalMemory

public static void setupThreadLocalMemory(int len)

cleanupThreadLocalMemory

public static void cleanupThreadLocalMemory()

allocDVector

protected static double[] allocDVector(int len,
                                       boolean reset)

allocIVector

protected static int[] allocIVector(int len,
                                    boolean reset)