org.apache.sysds.runtime.compress.colgroup

Class ColGroupValue

java.lang.Object

org.apache.sysds.runtime.compress.colgroup.ColGroup

org.apache.sysds.runtime.compress.colgroup.ColGroupValue

All Implemented Interfaces:

Serializable

Direct Known Subclasses:

ColGroupDDC, ColGroupOffset

public abstract class ColGroupValue
extends ColGroup

Base class for column groups encoded with value dictionary. This include column groups such as DDC OLE and RLE.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.sysds.runtime.compress.colgroup.ColGroup

ColGroup.CompressionType

Method Summary

Modifier and Type	Method and Description
static void	cleanupThreadLocalMemory()
byte[]	getByteValues()
int[]	getCounts() Returns the counts of values inside the MatrixBlock returned in getValuesAsBlock Throws an exception if the getIfCountsType is false.
abstract int[]	getCounts(int[] out)
int[]	getCounts(int rl, int ru) Returns the counts of values inside the MatrixBlock returned in getValuesAsBlock Throws an exception if the getIfCountsType is false.
abstract int[]	getCounts(int rl, int ru, int[] out)
long	getExactSizeOnDisk() Returns the exact serialized size of column group.
boolean	getIfCountsType() Returns true if in the getValuesAsBlock method returns values in groups (that needs to be counted) or individually potentially repeated values
int	getNumValues() Obtain number of distinct sets of values associated with the bitmaps in this column group.
double[]	getValues() Get all the values in the colGroup.
MatrixBlock	getValuesAsBlock() Returns the ColGroup as a MatrixBlock.
boolean	isLossy() Is Lossy
void	readFields(DataInput in) Deserialize column group from data input.
static void	setupThreadLocalMemory(int len)
String	toString()
void	unaryAggregateOperations(AggregateUnaryOperator op, double[] c) Unary Aggregate operator, since aggregate operators require new object output, the output becomes an uncompressed matrix.
void	unaryAggregateOperations(AggregateUnaryOperator op, double[] c, int rl, int ru) Unary Aggregate operator, since aggregate operators require new object output, the output becomes an uncompressed matrix.
void	write(DataOutput out) Serializes column group to data output.

Methods inherited from class org.apache.sysds.runtime.compress.colgroup.ColGroup

countNonZerosPerRow, decompressToBlock, decompressToBlock, decompressToBlock, estimateInMemorySize, get, getColIndex, getColIndices, getCompType, getIterator, getNumCols, getNumRows, getRowIterator, leftMultByMatrix, leftMultByRowVector, leftMultByRowVector, rightMultByMatrix, rightMultByVector, scalarOperation, shiftColIndices

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

getNumValues

public int getNumValues()

Obtain number of distinct sets of values associated with the bitmaps in this column group.

Returns:

the number of distinct sets of values associated with the bitmaps in this column group

getValues

public double[] getValues()

Description copied from class: ColGroup

Get all the values in the colGroup. Note that this is only the stored values not the way they are stored. Making the output a list of values used in that colGroup not the actual full column.

Specified by:

getValues in class ColGroup

Returns:

a double list of values.

getByteValues

public byte[] getByteValues()

getValuesAsBlock

public MatrixBlock getValuesAsBlock()

Description copied from class: ColGroup

Returns the ColGroup as a MatrixBlock. Used as a fall back solution in case a operation is not supported. Use in connection to getIfCountsType to get if the values are repeated.

Specified by:

getValuesAsBlock in class ColGroup

Returns:

Matrix Block of the contained Values. Possibly contained in groups.

getCounts

public final int[] getCounts()

Returns the counts of values inside the MatrixBlock returned in getValuesAsBlock Throws an exception if the getIfCountsType is false. The returned counts always contains the number of zeros as well if there are some contained, even if they are not materialized.

Returns:

the count of each value in the MatrixBlock.

getCounts

public final int[] getCounts(int rl,
                             int ru)

Returns the counts of values inside the MatrixBlock returned in getValuesAsBlock Throws an exception if the getIfCountsType is false. The returned counts always contains the number of zeros as well if there are some contained, even if they are not materialized.

Parameters:

rl - the lower index of the interval of rows queried

ru - the the upper boundary of the interval of rows queried

Returns:

the count of each value in the MatrixBlock.

getIfCountsType

public boolean getIfCountsType()

Description copied from class: ColGroup

Returns true if in the getValuesAsBlock method returns values in groups (that needs to be counted) or individually potentially repeated values

Specified by:

getIfCountsType in class ColGroup

Returns:

boolean

unaryAggregateOperations

public void unaryAggregateOperations(AggregateUnaryOperator op,
                                     double[] c)

Description copied from class: ColGroup

Unary Aggregate operator, since aggregate operators require new object output, the output becomes an uncompressed matrix.

Specified by:

unaryAggregateOperations in class ColGroup

Parameters:

op - The operator used

c - Rhe output matrix block.

unaryAggregateOperations

public void unaryAggregateOperations(AggregateUnaryOperator op,
                                     double[] c,
                                     int rl,
                                     int ru)

Description copied from class: ColGroup

Unary Aggregate operator, since aggregate operators require new object output, the output becomes an uncompressed matrix.

Specified by:

unaryAggregateOperations in class ColGroup

Parameters:

op - The operator used

c - The output matrix block.

rl - The Starting Row to do aggregation from

ru - The last Row to do aggregation to (not included)

setupThreadLocalMemory

public static void setupThreadLocalMemory(int len)

cleanupThreadLocalMemory

public static void cleanupThreadLocalMemory()

toString

public String toString()

Overrides:

toString in class Object

isLossy

public boolean isLossy()

Description copied from class: ColGroup

Is Lossy

Specified by:

isLossy in class ColGroup

Returns:

returns if the ColGroup is compressed in a lossy manner.

readFields

public void readFields(DataInput in)
                throws IOException

Description copied from class: ColGroup

Deserialize column group from data input.

Specified by:

readFields in class ColGroup

Parameters:

in - data input

Throws:

IOException - if IOException occurs

write

public void write(DataOutput out)
           throws IOException

Description copied from class: ColGroup

Serializes column group to data output.

Specified by:

write in class ColGroup

Parameters:

out - data output

Throws:

IOException - if IOException occurs

getExactSizeOnDisk

public long getExactSizeOnDisk()

Description copied from class: ColGroup

Returns the exact serialized size of column group. This can be used for example for buffer preallocation.

Specified by:

getExactSizeOnDisk in class ColGroup

Returns:

exact serialized size for column group

getCounts

public abstract int[] getCounts(int[] out)

getCounts

public abstract int[] getCounts(int rl,
                                int ru,
                                int[] out)