org.apache.sysds.runtime.compress.colgroup

Class ADictionary

java.lang.Object

org.apache.sysds.runtime.compress.colgroup.ADictionary

Direct Known Subclasses:

Dictionary, QDictionary

public abstract class ADictionary
extends Object

This dictionary class aims to encapsulate the storage and operations over unique floating point values of a column group.

Constructor Summary

Constructors

Constructor and Description
ADictionary()

Method Summary

Modifier and Type	Method and Description
abstract double	aggregate(double init, Builtin fn) Aggregate all the contained values, useful in value only computations where the operation is iterating through all values contained in the dictionary.
void	aggregateCols(double[] c, Builtin fn, int[] colIndexes) Aggregates the columns into the target double array provided.
abstract ADictionary	apply(ScalarOperator op) Applies the scalar operation on the dictionary.
abstract ADictionary	applyScalarOp(ScalarOperator op, double newVal, int numCols) Applies the scalar operation on the dictionary.
abstract ADictionary	clone() Returns a deep clone of the dictionary.
abstract long	getExactSizeOnDisk() Calculate the space consumption if the dictionary is stored on disk.
abstract long	getInMemorySize() Returns the memory usage of the dictionary.
abstract int	getNumberOfValues(int ncol) Get the number of values given that the column group has n columns
abstract double	getValue(int i) Get Specific value contained in the dictionary at index.
abstract double[]	getValues() Get all the values contained in the dictionary as a linearized double array.
abstract int	getValuesLength() returns the count of values contained in the dictionary.
abstract int	hasZeroTuple(int ncol) Determines if the content has a zero tuple.
static ADictionary	read(DataInput in, boolean lossy) The read function to instantiate the dictionary.
abstract void	write(DataOutput out) Write the dictionary to a DataOutput.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ADictionary

public ADictionary()

Method Detail

getValues

public abstract double[] getValues()

Get all the values contained in the dictionary as a linearized double array.

Returns:

linearized double array

getValue

public abstract double getValue(int i)

Get Specific value contained in the dictionary at index.

Parameters:

i - The index to extract the value from

Returns:

The value contained at the index

hasZeroTuple

public abstract int hasZeroTuple(int ncol)

Determines if the content has a zero tuple. meaning all values at a specific row are zero value. This is useful information to find out if the dictionary is used in a dense context. To improve some specific operations.

Parameters:

ncol - The number of columns in the dictionary.

Returns:

The index at which the zero tuple is located.

getInMemorySize

public abstract long getInMemorySize()

Returns the memory usage of the dictionary.

Returns:

a long value in number of bytes for the dictionary.

aggregate

public abstract double aggregate(double init,
                                 Builtin fn)

Aggregate all the contained values, useful in value only computations where the operation is iterating through all values contained in the dictionary.

Parameters:

init - The initial Value, in cases such as Max value, this could be -infinity

fn - The Function to apply to values

Returns:

The aggregated value as a double.

getValuesLength

public abstract int getValuesLength()

returns the count of values contained in the dictionary.

Returns:

an integer of count of values.

apply

public abstract ADictionary apply(ScalarOperator op)

Applies the scalar operation on the dictionary. Note that this operation modifies the underlying data, and normally require a copy of the original Dictionary to preserve old objects.

Parameters:

op - The operator to apply to the dictionary values.

Returns:

this dictionary with modified values.

applyScalarOp

public abstract ADictionary applyScalarOp(ScalarOperator op,
                                          double newVal,
                                          int numCols)

Applies the scalar operation on the dictionary. The returned dictionary should contain a new instance of the underlying data. Therefore it will not modify the previous object.

Parameters:

op - The operator to apply to the dictionary values.

newVal - The value to append to the dictionary.

numCols - The number of columns stored in the dictionary.

Returns:

Another dictionary with modified values.

clone

public abstract ADictionary clone()

Returns a deep clone of the dictionary.

Overrides:

clone in class Object

aggregateCols

public void aggregateCols(double[] c,
                          Builtin fn,
                          int[] colIndexes)

Aggregates the columns into the target double array provided.

Parameters:

c - The target double array, this contains the full number of columns, therefore the colIndexes for this specific dictionary is needed.

fn - The function to apply to individual columns

colIndexes - The mapping to the target columns from the individual columns

read

public static ADictionary read(DataInput in,
                               boolean lossy)
                        throws IOException

The read function to instantiate the dictionary.

Parameters:

in - The data input source to read the stored dictionary from

lossy - Boolean specifying if the dictionary stored was lossy.

Returns:

The concrete dictionary.

Throws:

IOException - if the reading source throws it.

write

public abstract void write(DataOutput out)
                    throws IOException

Write the dictionary to a DataOutput.

Parameters:

out - the output sink to write the dictionary to.

Throws:

IOException - if the sink fails.

getExactSizeOnDisk

public abstract long getExactSizeOnDisk()

Calculate the space consumption if the dictionary is stored on disk.

Returns:

the long count of bytes to store the dictionary.

getNumberOfValues

public abstract int getNumberOfValues(int ncol)

Get the number of values given that the column group has n columns

Parameters:

ncol - The number of Columns in the ColumnGroup.

Returns:

the number of value tuples contained in the dictionary.