org.apache.sysds.runtime.transform.encode

Class ColumnEncoder

java.lang.Object

org.apache.sysds.runtime.transform.encode.ColumnEncoder

All Implemented Interfaces:

Externalizable, Serializable, Comparable<ColumnEncoder>, Encoder

Direct Known Subclasses:

ColumnEncoderBin, ColumnEncoderComposite, ColumnEncoderDummycode, ColumnEncoderFeatureHash, ColumnEncoderPassThrough, ColumnEncoderRecode

public abstract class ColumnEncoder
extends Object
implements Externalizable, Encoder, Comparable<ColumnEncoder>

Base class for all transform encoders providing both a row and block interface for decoding frames to matrices.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ColumnEncoder.EncoderType

Method Summary

Modifier and Type	Method and Description
abstract MatrixBlock	apply(FrameBlock in, MatrixBlock out, int outputCol, int rowStart, int blk)
abstract MatrixBlock	apply(MatrixBlock in, MatrixBlock out, int outputCol)
abstract MatrixBlock	apply(MatrixBlock in, MatrixBlock out, int outputCol, int rowStart, int blk)
void	buildPartial(FrameBlock in) Partial build of internal data structures (e.g., in distributed spark operations).
int	compareTo(ColumnEncoder o)
int	getColID()
MatrixBlock	getColMapping(FrameBlock meta) Obtain the column mapping of encoded frames based on the passed meta data frame.
abstract List<Callable<Object>>	getPartialBuildTasks(FrameBlock in, int blockSize)
boolean	isApplicable() Indicates if this encoder is applicable, i.e, if there is a column to encode.
boolean	isApplicable(int colID) Indicates if this encoder is applicable for the given column ID, i.e., if it is subject to this transformation.
void	mergeAt(ColumnEncoder other) Merges another encoder, of a compatible type, in after a certain position.
abstract void	mergeBuildPartial(List<Future<Object>> futurePartials, int start, int end)
void	prepareBuildPartial() Allocates internal data structures for partial build.
void	readExternal(ObjectInput in) Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd deserialization.
void	setColID(int colID)
void	shiftCol(int columnOffset)
void	updateIndexRanges(long[] beginDims, long[] endDims, int colOffset) Update index-ranges to after encoding.
void	writeExternal(ObjectOutput os) Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd serialization.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.sysds.runtime.transform.encode.Encoder

apply, build, getMetaData, initMetaData

Method Detail

apply

public abstract MatrixBlock apply(MatrixBlock in,
                                  MatrixBlock out,
                                  int outputCol)

apply

public abstract MatrixBlock apply(MatrixBlock in,
                                  MatrixBlock out,
                                  int outputCol,
                                  int rowStart,
                                  int blk)

apply

public abstract MatrixBlock apply(FrameBlock in,
                                  MatrixBlock out,
                                  int outputCol,
                                  int rowStart,
                                  int blk)

isApplicable

public boolean isApplicable()

Indicates if this encoder is applicable, i.e, if there is a column to encode.

Returns:

true if a colID is set

isApplicable

public boolean isApplicable(int colID)

Indicates if this encoder is applicable for the given column ID, i.e., if it is subject to this transformation.

Parameters:

colID - column ID

Returns:

true if encoder is applicable for given column

prepareBuildPartial

public void prepareBuildPartial()

Allocates internal data structures for partial build.

Specified by:

prepareBuildPartial in interface Encoder

buildPartial

public void buildPartial(FrameBlock in)

Partial build of internal data structures (e.g., in distributed spark operations).

Specified by:

buildPartial in interface Encoder

Parameters:

in - input frame block

mergeAt

public void mergeAt(ColumnEncoder other)

Merges another encoder, of a compatible type, in after a certain position. Resizes as necessary. ColumnEncoders are compatible with themselves and EncoderComposite is compatible with every other ColumnEncoders. MultiColumnEncoders are compatible with every encoder

Parameters:

other - the encoder that should be merged in

updateIndexRanges

public void updateIndexRanges(long[] beginDims,
                              long[] endDims,
                              int colOffset)

Update index-ranges to after encoding. Note that only Dummycoding changes the ranges.

Specified by:

updateIndexRanges in interface Encoder

Parameters:

beginDims - begin dimensions of range

endDims - end dimensions of range

colOffset - is applied to begin and endDims

getColMapping

public MatrixBlock getColMapping(FrameBlock meta)

Obtain the column mapping of encoded frames based on the passed meta data frame.

Parameters:

meta - meta data frame block

Returns:

matrix with column mapping (one row per attribute)

writeExternal

public void writeExternal(ObjectOutput os)
                   throws IOException

Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd serialization.

Specified by:

writeExternal in interface Externalizable

Parameters:

os - object output

Throws:

IOException - if IOException occurs

readExternal

public void readExternal(ObjectInput in)
                  throws IOException

Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd deserialization.

Specified by:

readExternal in interface Externalizable

Parameters:

in - object input

Throws:

IOException - if IOException occur

getColID

public int getColID()

setColID

public void setColID(int colID)

shiftCol

public void shiftCol(int columnOffset)

compareTo

public int compareTo(ColumnEncoder o)

Specified by:

compareTo in interface Comparable<ColumnEncoder>

getPartialBuildTasks

public abstract List<Callable<Object>> getPartialBuildTasks(FrameBlock in,
                                                            int blockSize)

mergeBuildPartial

public abstract void mergeBuildPartial(List<Future<Object>> futurePartials,
                                       int start,
                                       int end)
                                throws ExecutionException,
                                       InterruptedException

Throws:

ExecutionException

InterruptedException