org.apache.sysds.runtime.transform.encode

Class ColumnEncoderRecode

java.lang.Object

org.apache.sysds.runtime.transform.encode.ColumnEncoder

org.apache.sysds.runtime.transform.encode.ColumnEncoderRecode

All Implemented Interfaces:

Externalizable, Serializable, Comparable<ColumnEncoder>, Encoder

public class ColumnEncoderRecode
extends ColumnEncoder

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.sysds.runtime.transform.encode.ColumnEncoder

ColumnEncoder.EncoderType

Field Summary

Fields

Modifier and Type	Field and Description
static boolean	SORT_RECODE_MAP

Constructor Summary

Constructors

Constructor and Description
ColumnEncoderRecode()
ColumnEncoderRecode(int colID)

Method Summary

Modifier and Type	Method and Description
MatrixBlock	apply(FrameBlock in, MatrixBlock out, int outputCol) Apply the generated metadata to the FrameBlock and saved the result in out.
MatrixBlock	apply(FrameBlock in, MatrixBlock out, int outputCol, int rowStart, int blk)
MatrixBlock	apply(MatrixBlock in, MatrixBlock out, int outputCol)
MatrixBlock	apply(MatrixBlock in, MatrixBlock out, int outputCol, int rowStart, int blk)
void	build(FrameBlock in) Build the transform meta data for the given block input.
void	buildPartial(FrameBlock in) Partial build of internal data structures (e.g., in distributed spark operations).
static String	constructRecodeMapEntry(String token, Long code) Returns the Recode map entry which consists of concatenation of code, delimiter and token.
boolean	equals(Object o)
HashMap<String,Long>	getCPRecodeMaps()
HashSet<Object>	getCPRecodeMapsPartial()
FrameBlock	getMetaData(FrameBlock meta) Construct a frame block out of the transform meta data.
int	getNumDistinctValues()
List<Callable<Object>>	getPartialBuildTasks(FrameBlock in, int blockSize)
HashMap<String,Long>	getRcdMap()
int	hashCode()
void	initMetaData(FrameBlock meta) Construct the recodemaps from the given input frame for all columns registered for recode.
void	mergeAt(ColumnEncoder other) Merges another encoder, of a compatible type, in after a certain position.
void	mergeBuildPartial(List<Future<Object>> futurePartials, int start, int end)
void	prepareBuildPartial() Allocates internal data structures for partial build.
void	readExternal(ObjectInput in) Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd deserialization.
void	sortCPRecodeMaps()
static String[]	splitRecodeMapEntry(String value) Splits a Recode map entry into its token and code.
void	writeExternal(ObjectOutput out) Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd serialization.

Methods inherited from class org.apache.sysds.runtime.transform.encode.ColumnEncoder

compareTo, getColID, getColMapping, isApplicable, isApplicable, setColID, shiftCol, updateIndexRanges

Methods inherited from class java.lang.Object

getClass, notify, notifyAll, toString, wait, wait, wait

Field Detail

SORT_RECODE_MAP

public static boolean SORT_RECODE_MAP

Constructor Detail

ColumnEncoderRecode

public ColumnEncoderRecode(int colID)

ColumnEncoderRecode

public ColumnEncoderRecode()

Method Detail

constructRecodeMapEntry

public static String constructRecodeMapEntry(String token,
                                             Long code)

Returns the Recode map entry which consists of concatenation of code, delimiter and token.

Parameters:

token - is part of Recode map

code - is code for token

Returns:

the concatenation of token and code with delimiter in between

splitRecodeMapEntry

public static String[] splitRecodeMapEntry(String value)

Splits a Recode map entry into its token and code.

Parameters:

value - concatenation of token and code with delimiter in between

Returns:

string array of token and code

getCPRecodeMaps

public HashMap<String,Long> getCPRecodeMaps()

getCPRecodeMapsPartial

public HashSet<Object> getCPRecodeMapsPartial()

sortCPRecodeMaps

public void sortCPRecodeMaps()

build

public void build(FrameBlock in)

Description copied from interface: Encoder

Build the transform meta data for the given block input. This call modifies and keeps meta data as encoder state.

Parameters:

in - input frame block

getPartialBuildTasks

public List<Callable<Object>> getPartialBuildTasks(FrameBlock in,
                                                   int blockSize)

Specified by:

getPartialBuildTasks in class ColumnEncoder

mergeBuildPartial

public void mergeBuildPartial(List<Future<Object>> futurePartials,
                              int start,
                              int end)
                       throws ExecutionException,
                              InterruptedException

Specified by:

mergeBuildPartial in class ColumnEncoder

Throws:

ExecutionException

InterruptedException

prepareBuildPartial

public void prepareBuildPartial()

Description copied from class: ColumnEncoder

Allocates internal data structures for partial build.

Specified by:

prepareBuildPartial in interface Encoder

Overrides:

prepareBuildPartial in class ColumnEncoder

buildPartial

public void buildPartial(FrameBlock in)

Description copied from class: ColumnEncoder

Partial build of internal data structures (e.g., in distributed spark operations).

Specified by:

buildPartial in interface Encoder

Overrides:

buildPartial in class ColumnEncoder

Parameters:

in - input frame block

apply

public MatrixBlock apply(FrameBlock in,
                         MatrixBlock out,
                         int outputCol)

Description copied from interface: Encoder

Apply the generated metadata to the FrameBlock and saved the result in out.

Parameters:

in - input frame block

out - output matrix block

outputCol - is a offset in the output matrix. column in FrameBlock + outputCol = column in out

Returns:

output matrix block

apply

public MatrixBlock apply(FrameBlock in,
                         MatrixBlock out,
                         int outputCol,
                         int rowStart,
                         int blk)

Specified by:

apply in class ColumnEncoder

apply

public MatrixBlock apply(MatrixBlock in,
                         MatrixBlock out,
                         int outputCol,
                         int rowStart,
                         int blk)

Specified by:

apply in class ColumnEncoder

apply

public MatrixBlock apply(MatrixBlock in,
                         MatrixBlock out,
                         int outputCol)

Specified by:

apply in class ColumnEncoder

mergeAt

public void mergeAt(ColumnEncoder other)

Description copied from class: ColumnEncoder

Merges another encoder, of a compatible type, in after a certain position. Resizes as necessary. ColumnEncoders are compatible with themselves and EncoderComposite is compatible with every other ColumnEncoders. MultiColumnEncoders are compatible with every encoder

Overrides:

mergeAt in class ColumnEncoder

Parameters:

other - the encoder that should be merged in

getNumDistinctValues

public int getNumDistinctValues()

getMetaData

public FrameBlock getMetaData(FrameBlock meta)

Description copied from interface: Encoder

Construct a frame block out of the transform meta data.

Parameters:

meta - output frame block

Returns:

output frame block?

initMetaData

public void initMetaData(FrameBlock meta)

Construct the recodemaps from the given input frame for all columns registered for recode.

Parameters:

meta - frame block

writeExternal

public void writeExternal(ObjectOutput out)
                   throws IOException

Description copied from class: ColumnEncoder

Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd serialization.

Specified by:

writeExternal in interface Externalizable

Overrides:

writeExternal in class ColumnEncoder

Parameters:

out - object output

Throws:

IOException - if IOException occurs

readExternal

public void readExternal(ObjectInput in)
                  throws IOException

Description copied from class: ColumnEncoder

Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd deserialization.

Specified by:

readExternal in interface Externalizable

Overrides:

readExternal in class ColumnEncoder

Parameters:

in - object input

Throws:

IOException - if IOException occur

equals

public boolean equals(Object o)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object

getRcdMap

public HashMap<String,Long> getRcdMap()