org.apache.sysds.lops

Class Lop

java.lang.Object

org.apache.sysds.lops.Lop

Direct Known Subclasses:

Append, AppendG, AppendGAlignedSP, AppendM, AppendR, Binary, BinaryM, BinaryScalar, BinaryUAggChain, CentralMoment, Checkpoint, Compression, CoVariance, CSVReBlock, Ctable, CumulativeOffsetBinary, CumulativePartialAggregate, Data, DataGen, DeCompression, DnnTransform, Federated, FunctionCallCP, GroupedAggregate, GroupedAggregateM, LeftIndex, MapMult, MapMultChain, MatMultCP, MMCJ, MMRJ, MMTSJ, MMZip, Nary, ParameterizedBuiltin, PartialAggregate, PickByCount, PMapMult, PMMJ, ReBlock, RightIndex, SortKeys, SpoofFused, Sql, Ternary, TernaryAggregate, Transform, UAggOuterChain, Unary, UnaryCP, WeightedCrossEntropy, WeightedCrossEntropyR, WeightedDivMM, WeightedDivMMR, WeightedSigmoid, WeightedSigmoidR, WeightedSquaredLoss, WeightedSquaredLossR, WeightedUnaryMM, WeightedUnaryMMR

public abstract class Lop
extends Object

Base class for all Lops.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Lop.SimpleInstType Lop types
static class	Lop.Type
static class	Lop.VisitStatus

Field Summary

Fields

Modifier and Type	Field and Description
int	_beginColumn
int	_beginLine
int	_endColumn
int	_endLine
String	_filename
static String	CP_CHILD_THREAD
static String	CP_ROOT_THREAD_ID
static String	DATATYPE_PREFIX
static String	FILE_SEPARATOR
static String	FRAME_VAR_NAME_PREFIX
static String	INSTRUCTION_DELIMITOR
static String	LITERAL_PREFIX
static String	MATRIX_VAR_NAME_PREFIX
static String	NAME_VALUE_SEPARATOR
static String	OPERAND_DELIMITOR
static String	PROCESS_PREFIX
static double	SAMPLE_FRACTION
static String	SCALAR_VAR_NAME_PREFIX
static String	UPDATE_INPLACE_PREFIX
static String	VALUETYPE_PREFIX
static String	VARIABLE_NAME_PLACEHOLDER

Constructor Summary

Constructors

Constructor and Description
Lop(Lop.Type t, Types.DataType dt, Types.ValueType vt) Constructor to be invoked by base class.

Method Summary

Modifier and Type	Method and Description
void	addInput(Lop op) Method to add input to Lop
void	addOutput(Lop op) Method to add output to Lop
void	addToDag(Dag<Lop> dag) Method to recursively add LOPS to a DAG
boolean[]	createReachable(int size)
int	getBeginColumn()
int	getBeginLine()
Types.DataType	getDataType() get data type of the output that is produced by this lop
int	getEndColumn()
int	getEndLine()
Types.ExecType	getExecType() Method to get the execution type (CP, CP_FILE, MR, SPARK, GPU, FED, INVALID) of LOP
String	getFilename()
long	getID() Method to return the ID of LOP
ArrayList<Lop>	getInputs() Method to get input of Lops
String	getInstructions() Method should be overridden if needed
String	getInstructions(int input, int rowl, int rowu, int coll, int colu, int leftRowDim, int leftColDim, int output)
String	getInstructions(String output) Method should be overridden if needed
String	getInstructions(String[] inputs, String outputs)
String	getInstructions(String[] inputs, String[] outputs) Method should be overridden if needed
String	getInstructions(String input1, String output) Method should be overridden if needed
String	getInstructions(String input1, String input2, String output) Method should be overridden if needed
String	getInstructions(String input1, String input2, String input3, String output) Method should be overridden if needed
String	getInstructions(String input1, String input2, String input3, String input4, String output) Method should be overridden if needed
String	getInstructions(String input1, String input2, String input3, String input4, String input5, String output) Method should be overridden if needed
String	getInstructions(String input1, String input2, String input3, String input4, String input5, String input6, String output) Method should be overridden if needed
String	getInstructions(String input1, String input2, String input3, String input4, String input5, String input6, String input7, String output)
int	getLevel()
OutputParameters	getOutputParameters() Method to get output parameters
ArrayList<Lop>	getOutputs() Method to get output of Lops
PrivacyConstraint	getPrivacyConstraint()
boolean	getProducesIntermediateOutput()
boolean[]	getReachable()
Lop.SimpleInstType	getSimpleInstructionType() Method should be overridden if needed
Lop.Type	getType() Method to get Lop type.
Types.ValueType	getValueType() get value type of the output that is produced by this lop
Lop.VisitStatus	getVisited() get visit status of node
boolean	isDataExecLocation()
boolean	isVariable() Function that determines if the output of a LOP is defined by a variable or not.
String	prepInputOperand(int index)
String	prepInputOperand(String label)
String	prepOperand(String label, Types.DataType dt, Types.ValueType vt) Method to prepare instruction operand with given parameters.
String	prepOperand(String label, Types.DataType dt, Types.ValueType vt, boolean literal) Method to prepare instruction operand with given parameters.
String	prepOutputOperand()
String	prepOutputOperand(int index)
String	prepOutputOperand(String label)
String	prepScalarInputOperand(String label)
String	prepScalarInputOperand(Types.ExecType et)
String	prepScalarLabel() Function to prepare label for scalar inputs while generating instructions.
String	prepScalarOperand(Types.ExecType et, String label) Function to be used in creating instructions for creating scalar operands.
String	printErrorLocation()
int	removeConsumer()
void	resetVisitStatus()
void	setAllPositions(String filename, int blp, int bcp, int elp, int ecp)
void	setBeginColumn(int passed)
void	setBeginLine(int passed)
void	setConsumerCount(int cc)
void	setDataType(Types.DataType dt) set data type of the output that is produced by this lop
void	setEndColumn(int passed)
void	setEndLine(int passed)
void	setExecType(Types.ExecType newExecType) Set the execution type of LOP.
void	setFederatedOutput(FEDInstruction.FederatedOutput fedOutput)
void	setFilename(String passed)
void	setPrivacyConstraint(PrivacyConstraint privacy) Method to set privacy constraint of Lop.
void	setValueType(Types.ValueType vt) set value type of the output that is produced by this lop
void	setVisited(Lop.VisitStatus visited) set visit status of node
abstract String	toString() Method to have Lops print their state.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

FILE_SEPARATOR

public static final String FILE_SEPARATOR

See Also:

Constant Field Values

PROCESS_PREFIX

public static final String PROCESS_PREFIX

See Also:

Constant Field Values

CP_ROOT_THREAD_ID

public static final String CP_ROOT_THREAD_ID

See Also:

Constant Field Values

CP_CHILD_THREAD

public static final String CP_CHILD_THREAD

See Also:

Constant Field Values

SAMPLE_FRACTION

public static final double SAMPLE_FRACTION

See Also:

Constant Field Values

INSTRUCTION_DELIMITOR

public static final String INSTRUCTION_DELIMITOR

See Also:

Constant Field Values

OPERAND_DELIMITOR

public static final String OPERAND_DELIMITOR

See Also:

Constant Field Values

VALUETYPE_PREFIX

public static final String VALUETYPE_PREFIX

See Also:

Constant Field Values

DATATYPE_PREFIX

public static final String DATATYPE_PREFIX

See Also:

Constant Field Values

LITERAL_PREFIX

public static final String LITERAL_PREFIX

See Also:

Constant Field Values

VARIABLE_NAME_PLACEHOLDER

public static final String VARIABLE_NAME_PLACEHOLDER

See Also:

Constant Field Values

NAME_VALUE_SEPARATOR

public static final String NAME_VALUE_SEPARATOR

See Also:

Constant Field Values

MATRIX_VAR_NAME_PREFIX

public static final String MATRIX_VAR_NAME_PREFIX

See Also:

Constant Field Values

FRAME_VAR_NAME_PREFIX

public static final String FRAME_VAR_NAME_PREFIX

See Also:

Constant Field Values

SCALAR_VAR_NAME_PREFIX

public static final String SCALAR_VAR_NAME_PREFIX

See Also:

Constant Field Values

UPDATE_INPLACE_PREFIX

public static final String UPDATE_INPLACE_PREFIX

See Also:

Constant Field Values

_beginLine

public int _beginLine

_beginColumn

public int _beginColumn

_endLine

public int _endLine

_endColumn

public int _endColumn

_filename

public String _filename

Constructor Detail

Lop

public Lop(Lop.Type t,
           Types.DataType dt,
           Types.ValueType vt)

Constructor to be invoked by base class.

Parameters:

t - lop type

dt - data type

vt - value type

Method Detail

getVisited

public Lop.VisitStatus getVisited()

get visit status of node

Returns:

visit status

setVisited

public void setVisited(Lop.VisitStatus visited)

set visit status of node

Parameters:

visited - visit status

getReachable

public boolean[] getReachable()

createReachable

public boolean[] createReachable(int size)

isDataExecLocation

public boolean isDataExecLocation()

getDataType

public Types.DataType getDataType()

get data type of the output that is produced by this lop

Returns:

data type

setDataType

public void setDataType(Types.DataType dt)

set data type of the output that is produced by this lop

Parameters:

dt - data type

getValueType

public Types.ValueType getValueType()

get value type of the output that is produced by this lop

Returns:

value type

setValueType

public void setValueType(Types.ValueType vt)

set value type of the output that is produced by this lop

Parameters:

vt - value type

getType

public Lop.Type getType()

Method to get Lop type.

Returns:

lop type

getInputs

public ArrayList<Lop> getInputs()

Method to get input of Lops

Returns:

list of input lops

getOutputs

public ArrayList<Lop> getOutputs()

Method to get output of Lops

Returns:

list of output lops

addInput

public void addInput(Lop op)

Method to add input to Lop

Parameters:

op - input lop

addOutput

public void addOutput(Lop op)

Method to add output to Lop

Parameters:

op - output lop

setPrivacyConstraint

public void setPrivacyConstraint(PrivacyConstraint privacy)

Method to set privacy constraint of Lop.

Parameters:

privacy - privacy constraint instance

getPrivacyConstraint

public PrivacyConstraint getPrivacyConstraint()

setFederatedOutput

public void setFederatedOutput(FEDInstruction.FederatedOutput fedOutput)

setConsumerCount

public void setConsumerCount(int cc)

removeConsumer

public int removeConsumer()

toString

public abstract String toString()

Method to have Lops print their state. This is for debugging purposes.

Overrides:

toString in class Object

resetVisitStatus

public void resetVisitStatus()

getID

public long getID()

Method to return the ID of LOP

Returns:

lop ID

getLevel

public int getLevel()

getExecType

public Types.ExecType getExecType()

Method to get the execution type (CP, CP_FILE, MR, SPARK, GPU, FED, INVALID) of LOP

Returns:

execution type

setExecType

public void setExecType(Types.ExecType newExecType)

Set the execution type of LOP.

Parameters:

newExecType - new execution type

getProducesIntermediateOutput

public boolean getProducesIntermediateOutput()

addToDag

public final void addToDag(Dag<Lop> dag)

Method to recursively add LOPS to a DAG

Parameters:

dag - lop DAG

getOutputParameters

public OutputParameters getOutputParameters()

Method to get output parameters

Returns:

output parameters

getInstructions

public String getInstructions(String output)

Method should be overridden if needed

Parameters:

output - output

Returns:

instructions as string

getInstructions

public String getInstructions(String input1,
                              String output)

Method should be overridden if needed

Parameters:

input1 - input 1

output - output

Returns:

instructions as string

getInstructions

public String getInstructions(String input1,
                              String input2,
                              String output)

Method should be overridden if needed

Parameters:

input1 - input 1

input2 - input 2

output - output

Returns:

instructions as string

getInstructions

public String getInstructions(String input1,
                              String input2,
                              String input3,
                              String output)

Method should be overridden if needed

Parameters:

input1 - input 1

input2 - input 2

input3 - input 3

output - output

Returns:

instructions as string

getInstructions

public String getInstructions(String input1,
                              String input2,
                              String input3,
                              String input4,
                              String output)

Method should be overridden if needed

Parameters:

input1 - input 1

input2 - input 2

input3 - input 3

input4 - input 4

output - output

Returns:

instructions as string

getInstructions

public String getInstructions(String input1,
                              String input2,
                              String input3,
                              String input4,
                              String input5,
                              String output)

Method should be overridden if needed

Parameters:

input1 - input 1

input2 - input 2

input3 - input 3

input4 - input 4

input5 - input 5

output - output

Returns:

instructions as string

getInstructions

public String getInstructions(String input1,
                              String input2,
                              String input3,
                              String input4,
                              String input5,
                              String input6,
                              String output)

Method should be overridden if needed

Parameters:

input1 - input 1

input2 - input 2

input3 - input 3

input4 - input 4

input5 - input 5

input6 - input 6

output - output

Returns:

instructions as string

getInstructions

public String getInstructions(String input1,
                              String input2,
                              String input3,
                              String input4,
                              String input5,
                              String input6,
                              String input7,
                              String output)

getInstructions

public String getInstructions(String[] inputs,
                              String outputs)

getInstructions

public String getInstructions(String[] inputs,
                              String[] outputs)

Method should be overridden if needed

Parameters:

inputs - array of inputs

outputs - array of outputs

Returns:

instructions as string

getInstructions

public String getInstructions()

Method should be overridden if needed

Returns:

instructions as string

getSimpleInstructionType

public Lop.SimpleInstType getSimpleInstructionType()

Method should be overridden if needed

Returns:

simple instruction type

setBeginLine

public void setBeginLine(int passed)

setBeginColumn

public void setBeginColumn(int passed)

setEndLine

public void setEndLine(int passed)

setEndColumn

public void setEndColumn(int passed)

setFilename

public void setFilename(String passed)

setAllPositions

public void setAllPositions(String filename,
                            int blp,
                            int bcp,
                            int elp,
                            int ecp)

getBeginLine

public int getBeginLine()

getBeginColumn

public int getBeginColumn()

getEndLine

public int getEndLine()

getEndColumn

public int getEndColumn()

getFilename

public String getFilename()

printErrorLocation

public String printErrorLocation()

getInstructions

public String getInstructions(int input,
                              int rowl,
                              int rowu,
                              int coll,
                              int colu,
                              int leftRowDim,
                              int leftColDim,
                              int output)

isVariable

public boolean isVariable()

Function that determines if the output of a LOP is defined by a variable or not.

Returns:

true if lop output defined by a variable

prepOperand

public String prepOperand(String label,
                          Types.DataType dt,
                          Types.ValueType vt)

Method to prepare instruction operand with given parameters.

Parameters:

label - instruction label

dt - data type

vt - value type

Returns:

instruction operand with data type and value type

prepOperand

public String prepOperand(String label,
                          Types.DataType dt,
                          Types.ValueType vt,
                          boolean literal)

Method to prepare instruction operand with given parameters.

Parameters:

label - instruction label

dt - data type

vt - value type

literal - true if literal

Returns:

instruction operand with data type, value type, and literal status

prepOutputOperand

public String prepOutputOperand()

prepOutputOperand

public String prepOutputOperand(int index)

prepOutputOperand

public String prepOutputOperand(String label)

prepScalarLabel

public String prepScalarLabel()

Function to prepare label for scalar inputs while generating instructions. It attaches placeholder suffix and prefixes if the Lop denotes a variable.

Returns:

prepared scalar label

prepScalarOperand

public String prepScalarOperand(Types.ExecType et,
                                String label)

Function to be used in creating instructions for creating scalar operands. It decides whether or not attach placeholders for instruction patching. Resulting string also encodes if the operand is a literal. For non-literals: Placeholder prefix and suffix need to be attached for Instruction Patching during execution. However, they should NOT be attached IF: - the operand is a literal OR - the execution type is CP. This is because CP runtime has access to symbol table and the instruction encodes sufficient information to determine if an operand is a literal or not.

Parameters:

et - execution type

label - instruction label

Returns:

prepared scalar operand

prepScalarInputOperand

public String prepScalarInputOperand(Types.ExecType et)

prepScalarInputOperand

public String prepScalarInputOperand(String label)

prepInputOperand

public String prepInputOperand(int index)

prepInputOperand

public String prepInputOperand(String label)