Package org.apache.sysds.lops

Class Data

java.lang.Object

org.apache.sysds.lops.Lop

org.apache.sysds.lops.Data

public class Data
extends Lop

Lop to represent data objects. Data objects represent matrices, vectors, variables, literals. Can be for both input and output.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.sysds.lops.Lop

Lop.SimpleInstType, Lop.Type, Lop.VisitStatus

Field Summary

Fields

Modifier and Type	Field	Description
static String	PREAD_PREFIX

Fields inherited from class org.apache.sysds.lops.Lop

_beginColumn, _beginLine, _endColumn, _endLine, _filename, CP_CHILD_THREAD, CP_ROOT_THREAD_ID, DATATYPE_PREFIX, FILE_SEPARATOR, FRAME_VAR_NAME_PREFIX, INSTRUCTION_DELIMITOR, LITERAL_PREFIX, MATRIX_VAR_NAME_PREFIX, NAME_VALUE_SEPARATOR, OPERAND_DELIMITOR, PROCESS_PREFIX, SAMPLE_FRACTION, SCALAR_VAR_NAME_PREFIX, UPDATE_INPLACE_PREFIX, VALUETYPE_PREFIX, VARIABLE_NAME_PLACEHOLDER

Constructor Summary

Constructors

Constructor	Description
Data(Types.OpOpData op, Lop input, HashMap<String,Lop> inputParametersLops, String name, String literal, Types.DataType dt, Types.ValueType vt, Types.FileFormat fmt)	Constructor to setup read or write LOP In case of write: input must be provided.

Method Summary

Modifier and Type	Method	Description
static Data	createLiteralLop(Types.ValueType vt, String literalValue)	Method to create literal LOPs.
boolean	getBooleanValue()
String	getCreateVarInstructions(String outputFileName, String outputLabel)
double	getDoubleValue()
Types.FileFormat	getFileFormatType()	method to get format type for input, output files.
HashMap<String,Lop>	getInputParams()	method to get inputParams
String	getInstructions()	Method to generate createvar instruction that updates symbol table with metadata, hdfsfile name, etc.
String	getInstructions(String outputFileName)	Method should be overridden if needed
String	getInstructions(String input1, String input2)	Method to get CP instructions for reading/writing scalars and matrices from/to HDFS.
long	getLongValue()
Lop	getNamedInputLop(String name)
Lop	getNamedInputLop(String name, String defaultVal)
Types.OpOpData	getOperationType()	method to get operation type, i.e.
String	getStringValue()
boolean	isLiteral()	method to check if this data lop represents a literal.
boolean	isPersistentRead()
boolean	isPersistentWrite()
void	setExecType(Types.ExecType et)	Data-Lop-specific method to set the execution type for persistent write.
String	toString()	Method to have Lops print their state.

Methods inherited from class org.apache.sysds.lops.Lop

activatePrefetch, addInput, addOutput, addToDag, createReachable, getAggType, getBeginColumn, getBeginLine, getBroadcastInput, getDataType, getEndColumn, getEndLine, getExecType, getFederatedOutput, getFilename, getID, getInputs, getInstructions, getInstructions, getInstructions, getInstructions, getInstructions, getInstructions, getInstructions, getInstructions, getInstructions, getLevel, getOutputParameters, getOutputs, getPrivacyConstraint, getProducesIntermediateOutput, getReachable, getSimpleInstructionType, getType, getValueType, getVisited, isAllOutputsCP, isDataExecLocation, isVariable, prefetchActivated, prepInputOperand, prepInputOperand, prepOperand, prepOperand, prepOutputOperand, prepOutputOperand, prepOutputOperand, prepScalarInputOperand, prepScalarInputOperand, prepScalarLabel, prepScalarOperand, printErrorLocation, removeConsumer, removeOutput, replaceInput, resetVisitStatus, setAllPositions, setBeginColumn, setBeginLine, setConsumerCount, setDataType, setEndColumn, setEndLine, setFederatedOutput, setFilename, setPrivacyConstraint, setValueType, setVisited

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

PREAD_PREFIX

public static final String PREAD_PREFIX

See Also:

Constant Field Values

Constructor Detail

Data

public Data(Types.OpOpData op,
            Lop input,
            HashMap<String,Lop> inputParametersLops,
            String name,
            String literal,
            Types.DataType dt,
            Types.ValueType vt,
            Types.FileFormat fmt)

Constructor to setup read or write LOP In case of write: input must be provided. This will always be added as the first element in input array. For literals: this function is invoked through the static method createLiteralLop.

Parameters:

op - operation type

input - low-level operator

inputParametersLops - input lops

name - string name

literal - string literal

dt - data type

vt - value type

fmt - file format

Method Detail

createLiteralLop

public static Data createLiteralLop(Types.ValueType vt,
                                    String literalValue)

Method to create literal LOPs.

Parameters:

vt - value type

literalValue - literal value

Returns:

literal low-level operator

setExecType

public void setExecType(Types.ExecType et)

Data-Lop-specific method to set the execution type for persistent write. TODO: split lops into MR/CP lop.

Overrides:

setExecType in class Lop

Parameters:

et - execution type

getFileFormatType

public Types.FileFormat getFileFormatType()

method to get format type for input, output files.

Returns:

file format

toString

public String toString()

Description copied from class: Lop

Method to have Lops print their state. This is for debugging purposes.

Specified by:

toString in class Lop

getOperationType

public Types.OpOpData getOperationType()

method to get operation type, i.e. read/write.

Returns:

operation type

getInputParams

public HashMap<String,Lop> getInputParams()

method to get inputParams

Returns:

input parameters

getNamedInputLop

public Lop getNamedInputLop(String name)

getNamedInputLop

public Lop getNamedInputLop(String name,
                            String defaultVal)

isLiteral

public boolean isLiteral()

method to check if this data lop represents a literal.

Returns:

true if data lop is a literal

getBooleanValue

public boolean getBooleanValue()

getDoubleValue

public double getDoubleValue()

getLongValue

public long getLongValue()

getStringValue

public String getStringValue()

isPersistentWrite

public boolean isPersistentWrite()

isPersistentRead

public boolean isPersistentRead()

getInstructions

public String getInstructions(String input1,
                              String input2)

Method to get CP instructions for reading/writing scalars and matrices from/to HDFS. This method generates CP read/write instructions.

Overrides:

getInstructions in class Lop

Parameters:

input1 - input 1

input2 - output

Returns:

instructions as string

getInstructions

public String getInstructions()

Method to generate createvar instruction that updates symbol table with metadata, hdfsfile name, etc.

Overrides:

getInstructions in class Lop

Returns:

instructions as string

getInstructions

public String getInstructions(String outputFileName)

Description copied from class: Lop

Method should be overridden if needed

Overrides:

getInstructions in class Lop

Parameters:

outputFileName - output

Returns:

instructions as string

getCreateVarInstructions

public String getCreateVarInstructions(String outputFileName,
                                       String outputLabel)