Package org.apache.sysds.lops

Class Lop

java.lang.Object
org.apache.sysds.lops.Lop
Direct Known Subclasses:
Append, AppendG, AppendGAlignedSP, AppendM, AppendR, Binary, BinaryM, BinaryScalar, BinaryUAggChain, CentralMoment, Checkpoint, Compression, CoVariance, CSVReBlock, Ctable, CumulativeOffsetBinary, CumulativePartialAggregate, Data, DataGen, DeCompression, DnnTransform, Federated, FunctionCallCP, GroupedAggregate, GroupedAggregateM, LeftIndex, Local, MapMult, MapMultChain, MatMultCP, MMCJ, MMRJ, MMTSJ, MMZip, Nary, ParameterizedBuiltin, PartialAggregate, PickByCount, PMapMult, PMMJ, ReBlock, RightIndex, SortKeys, SpoofFused, Sql, Tee, Ternary, TernaryAggregate, Transform, UAggOuterChain, Unary, UnaryCP, WeightedCrossEntropy, WeightedCrossEntropyR, WeightedDivMM, WeightedDivMMR, WeightedSigmoid, WeightedSigmoidR, WeightedSquaredLoss, WeightedSquaredLossR, WeightedUnaryMM, WeightedUnaryMMR
public abstract class Lop extends Object
Base class for all Lops.

  • Field Details

  • Constructor Details

    • Lop

      public Lop(Lop.Type t, Types.DataType dt, Types.ValueType vt)
      Constructor to be invoked by base class.
      Parameters:
      t - lop type
      dt - data type of the output
      vt - value type of the output

  • Method Details

    • getVisited

      public Lop.VisitStatus getVisited()
      get visit status of node
      Returns:
      visit status

    • setVisited

      public void setVisited(Lop.VisitStatus visited)
      set visit status of node
      Parameters:
      visited - visit status

    • setVisited

      public void setVisited()

    • isVisited

      public boolean isVisited()

    • getReachable

      public boolean[] getReachable()

    • createReachable

      public boolean[] createReachable(int size)

    • isDataExecLocation

      public boolean isDataExecLocation()

    • getDataType

      public Types.DataType getDataType()
      get data type of the output that is produced by this lop
      Returns:
      data type

    • setDataType

      public void setDataType(Types.DataType dt)
      set data type of the output that is produced by this lop
      Parameters:
      dt - data type

    • getValueType

      public Types.ValueType getValueType()
      get value type of the output that is produced by this lop
      Returns:
      value type

    • setValueType

      public void setValueType(Types.ValueType vt)
      set value type of the output that is produced by this lop
      Parameters:
      vt - value type

    • getType

      public Lop.Type getType()
      Method to get Lop type.
      Returns:
      lop type

    • getInputs

      public ArrayList<Lop> getInputs()
      Method to get input of Lops
      Returns:
      list of input lops

    • getInput

      public Lop getInput(int index)

    • getOutputs

      public ArrayList<Lop> getOutputs()
      Method to get output of Lops
      Returns:
      list of output lops

    • addInput

      public void addInput(Lop op)
      Method to add input to Lop
      Parameters:
      op - input lop

    • replaceInput

      public void replaceInput(Lop oldInp, Lop newInp)
      Method to replace an input to a Lop
      Parameters:
      oldInp - old input Lop
      newInp - new input Lop

    • replaceAllInputs

      public void replaceAllInputs(ArrayList<Lop> newInputs)

    • replaceAllOutputs

      public void replaceAllOutputs(ArrayList<Lop> newOutputs)

    • removeInput

      public void removeInput(Lop op)

    • addOutput

      public void addOutput(Lop op)
      Method to add output to Lop
      Parameters:
      op - output lop

    • removeOutput

      public void removeOutput(Lop op)
      Method to remove output from Lop
      Parameters:
      op - Lop to remove

    • activatePrefetch

      public void activatePrefetch()

    • prefetchActivated

      public boolean prefetchActivated()

    • setFederatedOutput

      public void setFederatedOutput(FEDInstruction.FederatedOutput fedOutput)

    • getFederatedOutput

      public FEDInstruction.FederatedOutput getFederatedOutput()

    • setConsumerCount

      public void setConsumerCount(int cc)

    • removeConsumer

      public int removeConsumer()

    • setAsynchronous

      public void setAsynchronous(boolean isAsync)

    • isAsynchronousOp

      public boolean isAsynchronousOp()

    • setPipelineID

      public void setPipelineID(int id)

    • getPipelineID

      public int getPipelineID()

    • setMemoryEstimates

      public void setMemoryEstimates(double outMem, double totMem, double interMem, double bcMem)

    • getTotalMemoryEstimate

      public double getTotalMemoryEstimate()

    • getOutputMemoryEstimate

      public double getOutputMemoryEstimate()

    • setComputeEstimate

      public void setComputeEstimate(double compCost)

    • getComputeEstimate

      public double getComputeEstimate()

    • toString

      public abstract String toString()
      Method to have Lops print their state. This is for debugging purposes.
      Overrides:
      toString in class Object

    • resetVisitStatus

      public void resetVisitStatus()

    • getID

      public long getID()
      Method to return the ID of LOP
      Returns:
      lop ID

    • setNewID

      public void setNewID()

    • getLevel

      public int getLevel()

    • getExecType

      public Types.ExecType getExecType()
      Method to get the execution type (CP, CP_FILE, MR, SPARK, GPU, FED, INVALID) of LOP
      Returns:
      execution type

    • setExecType

      public void setExecType(Types.ExecType newExecType)
      Set the execution type of LOP.
      Parameters:
      newExecType - new execution type

    • isExecSpark

      public boolean isExecSpark()

    • isExecGPU

      public boolean isExecGPU()

    • isExecCP

      public boolean isExecCP()

    • getProducesIntermediateOutput

      public boolean getProducesIntermediateOutput()

    • addToDag

      public final void addToDag(Dag<Lop> dag)
      Method to recursively add LOPS to a DAG
      Parameters:
      dag - lop DAG

    • getOutputParameters

      public OutputParameters getOutputParameters()
      Method to get output parameters
      Returns:
      output parameters

    • getNumRows

      public long getNumRows()

    • getNumCols

      public long getNumCols()

    • getNnz

      public long getNnz()

    • getAggType

      public AggBinaryOp.SparkAggType getAggType()
      Method to get aggregate type if applicable. This method is overridden by the Lops with aggregate types (e.g. MapMult)
      Returns:
      SparkAggType

    • getBroadcastInput

      public Lop getBroadcastInput()
      Method to get the input to be broadcast. This method is overridden by the Lops which require broadcasts (e.g. AppendM)
      Returns:
      An input Lop or Null

    • getInstructions

      public String getInstructions(String output)
      Method should be overridden if needed
      Parameters:
      output - output
      Returns:
      instructions as string

    • getInstructions

      public String getInstructions(String input1, String output)
      Method should be overridden if needed
      Parameters:
      input1 - input 1
      output - output
      Returns:
      instructions as string

    • getInstructions

      public String getInstructions(String input1, String input2, String output)
      Method should be overridden if needed
      Parameters:
      input1 - input 1
      input2 - input 2
      output - output
      Returns:
      instructions as string

    • getInstructions

      public String getInstructions(String input1, String input2, String input3, String output)
      Method should be overridden if needed
      Parameters:
      input1 - input 1
      input2 - input 2
      input3 - input 3
      output - output
      Returns:
      instructions as string

    • getInstructions

      public String getInstructions(String input1, String input2, String input3, String input4, String output)
      Method should be overridden if needed
      Parameters:
      input1 - input 1
      input2 - input 2
      input3 - input 3
      input4 - input 4
      output - output
      Returns:
      instructions as string

    • getInstructions

      public String getInstructions(String input1, String input2, String input3, String input4, String input5, String output)
      Method should be overridden if needed
      Parameters:
      input1 - input 1
      input2 - input 2
      input3 - input 3
      input4 - input 4
      input5 - input 5
      output - output
      Returns:
      instructions as string

    • getInstructions

      public String getInstructions(String input1, String input2, String input3, String input4, String input5, String input6, String output)
      Method should be overridden if needed
      Parameters:
      input1 - input 1
      input2 - input 2
      input3 - input 3
      input4 - input 4
      input5 - input 5
      input6 - input 6
      output - output
      Returns:
      instructions as string

    • getInstructions

      public String getInstructions(String input1, String input2, String input3, String input4, String input5, String input6, String input7, String output)

    • getInstructions

      public String getInstructions(String[] inputs, String outputs)

    • getInstructions

      public String getInstructions(String[] inputs, String[] outputs)
      Method should be overridden if needed
      Parameters:
      inputs - array of inputs
      outputs - array of outputs
      Returns:
      instructions as string

    • getInstructions

      public String getInstructions()
      Method should be overridden if needed
      Returns:
      instructions as string

    • getSimpleInstructionType

      public Lop.SimpleInstType getSimpleInstructionType()
      Method should be overridden if needed
      Returns:
      simple instruction type

    • setBeginLine

      public void setBeginLine(int passed)

    • setBeginColumn

      public void setBeginColumn(int passed)

    • setEndLine

      public void setEndLine(int passed)

    • setEndColumn

      public void setEndColumn(int passed)

    • setFilename

      public void setFilename(String passed)

    • setAllPositions

      public void setAllPositions(String filename, int blp, int bcp, int elp, int ecp)

    • getBeginLine

      public int getBeginLine()

    • getBeginColumn

      public int getBeginColumn()

    • getEndLine

      public int getEndLine()

    • getEndColumn

      public int getEndColumn()

    • getFilename

      public String getFilename()

    • printErrorLocation

      public String printErrorLocation()

    • getInstructions

      public String getInstructions(int input, int rowl, int rowu, int coll, int colu, int leftRowDim, int leftColDim, int output)

    • isVariable

      public boolean isVariable()
      Function that determines if the output of a LOP is defined by a variable or not.
      Returns:
      true if lop output defined by a variable

    • isAllOutputsCP

      public boolean isAllOutputsCP()
      Function that determines if all the outputs of a LOP are of CP execution types
      Returns:
      true if all outputs are CP

    • isAllOutputsGPU

      public boolean isAllOutputsGPU()
      Function that determines if all the outputs of a LOP are of GPU execution types
      Returns:
      true if all outputs are CP

    • prepOperand

      public String prepOperand(String label, Types.DataType dt, Types.ValueType vt)
      Method to prepare instruction operand with given parameters.
      Parameters:
      label - instruction label
      dt - data type
      vt - value type
      Returns:
      instruction operand with data type and value type

    • prepOperand

      public String prepOperand(String label, Types.DataType dt, Types.ValueType vt, boolean literal)
      Method to prepare instruction operand with given parameters.
      Parameters:
      label - instruction label
      dt - data type
      vt - value type
      literal - true if literal
      Returns:
      instruction operand with data type, value type, and literal status

    • prepOutputOperand

      public String prepOutputOperand()

    • prepOutputOperand

      public String prepOutputOperand(int index)

    • prepOutputOperand

      public String prepOutputOperand(String label)

    • prepScalarLabel

      public String prepScalarLabel()
      Function to prepare label for scalar inputs while generating instructions. It attaches placeholder suffix and prefixes if the Lop denotes a variable.
      Returns:
      prepared scalar label

    • prepScalarOperand

      public String prepScalarOperand(Types.ExecType et, String label)
      Function to be used in creating instructions for creating scalar operands. It decides whether or not attach placeholders for instruction patching. Resulting string also encodes if the operand is a literal. For non-literals: Placeholder prefix and suffix need to be attached for Instruction Patching during execution. However, they should NOT be attached IF: - the operand is a literal OR - the execution type is CP. This is because CP runtime has access to symbol table and the instruction encodes sufficient information to determine if an operand is a literal or not.
      Parameters:
      et - execution type
      label - instruction label
      Returns:
      prepared scalar operand

    • prepScalarInputOperand

      public String prepScalarInputOperand(Types.ExecType et)

    • prepScalarInputOperand

      public String prepScalarInputOperand(String label)

    • prepInputOperand

      public String prepInputOperand(int index)

    • prepInputOperand

      public String prepInputOperand(String label)