org.apache.sysds.runtime.controlprogram.context

Class ExecutionContext

java.lang.Object

org.apache.sysds.runtime.controlprogram.context.ExecutionContext

Direct Known Subclasses:

SparkExecutionContext

public class ExecutionContext
extends Object

Constructor Summary

Constructors

Constructor and Description
ExecutionContext(LocalVariableMap vars)

Method Summary

Modifier and Type	Method and Description
MatrixObject	allocateGPUMatrixObject(String varName, long numRows, long numCols) Allocates the GPUObject for a given LOPS Variable (eg.
void	cleanupCacheableData(CacheableData<?> mo)
void	cleanupDataObject(Data dat)
boolean	containsVariable(CPOperand operand)
boolean	containsVariable(String name)
static CacheableData<?>	createCacheableData(CacheBlock cb)
static FrameObject	createFrameObject(FrameBlock fb)
static MatrixObject	createMatrixObject(MatrixBlock mb)
CacheableData<?>	getCacheableData(CPOperand input)
CacheableData<?>	getCacheableData(String varname)
DataCharacteristics	getDataCharacteristics(String varname)
Pair<MatrixObject,Boolean>	getDenseMatrixOutputForGPUInstruction(String varName, long numRows, long numCols) Allocates a dense matrix on the GPU (for output)
FrameBlock	getFrameInput(String varName) Pins a frame variable into memory and returns the internal frame block.
FrameObject	getFrameObject(CPOperand input)
FrameObject	getFrameObject(String varname)
GPUContext	getGPUContext(int index) Get the i-th GPUContext
List<GPUContext>	getGPUContexts() Gets the list of GPUContexts
long	getGPUDensePointerAddress(MatrixObject obj)
CSRPointer	getGPUSparsePointerAddress(MatrixObject obj)
Lineage	getLineage()
LineageItem	getLineageItem(CPOperand input)
ListObject	getListObject(CPOperand input)
ListObject	getListObject(String name)
MatrixBlock	getMatrixInput(String varName) Pins a matrix variable into memory and returns the internal matrix block.
MatrixObject	getMatrixInputForGPUInstruction(String varName, String opcode)
List<MatrixBlock>	getMatrixInputs(CPOperand[] inputs)
List<MatrixBlock>	getMatrixInputs(CPOperand[] inputs, boolean includeList)
MatrixObject	getMatrixObject(CPOperand input)
MatrixObject	getMatrixObject(String varname)
MetaData	getMetaData(String varname)
int	getNumGPUContexts() Gets the number of GPUContexts
LineageItem	getOrCreateLineageItem(CPOperand input)
Program	getProgram()
ScalarObject	getScalarInput(CPOperand input)
ScalarObject	getScalarInput(String name, Types.ValueType vt, boolean isLiteral)
List<ScalarObject>	getScalarInputs(CPOperand[] inputs)
Pair<MatrixObject,Boolean>	getSparseMatrixOutputForGPUInstruction(String varName, long numRows, long numCols, long nnz) Allocates a sparse matrix in CSR format on the GPU.
TensorBlock	getTensorInput(String varName) Pins a matrix variable into memory and returns the internal matrix block.
TensorObject	getTensorObject(String varname)
long	getTID()
Data	getVariable(CPOperand operand)
Data	getVariable(String name)
LocalVariableMap	getVariables()
ArrayList<String>	getVarList() NOTE: No order guaranteed, so keep same list for pin and unpin.
ArrayList<String>	getVarListPartitioned() NOTE: No order guaranteed, so keep same list for pin and unpin.
boolean	isAutoCreateVars()
boolean	isFederated(CPOperand input)
boolean	isFederated(CPOperand input, FederationMap.FType type)
boolean	isFrameObject(String varname)
boolean	isMatrixObject(String varname)
void	maintainLineageDebuggerInfo(Instruction inst)
boolean[]	pinVariables(List<String> varList) Pin a given list of variables i.e., set the "clean up" state in corresponding matrix objects, so that the cached data inside these objects is not cleared and the corresponding HDFS files are not deleted (through rmvar instructions).
void	releaseCacheableData(String varname)
void	releaseFrameInput(String varName) Unpins a currently pinned frame variable.
void	releaseMatrixInput(String... varNames)
void	releaseMatrixInput(String varName) Unpins a currently pinned matrix variable and update fine-grained statistics.
void	releaseMatrixInputForGPUInstruction(String varName)
void	releaseMatrixInputs(CPOperand[] inputs)
void	releaseMatrixInputs(CPOperand[] inputs, boolean includeList)
void	releaseMatrixOutputForGPUInstruction(String varName)
void	releaseTensorInput(String... varNames)
void	releaseTensorInput(String varName)
Data	removeVariable(String name)
void	setAutoCreateVars(boolean flag)
void	setFrameOutput(String varName, FrameBlock outputData)
void	setGPUContexts(List<GPUContext> gpuContexts) Sets the list of GPUContexts
void	setLineage(Lineage lineage)
void	setMatrixOutput(String varName, MatrixBlock outputData)
void	setMatrixOutput(String varName, MatrixBlock outputData, MatrixObject.UpdateType flag)
void	setMatrixOutputAndLineage(String varName, MatrixBlock outputData, LineageItem li)
void	setMetaData(String varName, long nrows, long ncols)
void	setMetaData(String fname, MetaData md)
void	setProgram(Program prog)
void	setScalarOutput(String varName, ScalarObject so)
void	setTensorOutput(String varName, TensorBlock outputData)
void	setTID(long tid)
void	setVariable(String name, Data val)
void	setVariables(LocalVariableMap vars)
String	toString()
void	traceLineage(Instruction inst)
void	unpinVariables(List<String> varList, boolean[] varsState) Unpin the a given list of variables by setting their "cleanup" status to the values specified by varsStats.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

ExecutionContext

public ExecutionContext(LocalVariableMap vars)

Method Detail

getProgram

public Program getProgram()

setProgram

public void setProgram(Program prog)

getVariables

public LocalVariableMap getVariables()

setVariables

public void setVariables(LocalVariableMap vars)

getLineage

public Lineage getLineage()

setLineage

public void setLineage(Lineage lineage)

isAutoCreateVars

public boolean isAutoCreateVars()

setAutoCreateVars

public void setAutoCreateVars(boolean flag)

setTID

public void setTID(long tid)

getTID

public long getTID()

getGPUContext

public GPUContext getGPUContext(int index)

Get the i-th GPUContext

Parameters:

index - index of the GPUContext

Returns:

a valid GPUContext or null if the indexed GPUContext does not exist.

setGPUContexts

public void setGPUContexts(List<GPUContext> gpuContexts)

Sets the list of GPUContexts

Parameters:

gpuContexts - a collection of GPUContexts

getGPUContexts

public List<GPUContext> getGPUContexts()

Gets the list of GPUContexts

Returns:

a list of GPUContexts

getNumGPUContexts

public int getNumGPUContexts()

Gets the number of GPUContexts

Returns:

number of GPUContexts

getVariable

public Data getVariable(String name)

getVariable

public Data getVariable(CPOperand operand)

setVariable

public void setVariable(String name,
                        Data val)

containsVariable

public boolean containsVariable(CPOperand operand)

containsVariable

public boolean containsVariable(String name)

removeVariable

public Data removeVariable(String name)

setMetaData

public void setMetaData(String fname,
                        MetaData md)

getMetaData

public MetaData getMetaData(String varname)

isMatrixObject

public boolean isMatrixObject(String varname)

getMatrixObject

public MatrixObject getMatrixObject(CPOperand input)

getMatrixObject

public MatrixObject getMatrixObject(String varname)

getTensorObject

public TensorObject getTensorObject(String varname)

isFrameObject

public boolean isFrameObject(String varname)

getFrameObject

public FrameObject getFrameObject(CPOperand input)

getFrameObject

public FrameObject getFrameObject(String varname)

getCacheableData

public CacheableData<?> getCacheableData(CPOperand input)

getCacheableData

public CacheableData<?> getCacheableData(String varname)

releaseCacheableData

public void releaseCacheableData(String varname)

getDataCharacteristics

public DataCharacteristics getDataCharacteristics(String varname)

getMatrixInput

public MatrixBlock getMatrixInput(String varName)

Pins a matrix variable into memory and returns the internal matrix block.

Parameters:

varName - variable name

Returns:

matrix block

getTensorInput

public TensorBlock getTensorInput(String varName)

Pins a matrix variable into memory and returns the internal matrix block.

Parameters:

varName - variable name

Returns:

matrix block

setMetaData

public void setMetaData(String varName,
                        long nrows,
                        long ncols)

getDenseMatrixOutputForGPUInstruction

public Pair<MatrixObject,Boolean> getDenseMatrixOutputForGPUInstruction(String varName,
                                                                        long numRows,
                                                                        long numCols)

Allocates a dense matrix on the GPU (for output)

Parameters:

varName - name of the output matrix (known by this ExecutionContext)

numRows - number of rows of matrix object

numCols - number of columns of matrix object

Returns:

a pair containing the wrapping MatrixObject and a boolean indicating whether a cuda memory allocation took place (as opposed to the space already being allocated)

getSparseMatrixOutputForGPUInstruction

public Pair<MatrixObject,Boolean> getSparseMatrixOutputForGPUInstruction(String varName,
                                                                         long numRows,
                                                                         long numCols,
                                                                         long nnz)

Allocates a sparse matrix in CSR format on the GPU. Assumes that mat.getNumRows() returns a valid number

Parameters:

varName - variable name

numRows - number of rows of matrix object

numCols - number of columns of matrix object

nnz - number of non zeroes

Returns:

matrix object

allocateGPUMatrixObject

public MatrixObject allocateGPUMatrixObject(String varName,
                                            long numRows,
                                            long numCols)

Allocates the GPUObject for a given LOPS Variable (eg. _mVar3)

Parameters:

varName - variable name

numRows - number of rows of matrix object

numCols - number of columns of matrix object

Returns:

matrix object

getGPUDensePointerAddress

public long getGPUDensePointerAddress(MatrixObject obj)

getGPUSparsePointerAddress

public CSRPointer getGPUSparsePointerAddress(MatrixObject obj)

getMatrixInputForGPUInstruction

public MatrixObject getMatrixInputForGPUInstruction(String varName,
                                                    String opcode)

releaseMatrixInput

public void releaseMatrixInput(String varName)

Unpins a currently pinned matrix variable and update fine-grained statistics.

Parameters:

varName - variable name

releaseMatrixInput

public void releaseMatrixInput(String... varNames)

releaseMatrixInputForGPUInstruction

public void releaseMatrixInputForGPUInstruction(String varName)

getFrameInput

public FrameBlock getFrameInput(String varName)

Pins a frame variable into memory and returns the internal frame block.

Parameters:

varName - variable name

Returns:

frame block

releaseFrameInput

public void releaseFrameInput(String varName)

Unpins a currently pinned frame variable.

Parameters:

varName - variable name

releaseTensorInput

public void releaseTensorInput(String varName)

releaseTensorInput

public void releaseTensorInput(String... varNames)

getScalarInput

public ScalarObject getScalarInput(CPOperand input)

getScalarInput

public ScalarObject getScalarInput(String name,
                                   Types.ValueType vt,
                                   boolean isLiteral)

setScalarOutput

public void setScalarOutput(String varName,
                            ScalarObject so)

getListObject

public ListObject getListObject(CPOperand input)

getListObject

public ListObject getListObject(String name)

releaseMatrixOutputForGPUInstruction

public void releaseMatrixOutputForGPUInstruction(String varName)

setMatrixOutput

public void setMatrixOutput(String varName,
                            MatrixBlock outputData)

setMatrixOutputAndLineage

public void setMatrixOutputAndLineage(String varName,
                                      MatrixBlock outputData,
                                      LineageItem li)

setMatrixOutput

public void setMatrixOutput(String varName,
                            MatrixBlock outputData,
                            MatrixObject.UpdateType flag)

setTensorOutput

public void setTensorOutput(String varName,
                            TensorBlock outputData)

setFrameOutput

public void setFrameOutput(String varName,
                           FrameBlock outputData)

createCacheableData

public static CacheableData<?> createCacheableData(CacheBlock cb)

createMatrixObject

public static MatrixObject createMatrixObject(MatrixBlock mb)

createFrameObject

public static FrameObject createFrameObject(FrameBlock fb)

getMatrixInputs

public List<MatrixBlock> getMatrixInputs(CPOperand[] inputs)

getMatrixInputs

public List<MatrixBlock> getMatrixInputs(CPOperand[] inputs,
                                         boolean includeList)

getScalarInputs

public List<ScalarObject> getScalarInputs(CPOperand[] inputs)

releaseMatrixInputs

public void releaseMatrixInputs(CPOperand[] inputs)

releaseMatrixInputs

public void releaseMatrixInputs(CPOperand[] inputs,
                                boolean includeList)

pinVariables

public boolean[] pinVariables(List<String> varList)

Pin a given list of variables i.e., set the "clean up" state in corresponding matrix objects, so that the cached data inside these objects is not cleared and the corresponding HDFS files are not deleted (through rmvar instructions). This is necessary for: function input variables, parfor result variables, parfor shared inputs that are passed to functions. The function returns the OLD "clean up" state of matrix objects.

Parameters:

varList - variable list

Returns:

indicator vector of old cleanup state of matrix objects

unpinVariables

public void unpinVariables(List<String> varList,
                           boolean[] varsState)

Unpin the a given list of variables by setting their "cleanup" status to the values specified by varsStats. Typical usage: oldStatus = pinVariables(varList); ... unpinVariables(varList, oldStatus); i.e., a call to unpinVariables() is preceded by pinVariables().

Parameters:

varList - variable list

varsState - variable state

getVarList

public ArrayList<String> getVarList()

NOTE: No order guaranteed, so keep same list for pin and unpin.

Returns:

list of all variable names.

getVarListPartitioned

public ArrayList<String> getVarListPartitioned()

NOTE: No order guaranteed, so keep same list for pin and unpin.

Returns:

list of all variable names of partitioned matrices.

cleanupDataObject

public final void cleanupDataObject(Data dat)

cleanupCacheableData

public void cleanupCacheableData(CacheableData<?> mo)

isFederated

public boolean isFederated(CPOperand input)

isFederated

public boolean isFederated(CPOperand input,
                           FederationMap.FType type)

traceLineage

public void traceLineage(Instruction inst)

maintainLineageDebuggerInfo

public void maintainLineageDebuggerInfo(Instruction inst)

getLineageItem

public LineageItem getLineageItem(CPOperand input)

getOrCreateLineageItem

public LineageItem getOrCreateLineageItem(CPOperand input)

toString

public String toString()

Overrides:

toString in class Object