SystemDSContext¶
All operations using SystemDS need a java instance running.
The connection is ensured by an SystemDSContext
object.
An SystemDSContext
object can be created using:
from systemds.context import SystemDSContext
sds = SystemDSContext()
When the calculations are finished the context has to be closed again:
sds.close()
Since it is annoying that it is always necessary to close the context, SystemDSContext
implements the python context management protocol, which supports the following syntax:
with SystemDSContext() as sds:
# do something with sds which is an SystemDSContext
pass
This will automatically close the SystemDSContext
once the with-block is left.
Note
Creating a context is an expensive procedure, because a sub-process starting a JVM might have to start, therefore try to do this only once for your program, or always leave at least one context open.
-
class
systemds.context.
SystemDSContext
¶ A context with a connection to a java instance with which SystemDS operations are executed. The java process is started and is running using a random tcp port for instruction parsing.
-
__init__
()¶ Starts a new instance of SystemDSContext, in which the connection to a JVM systemds instance is handled Any new instance of this SystemDS Context, would start a separate new JVM.
Standard out and standard error form the JVM is also handled in this class, filling up Queues, that can be read from to get the printed statements from the JVM.
-
close
()¶ Close the connection to the java process and do necessary cleanup.
-
exception_and_close
(e)¶ Method for printing exception, printing stdout and error, while also closing the context correctly.
-
full
(shape: Tuple[int, int], value: Union[float, int]) → systemds.operator.operation_node.OperationNode¶ Generates a matrix completely filled with a value
- Parameters
sds_context – SystemDS context
shape – shape (rows and cols) of the matrix TODO tensor
value – the value to fill all cells with
- Returns
the OperationNode representing this operation
-
get_stderr
(lines: int = - 1)¶ Getter for the stderr of the java subprocess The output is taken from the stderr queue and returned in a new list. :param lines: The number of lines to try to read from the stderr queue. default -1 prints all current lines in the queue.
-
get_stdout
(lines: int = - 1)¶ Getter for the stdout of the java subprocess The output is taken from the stdout queue and returned in a new list. :param lines: The number of lines to try to read from the stdout queue. default -1 prints all current lines in the queue.
-
rand
(rows: int, cols: int, min: Union[float, int] = None, max: Union[float, int] = None, pdf: str = 'uniform', sparsity: Union[float, int] = None, seed: Union[float, int] = None, lambd: Union[float, int] = 1) → systemds.operator.operation_node.OperationNode¶ Generates a matrix filled with random values
- Parameters
sds_context – SystemDS context
rows – number of rows
cols – number of cols
min – min value for cells
max – max value for cells
pdf – “uniform”/”normal”/”poison” distribution
sparsity – fraction of non-zero cells
seed – random seed
lambd – lamda value for “poison” distribution
- Returns
-
read
(path: os.PathLike, **kwargs: Dict[str, Union[DAGNode, str, int, float, bool]]) → OperationNode¶ Read an file from disk. Supportted types include: CSV, Matrix Market(coordinate), Text(i,j,v), SystemDS Binay See: http://apache.github.io/systemds/site/dml-language-reference#readwrite-built-in-functions for more details :return: an Operation Node, containing the read data.
-
scalar
(v: Dict[str, Union[DAGNode, str, int, float, bool]]) → OperationNode¶ Construct an scalar value, this can contain str, float, double, integers and booleans. :return: An OperationNode containing the scalar value.
-
seq
(start: Union[float, int], stop: Union[float, int] = None, step: Union[float, int] = 1) → systemds.operator.operation_node.OperationNode¶ Create a single column vector with values from start to stop and an increment of step. If no stop is defined and only one parameter is given, then start will be 0 and the parameter will be interpreted as stop.
- Parameters
sds_context – SystemDS context
start – the starting value
stop – the maximum value
step – the step size
- Returns
the OperationNode representing this operation
-