Uses of Class
org.apache.sysds.runtime.controlprogram.context.SparkExecutionContext

Packages that use SparkExecutionContext

Package	Description
org.apache.sysds.api.mlcontext
org.apache.sysds.runtime.controlprogram.paramserv
org.apache.sysds.runtime.controlprogram.paramserv.dp
org.apache.sysds.runtime.instructions.spark
org.apache.sysds.runtime.instructions.spark.utils

Uses of SparkExecutionContext in org.apache.sysds.api.mlcontext

Methods in org.apache.sysds.api.mlcontext with parameters of type SparkExecutionContext

Modifier and Type	Method and Description
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>	MLContextConversionUtil.frameObjectToBinaryBlocks(FrameObject frameObject, SparkExecutionContext sparkExecutionContext) Convert a FrameObject to a JavaPairRDD<Long, FrameBlock>.
static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	MLContextConversionUtil.frameObjectToDataFrame(FrameObject frameObject, SparkExecutionContext sparkExecutionContext) Convert a FrameObject to a DataFrame.
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>	MLContextConversionUtil.matrixObjectToBinaryBlocks(MatrixObject matrixObject, SparkExecutionContext sparkExecutionContext) Convert a MatrixObject to a JavaPairRDD<MatrixIndexes, MatrixBlock>.
static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	MLContextConversionUtil.matrixObjectToDataFrame(MatrixObject matrixObject, SparkExecutionContext sparkExecutionContext, boolean isVectorDF) Convert a MatrixObject to a DataFrame.

Constructors in org.apache.sysds.api.mlcontext with parameters of type SparkExecutionContext

Constructor and Description
Frame(FrameObject frameObject, SparkExecutionContext sparkExecutionContext)
Matrix(MatrixObject matrixObject, SparkExecutionContext sparkExecutionContext)

Uses of SparkExecutionContext in org.apache.sysds.runtime.controlprogram.paramserv

Methods in org.apache.sysds.runtime.controlprogram.paramserv with parameters of type SparkExecutionContext

Modifier and Type	Method and Description
static org.apache.spark.api.java.JavaPairRDD<Integer,scala.Tuple2<MatrixBlock,MatrixBlock>>	SparkParamservUtils.doPartitionOnSpark(SparkExecutionContext sec, MatrixObject features, MatrixObject labels, Statement.PSScheme scheme, int workerNum)

Uses of SparkExecutionContext in org.apache.sysds.runtime.controlprogram.paramserv.dp

Constructors in org.apache.sysds.runtime.controlprogram.paramserv.dp with parameters of type SparkExecutionContext

Constructor and Description
DataPartitionerSparkMapper(Statement.PSScheme scheme, int workersNum, SparkExecutionContext sec, int numEntries)

Uses of SparkExecutionContext in org.apache.sysds.runtime.instructions.spark

Methods in org.apache.sysds.runtime.instructions.spark with parameters of type SparkExecutionContext

Modifier and Type	Method and Description
void	UnaryFrameSPInstruction.detectSchema(SparkExecutionContext sec)

Uses of SparkExecutionContext in org.apache.sysds.runtime.instructions.spark.utils

Methods in org.apache.sysds.runtime.instructions.spark.utils with parameters of type SparkExecutionContext

Modifier and Type	Method and Description
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>	RDDSortUtils.sortDataByValMemSort(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> val, org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> data, boolean asc, long rlen, long clen, int blen, SparkExecutionContext sec, ReorgOperator r_op) This function collects and sorts value column in memory and then broadcasts it.