Package org.apache.sysds.runtime.controlprogram.paramserv.dp

Class Summary

Class	Description
DataPartitionerSparkAggregator
DataPartitionerSparkMapper
DataPartitionLocalScheme
DataPartitionSparkScheme
DCLocalScheme	Disjoint_Contiguous data partitioner: for each worker, use a right indexing operation X[beg:end,] to obtain contiguous, non-overlapping partitions of rows.
DCSparkScheme	Spark Disjoint_Contiguous data partitioner:
DRLocalScheme	Data partitioner Disjoint_Random: for each worker, use a permutation multiply P[beg:end,] %*% X, where P is constructed for example with P=table(seq(1,nrow(X)),sample(nrow(X), nrow(X))), i.e., sampling without replacement to ensure disjointness.
DRRLocalScheme	Disjoint_Round_Robin data partitioner: for each worker, use a permutation multiply or simpler a removeEmpty such as removeEmpty (target=X, margin=rows, select=(seq(1,nrow(X))%%k)==id)
DRRSparkScheme	Spark Disjoint_Round_Robin data partitioner:
DRSparkScheme	Spark data partitioner Disjoint_Random: For the current row block, find all the shifted place for each row (WorkerID => (row block ID, matrix)
LocalDataPartitioner
ORLocalScheme	Data partitioner Overlap_Reshuffle: for each worker, use a new permutation multiply P %*% X, where P is constructed for example with P=table(seq(1,nrow(X),sample(nrow(X), nrow(X))))
ORSparkScheme	Spark data partitioner Overlap_Reshuffle:
SparkDataPartitioner