| Modifier and Type | Field and Description |
|---|---|
MatrixIndexes |
MLBlock.indexes |
| Modifier and Type | Method and Description |
|---|---|
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
MLOutput.getBinaryBlockedRDD(String varName) |
| Modifier and Type | Method and Description |
|---|---|
void |
MLContext.registerInput(String varName,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> rdd,
long rlen,
long clen)
Register binary blocked RDD with given dimensions, default block sizes and no nnz
|
void |
MLContext.registerInput(String varName,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> rdd,
long rlen,
long clen,
int brlen,
int bclen)
Register binary blocked RDD with given dimensions, given block sizes and no nnz
|
void |
MLContext.registerInput(String varName,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> rdd,
long rlen,
long clen,
int brlen,
int bclen,
long nnz)
Register binary blocked RDD with given dimensions, given block sizes and given nnz (preferred).
|
void |
MLContext.registerInput(String varName,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> rdd,
MatrixCharacteristics mc) |
| Constructor and Description |
|---|
MLBlock(MatrixIndexes indexes,
MatrixBlock block) |
| Modifier and Type | Method and Description |
|---|---|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
MLContextConversionUtil.dataFrameToMatrixBinaryBlocks(org.apache.spark.sql.DataFrame dataFrame)
Convert a
DataFrame to a
JavaPairRDD<MatrixIndexes, MatrixBlock> binary-block matrix. |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
MLContextConversionUtil.dataFrameToMatrixBinaryBlocks(org.apache.spark.sql.DataFrame dataFrame,
MatrixMetadata matrixMetadata)
Convert a
DataFrame to a
JavaPairRDD<MatrixIndexes, MatrixBlock> binary-block matrix. |
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
BinaryBlockMatrix.getBinaryBlocks()
Obtain a SystemML binary-block matrix as a
JavaPairRDD<MatrixIndexes, MatrixBlock> |
| Modifier and Type | Method and Description |
|---|---|
static org.apache.sysml.runtime.controlprogram.caching.MatrixObject |
MLContextConversionUtil.binaryBlocksToMatrixObject(String variableName,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> binaryBlocks)
Convert a
JavaPairRDD<MatrixIndexes, MatrixBlock> to a
MatrixObject. |
static org.apache.sysml.runtime.controlprogram.caching.MatrixObject |
MLContextConversionUtil.binaryBlocksToMatrixObject(String variableName,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> binaryBlocks,
MatrixMetadata matrixMetadata)
Convert a
JavaPairRDD<MatrixIndexes, MatrixBlock> to a
MatrixObject. |
void |
BinaryBlockMatrix.setBinaryBlocks(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> binaryBlocks)
Set the SystemML binary-block matrix as a
JavaPairRDD<MatrixIndexes, MatrixBlock> |
| Constructor and Description |
|---|
BinaryBlockMatrix(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> binaryBlocks,
MatrixCharacteristics matrixCharacteristics)
Create a BinaryBlockMatrix, specifying the SystemML binary-block matrix
and its metadata.
|
| Modifier and Type | Method and Description |
|---|---|
MatrixValue |
CompressedMatrixBlock.aggregateBinaryOperations(MatrixIndexes m1Index,
MatrixValue m1Value,
MatrixIndexes m2Index,
MatrixValue m2Value,
MatrixValue result,
org.apache.sysml.runtime.matrix.operators.AggregateBinaryOperator op) |
MatrixValue |
CompressedMatrixBlock.aggregateUnaryOperations(org.apache.sysml.runtime.matrix.operators.AggregateUnaryOperator op,
MatrixValue result,
int blockingFactorRow,
int blockingFactorCol,
MatrixIndexes indexesIn) |
MatrixValue |
CompressedMatrixBlock.aggregateUnaryOperations(org.apache.sysml.runtime.matrix.operators.AggregateUnaryOperator op,
MatrixValue result,
int blockingFactorRow,
int blockingFactorCol,
MatrixIndexes indexesIn,
boolean inCP) |
void |
CompressedMatrixBlock.ternaryOperations(org.apache.sysml.runtime.matrix.operators.Operator op,
MatrixIndexes ix1,
double scalar,
boolean left,
int brlen,
CTableMap resultMap,
MatrixBlock resultBlock) |
| Modifier and Type | Method and Description |
|---|---|
Iterable<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
ParameterizedBuiltinSPInstruction.RDDRemoveEmptyFunctionInMem.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
Iterable<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
ParameterizedBuiltinSPInstruction.RDDRExpandFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
Iterable<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
ParameterizedBuiltinSPInstruction.RDDMapGroupedAggFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
scala.Tuple2<MatrixIndexes,MatrixBlock> |
ParameterizedBuiltinSPInstruction.RDDTransformDecodeExpandFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> in) |
Iterable<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
AppendGSPInstruction.ShiftMatrix.call(scala.Tuple2<MatrixIndexes,MatrixBlock> kv) |
scala.Tuple2<MatrixIndexes,MatrixBlock> |
AppendGAlignedSPInstruction.ShiftColumnIndex.call(scala.Tuple2<MatrixIndexes,MatrixBlock> kv) |
scala.Tuple2<MatrixIndexes,MatrixBlock> |
AppendGSPInstruction.MergeWithShiftedBlocks.call(scala.Tuple2<MatrixIndexes,scala.Tuple2<Iterable<MatrixBlock>,Iterable<MatrixBlock>>> kv) |
Iterable<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
ParameterizedBuiltinSPInstruction.RDDRemoveEmptyFunction.call(scala.Tuple2<MatrixIndexes,scala.Tuple2<MatrixBlock,MatrixBlock>> arg0) |
protected org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
CSVReblockSPInstruction.processMatrixCSVReblockInstruction(org.apache.sysml.runtime.controlprogram.context.SparkExecutionContext sec,
MatrixCharacteristics mcOut) |
| Modifier and Type | Method and Description |
|---|---|
Iterable<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
ParameterizedBuiltinSPInstruction.RDDRemoveEmptyFunctionInMem.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
Iterable<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
ParameterizedBuiltinSPInstruction.RDDRExpandFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
Iterable<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
ParameterizedBuiltinSPInstruction.RDDMapGroupedAggFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
MatrixBlock |
ParameterizedBuiltinSPInstruction.RDDMapGroupedAggFunction2.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
scala.Tuple2<Long,FrameBlock> |
ParameterizedBuiltinSPInstruction.RDDTransformDecodeFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> in) |
scala.Tuple2<MatrixIndexes,MatrixBlock> |
ParameterizedBuiltinSPInstruction.RDDTransformDecodeExpandFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> in) |
Iterable<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
AppendGSPInstruction.ShiftMatrix.call(scala.Tuple2<MatrixIndexes,MatrixBlock> kv) |
scala.Tuple2<MatrixIndexes,MatrixBlock> |
AppendGAlignedSPInstruction.ShiftColumnIndex.call(scala.Tuple2<MatrixIndexes,MatrixBlock> kv) |
scala.Tuple2<MatrixIndexes,MatrixBlock> |
AppendGSPInstruction.MergeWithShiftedBlocks.call(scala.Tuple2<MatrixIndexes,scala.Tuple2<Iterable<MatrixBlock>,Iterable<MatrixBlock>>> kv) |
Iterable<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
ParameterizedBuiltinSPInstruction.RDDRemoveEmptyFunction.call(scala.Tuple2<MatrixIndexes,scala.Tuple2<MatrixBlock,MatrixBlock>> arg0) |
void |
ParameterizedBuiltinSPInstruction.setOutputCharacteristicsForGroupedAgg(MatrixCharacteristics mc1,
MatrixCharacteristics mcOut,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixCell> out) |
| Modifier and Type | Method and Description |
|---|---|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.aggByKeyStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
org.apache.sysml.runtime.matrix.operators.AggregateOperator aop) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
FrameRDDConverterUtils.binaryBlockToMatrixBlock(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> input,
MatrixCharacteristics mcIn,
MatrixCharacteristics mcOut) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDConverterUtils.binaryCellToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixCell> input,
MatrixCharacteristics mcOut,
boolean outputEmptyBlocks) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixCell> |
SparkUtils.cacheBinaryCellRDD(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixCell> input) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDConverterUtilsExt.coordinateMatrixToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.mllib.linalg.distributed.CoordinateMatrix input,
MatrixCharacteristics mcIn,
boolean outputEmptyBlocks)
Example usage:
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDConverterUtilsExt.coordinateMatrixToBinaryBlock(org.apache.spark.SparkContext sc,
org.apache.spark.mllib.linalg.distributed.CoordinateMatrix input,
MatrixCharacteristics mcIn,
boolean outputEmptyBlocks) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
SparkUtils.copyBinaryBlockMatrix(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in)
Creates a partitioning-preserving deep copy of the input matrix RDD, where
the indexes and values are copied.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
SparkUtils.copyBinaryBlockMatrix(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
boolean deep)
Creates a partitioning-preserving copy of the input matrix RDD.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDConverterUtils.csvToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> input,
MatrixCharacteristics mc,
boolean hasHeader,
String delim,
boolean fill,
double fillValue) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDConverterUtils.csvToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.api.java.JavaRDD<String> input,
MatrixCharacteristics mcOut,
boolean hasHeader,
String delim,
boolean fill,
double fillValue)
Example usage:
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDConverterUtils.dataFrameToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.sql.DataFrame df,
MatrixCharacteristics mc,
boolean containsID,
boolean isVector) |
static ArrayList<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
SparkUtils.fromIndexedMatrixBlock(ArrayList<org.apache.sysml.runtime.matrix.mapred.IndexedMatrixValue> in) |
static scala.Tuple2<MatrixIndexes,MatrixBlock> |
SparkUtils.fromIndexedMatrixBlock(org.apache.sysml.runtime.matrix.mapred.IndexedMatrixValue in) |
static ArrayList<Pair<MatrixIndexes,MatrixBlock>> |
SparkUtils.fromIndexedMatrixBlockToPair(ArrayList<org.apache.sysml.runtime.matrix.mapred.IndexedMatrixValue> in) |
static Pair<MatrixIndexes,MatrixBlock> |
SparkUtils.fromIndexedMatrixBlockToPair(org.apache.sysml.runtime.matrix.mapred.IndexedMatrixValue in) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
SparkUtils.getEmptyBlockRDD(org.apache.spark.api.java.JavaSparkContext sc,
MatrixCharacteristics mc) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.mergeByKey(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in)
Merges disjoint data of all blocks per key.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.mergeRowsByKey(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,org.apache.sysml.runtime.instructions.spark.data.RowMatrixBlock> in)
Merges disjoint data of all blocks per key.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortByVal(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in2,
long rlen,
int brlen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortByVal(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
long rlen,
int brlen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortDataByVal(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> val,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> data,
boolean asc,
long rlen,
long clen,
int brlen,
int bclen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortDataByValMemSort(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> val,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> data,
boolean asc,
long rlen,
long clen,
int brlen,
int bclen,
org.apache.sysml.runtime.controlprogram.context.SparkExecutionContext sec,
org.apache.sysml.runtime.matrix.operators.ReorgOperator r_op)
This function collects and sorts value column in memory and then broadcasts it.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortIndexesByVal(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> val,
boolean asc,
long rlen,
int brlen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.sumByKeyStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,Double> |
RDDAggregateUtils.sumCellsByKeyStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,Double> in) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDConverterUtils.textCellToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> input,
MatrixCharacteristics mcOut,
boolean outputEmptyBlocks) |
| Modifier and Type | Method and Description |
|---|---|
static org.apache.sysml.runtime.matrix.mapred.IndexedMatrixValue |
SparkUtils.toIndexedMatrixBlock(MatrixIndexes ix,
MatrixBlock mb) |
| Modifier and Type | Method and Description |
|---|---|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.aggByKeyStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
org.apache.sysml.runtime.matrix.operators.AggregateOperator aop) |
static MatrixBlock |
RDDAggregateUtils.aggStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
org.apache.sysml.runtime.matrix.operators.AggregateOperator aop)
Single block aggregation over pair rdds with corrections for numerical stability.
|
static org.apache.spark.api.java.JavaRDD<String> |
RDDConverterUtils.binaryBlockToCsv(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
MatrixCharacteristics mcIn,
CSVFileFormatProperties props,
boolean strict) |
static org.apache.spark.sql.DataFrame |
RDDConverterUtils.binaryBlockToDataFrame(org.apache.spark.sql.SQLContext sqlctx,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
MatrixCharacteristics mc,
boolean toVector) |
static org.apache.spark.api.java.JavaRDD<org.apache.spark.mllib.regression.LabeledPoint> |
RDDConverterUtils.binaryBlockToLabeledPoints(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in)
Converter from binary block rdd to rdd of labeled points.
|
static org.apache.spark.api.java.JavaRDD<String> |
RDDConverterUtils.binaryBlockToTextCell(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
MatrixCharacteristics mc) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDConverterUtils.binaryCellToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixCell> input,
MatrixCharacteristics mcOut,
boolean outputEmptyBlocks) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixCell> |
SparkUtils.cacheBinaryCellRDD(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixCell> input) |
static MatrixCharacteristics |
SparkUtils.computeMatrixCharacteristics(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixCell> input)
Utility to compute dimensions and non-zeros in a given RDD of binary cells.
|
static long |
SparkUtils.computeNNZFromBlocks(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> rdd)
Utility to compute number of non-zeros from the given RDD of MatrixBlocks
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
SparkUtils.copyBinaryBlockMatrix(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in)
Creates a partitioning-preserving deep copy of the input matrix RDD, where
the indexes and values are copied.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
SparkUtils.copyBinaryBlockMatrix(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
boolean deep)
Creates a partitioning-preserving copy of the input matrix RDD.
|
static org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.LongWritable,FrameBlock> |
FrameRDDConverterUtils.matrixBlockToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> input,
MatrixCharacteristics mcIn) |
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> |
FrameRDDConverterUtils.matrixBlockToBinaryBlockLongIndex(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> input,
MatrixCharacteristics mcIn) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.mergeByKey(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in)
Merges disjoint data of all blocks per key.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.mergeRowsByKey(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,org.apache.sysml.runtime.instructions.spark.data.RowMatrixBlock> in)
Merges disjoint data of all blocks per key.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortByVal(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in2,
long rlen,
int brlen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortByVal(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in2,
long rlen,
int brlen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortByVal(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
long rlen,
int brlen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortDataByVal(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> val,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> data,
boolean asc,
long rlen,
long clen,
int brlen,
int bclen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortDataByVal(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> val,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> data,
boolean asc,
long rlen,
long clen,
int brlen,
int bclen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortDataByValMemSort(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> val,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> data,
boolean asc,
long rlen,
long clen,
int brlen,
int bclen,
org.apache.sysml.runtime.controlprogram.context.SparkExecutionContext sec,
org.apache.sysml.runtime.matrix.operators.ReorgOperator r_op)
This function collects and sorts value column in memory and then broadcasts it.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortDataByValMemSort(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> val,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> data,
boolean asc,
long rlen,
long clen,
int brlen,
int bclen,
org.apache.sysml.runtime.controlprogram.context.SparkExecutionContext sec,
org.apache.sysml.runtime.matrix.operators.ReorgOperator r_op)
This function collects and sorts value column in memory and then broadcasts it.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortIndexesByVal(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> val,
boolean asc,
long rlen,
int brlen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.sumByKeyStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,Double> |
RDDAggregateUtils.sumCellsByKeyStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,Double> in) |
static MatrixBlock |
RDDAggregateUtils.sumStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in) |
static org.apache.sysml.runtime.matrix.mapred.IndexedMatrixValue |
SparkUtils.toIndexedMatrixBlock(scala.Tuple2<MatrixIndexes,MatrixBlock> in) |
| Modifier and Type | Method and Description |
|---|---|
void |
CombineMR.InnerReducer.reduce(MatrixIndexes indexes,
Iterator<TaggedMatrixValue> values,
org.apache.hadoop.mapred.OutputCollector<MatrixIndexes,WeightedPair> out,
org.apache.hadoop.mapred.Reporter reporter) |
| Modifier and Type | Method and Description |
|---|---|
void |
CombineMR.InnerReducer.reduce(MatrixIndexes indexes,
Iterator<TaggedMatrixValue> values,
org.apache.hadoop.mapred.OutputCollector<MatrixIndexes,WeightedPair> out,
org.apache.hadoop.mapred.Reporter reporter) |
| Modifier and Type | Method and Description |
|---|---|
Pair<MatrixIndexes,MatrixCell> |
TextToBinaryCellConverter.next() |
Pair<MatrixIndexes,MatrixBlock> |
TextCellToRowBlockConverter.next() |
Pair<MatrixIndexes,MatrixBlock> |
BinaryCellToRowBlockConverter.next() |
Pair<MatrixIndexes,MatrixBlock> |
BinaryBlockToRowBlockConverter.next() |
Pair<MatrixIndexes,MatrixCell> |
BinaryBlockToBinaryCellConverter.next() |
Pair<MatrixIndexes,WeightedPair> |
AddDummyWeightConverter.next() |
| Modifier and Type | Method and Description |
|---|---|
abstract MatrixValue |
MatrixValue.aggregateBinaryOperations(MatrixIndexes m1Index,
MatrixValue m1Value,
MatrixIndexes m2Index,
MatrixValue m2Value,
MatrixValue result,
org.apache.sysml.runtime.matrix.operators.AggregateBinaryOperator op) |
MatrixValue |
MatrixCell.aggregateBinaryOperations(MatrixIndexes m1Index,
MatrixValue m1Value,
MatrixIndexes m2Index,
MatrixValue m2Value,
MatrixValue result,
org.apache.sysml.runtime.matrix.operators.AggregateBinaryOperator op) |
MatrixValue |
MatrixBlock.aggregateBinaryOperations(MatrixIndexes m1Index,
MatrixValue m1Value,
MatrixIndexes m2Index,
MatrixValue m2Value,
MatrixValue result,
org.apache.sysml.runtime.matrix.operators.AggregateBinaryOperator op) |
MatrixValue |
CM_N_COVCell.aggregateBinaryOperations(MatrixIndexes m1Index,
MatrixValue m1Value,
MatrixIndexes m2Index,
MatrixValue m2Value,
MatrixValue result,
org.apache.sysml.runtime.matrix.operators.AggregateBinaryOperator op) |
MatrixValue |
WeightedCell.aggregateUnaryOperations(org.apache.sysml.runtime.matrix.operators.AggregateUnaryOperator op,
MatrixValue result,
int brlen,
int bclen,
MatrixIndexes indexesIn) |
abstract MatrixValue |
MatrixValue.aggregateUnaryOperations(org.apache.sysml.runtime.matrix.operators.AggregateUnaryOperator op,
MatrixValue result,
int brlen,
int bclen,
MatrixIndexes indexesIn) |
MatrixValue |
MatrixCell.aggregateUnaryOperations(org.apache.sysml.runtime.matrix.operators.AggregateUnaryOperator op,
MatrixValue result,
int brlen,
int bclen,
MatrixIndexes indexesIn) |
MatrixValue |
MatrixBlock.aggregateUnaryOperations(org.apache.sysml.runtime.matrix.operators.AggregateUnaryOperator op,
MatrixValue result,
int blockingFactorRow,
int blockingFactorCol,
MatrixIndexes indexesIn) |
MatrixValue |
CM_N_COVCell.aggregateUnaryOperations(org.apache.sysml.runtime.matrix.operators.AggregateUnaryOperator op,
MatrixValue result,
int brlen,
int bclen,
MatrixIndexes indexesIn) |
abstract MatrixValue |
MatrixValue.aggregateUnaryOperations(org.apache.sysml.runtime.matrix.operators.AggregateUnaryOperator op,
MatrixValue result,
int blockingFactorRow,
int blockingFactorCol,
MatrixIndexes indexesIn,
boolean inCP) |
MatrixValue |
MatrixCell.aggregateUnaryOperations(org.apache.sysml.runtime.matrix.operators.AggregateUnaryOperator op,
MatrixValue result,
int blockingFactorRow,
int blockingFactorCol,
MatrixIndexes indexesIn,
boolean inCP) |
MatrixValue |
MatrixBlock.aggregateUnaryOperations(org.apache.sysml.runtime.matrix.operators.AggregateUnaryOperator op,
MatrixValue result,
int blockingFactorRow,
int blockingFactorCol,
MatrixIndexes indexesIn,
boolean inCP) |
MatrixValue |
CM_N_COVCell.aggregateUnaryOperations(org.apache.sysml.runtime.matrix.operators.AggregateUnaryOperator op,
MatrixValue result,
int blockingFactorRow,
int blockingFactorCol,
MatrixIndexes indexesIn,
boolean inCP) |
int |
MatrixIndexes.compare(MatrixIndexes m1,
MatrixIndexes m2) |
int |
MatrixIndexes.compareTo(MatrixIndexes other) |
void |
BinaryBlockToTextCellConverter.convert(MatrixIndexes k1,
MatrixBlock v1)
Before calling convert, please make sure to setBlockSize(brlen, bclen);
|
void |
BinaryBlockToRowBlockConverter.convert(MatrixIndexes k1,
MatrixBlock v1) |
void |
BinaryBlockToBinaryCellConverter.convert(MatrixIndexes k1,
MatrixBlock v1) |
void |
BinaryCellToTextConverter.convert(MatrixIndexes k1,
MatrixCell v1) |
void |
BinaryCellToRowBlockConverter.convert(MatrixIndexes k1,
MatrixCell v1) |
void |
WeightedCellToSortInputConverter.convert(MatrixIndexes k1,
WeightedCell v1) |
static void |
OperationsOnMatrixValues.performAggregateBinary(MatrixIndexes indexes1,
MatrixValue value1,
MatrixIndexes indexes2,
MatrixValue value2,
MatrixIndexes indexesOut,
MatrixValue valueOut,
org.apache.sysml.runtime.matrix.operators.AggregateBinaryOperator op) |
static void |
OperationsOnMatrixValues.performAggregateUnary(MatrixIndexes indexesIn,
MatrixValue valueIn,
MatrixIndexes indexesOut,
MatrixValue valueOut,
org.apache.sysml.runtime.matrix.operators.AggregateUnaryOperator op,
int brlen,
int bclen) |
static void |
OperationsOnMatrixValues.performReorg(MatrixIndexes indexesIn,
MatrixValue valueIn,
MatrixIndexes indexesOut,
MatrixValue valueOut,
org.apache.sysml.runtime.matrix.operators.ReorgOperator op,
int startRow,
int startColumn,
int length) |
static void |
OperationsOnMatrixValues.performTernary(MatrixIndexes indexesIn1,
MatrixValue valueIn1,
double scalarIn2,
boolean left,
int brlen,
CTableMap resultMap,
MatrixBlock resultBlock,
org.apache.sysml.runtime.matrix.operators.Operator op) |
static void |
OperationsOnMatrixValues.performTernary(MatrixIndexes indexesIn1,
MatrixValue valueIn1,
double scalarIn2,
double scalarIn3,
CTableMap resultMap,
MatrixBlock resultBlock,
org.apache.sysml.runtime.matrix.operators.Operator op) |
static void |
OperationsOnMatrixValues.performTernary(MatrixIndexes indexesIn1,
MatrixValue valueIn1,
double scalarIn2,
MatrixIndexes indexesIn3,
MatrixValue valueIn3,
CTableMap resultMap,
MatrixBlock resultBlock,
org.apache.sysml.runtime.matrix.operators.Operator op) |
static void |
OperationsOnMatrixValues.performTernary(MatrixIndexes indexesIn1,
MatrixValue valueIn1,
MatrixIndexes indexesIn2,
MatrixValue valueIn2,
double scalarIn3,
CTableMap resultMap,
MatrixBlock resultBlock,
org.apache.sysml.runtime.matrix.operators.Operator op) |
static void |
OperationsOnMatrixValues.performTernary(MatrixIndexes indexesIn1,
MatrixValue valueIn1,
MatrixIndexes indexesIn2,
MatrixValue valueIn2,
MatrixIndexes indexesIn3,
MatrixValue valueIn3,
CTableMap resultMap,
MatrixBlock resultBlock,
org.apache.sysml.runtime.matrix.operators.Operator op) |
static void |
OperationsOnMatrixValues.performZeroOut(MatrixIndexes indexesIn,
MatrixValue valueIn,
MatrixIndexes indexesOut,
MatrixValue valueOut,
org.apache.sysml.runtime.util.IndexRange range,
boolean complementary) |
static void |
LibMatrixAgg.recomputeIndexes(MatrixBlock out,
org.apache.sysml.runtime.matrix.operators.AggregateUnaryOperator op,
int brlen,
int bclen,
MatrixIndexes ix)
Recompute outputs (e.g., maxindex or minindex) according to block indexes from MR.
|
static void |
LibMatrixOuterAgg.resetOutputMatix(MatrixIndexes in1Ix,
MatrixBlock in1Val,
MatrixIndexes outIx,
MatrixBlock outVal,
org.apache.sysml.runtime.matrix.operators.AggregateUnaryOperator uaggOp)
ReSet output matrix
|
void |
MatrixIndexes.setIndexes(MatrixIndexes that) |
abstract void |
MatrixValue.ternaryOperations(org.apache.sysml.runtime.matrix.operators.Operator op,
MatrixIndexes ix1,
double scalar_that,
boolean left,
int brlen,
CTableMap resultMap,
MatrixBlock resultBlock) |
void |
MatrixCell.ternaryOperations(org.apache.sysml.runtime.matrix.operators.Operator op,
MatrixIndexes ix1,
double scalarThat,
boolean left,
int brlen,
CTableMap resultMap,
MatrixBlock resultBlock) |
void |
MatrixBlock.ternaryOperations(org.apache.sysml.runtime.matrix.operators.Operator op,
MatrixIndexes ix1,
double scalarThat,
boolean left,
int brlen,
CTableMap resultMap,
MatrixBlock resultBlock)
Specific ctable case of ctable(seq(...),X), where X is the only
matrix input.
|
void |
CM_N_COVCell.ternaryOperations(org.apache.sysml.runtime.matrix.operators.Operator op,
MatrixIndexes ix1,
double scalarThat,
boolean left,
int brlen,
CTableMap resultMap,
MatrixBlock resultBlock) |
| Constructor and Description |
|---|
MatrixIndexes(MatrixIndexes indexes) |
TaggedMatrixIndexes(MatrixIndexes ix,
byte t) |
Copyright © 2017 The Apache Software Foundation. All rights reserved.