Modifier and Type | Method and Description |
---|---|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
MLContextConversionUtil.dataFrameToMatrixBinaryBlocks(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame)
Convert a
DataFrame to a
JavaPairRDD<MatrixIndexes, MatrixBlock> binary-block matrix. |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
MLContextConversionUtil.dataFrameToMatrixBinaryBlocks(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame,
MatrixMetadata matrixMetadata)
Convert a
DataFrame to a
JavaPairRDD<MatrixIndexes, MatrixBlock> binary-block matrix. |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
MLContextConversionUtil.matrixObjectToBinaryBlocks(MatrixObject matrixObject,
SparkExecutionContext sparkExecutionContext)
Convert a
MatrixObject to a
JavaPairRDD<MatrixIndexes, MatrixBlock> . |
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
Matrix.toBinaryBlocks()
Obtain the matrix as a
JavaPairRDD<MatrixIndexes, MatrixBlock> |
Modifier and Type | Method and Description |
---|---|
static MatrixBlock |
MLContextConversionUtil.binaryBlocksToMatrixBlock(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> binaryBlocks,
MatrixMetadata matrixMetadata)
Convert a
JavaPairRDD<MatrixIndexes, MatrixBlock> to a
MatrixBlock |
static MatrixObject |
MLContextConversionUtil.binaryBlocksToMatrixObject(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> binaryBlocks)
Convert a
JavaPairRDD<MatrixIndexes, MatrixBlock> to a
MatrixObject . |
static MatrixObject |
MLContextConversionUtil.binaryBlocksToMatrixObject(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> binaryBlocks,
MatrixMetadata matrixMetadata)
Convert a
JavaPairRDD<MatrixIndexes, MatrixBlock> to a
MatrixObject . |
Constructor and Description |
---|
Matrix(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> binaryBlocks,
MatrixMetadata matrixMetadata)
Create a Matrix, specifying the SystemDS binary-block matrix and its
metadata.
|
Modifier and Type | Method and Description |
---|---|
MatrixBlock |
AbstractCompressedMatrixBlock.aggregateBinaryOperations(MatrixIndexes m1Index,
MatrixBlock m1Value,
MatrixIndexes m2Index,
MatrixBlock m2Value,
MatrixBlock result,
AggregateBinaryOperator op) |
MatrixBlock |
CompressedMatrixBlock.aggregateUnaryOperations(AggregateUnaryOperator op,
MatrixValue result,
int blen,
MatrixIndexes indexesIn) |
MatrixBlock |
CompressedMatrixBlock.aggregateUnaryOperations(AggregateUnaryOperator op,
MatrixValue result,
int blen,
MatrixIndexes indexesIn,
boolean inCP) |
void |
AbstractCompressedMatrixBlock.ctableOperations(Operator op,
MatrixIndexes ix1,
double scalar,
boolean left,
int brlen,
CTableMap resultMap,
MatrixBlock resultBlock) |
Modifier and Type | Method and Description |
---|---|
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
SparkExecutionContext.getBinaryMatrixBlockRDDHandleForVariable(String varname)
Spark instructions should call this for all matrix inputs except broadcast
variables.
|
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
SparkExecutionContext.getBinaryMatrixBlockRDDHandleForVariable(String varname,
int numParts,
boolean inclEmpty) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
SparkExecutionContext.toMatrixJavaPairRDD(org.apache.spark.api.java.JavaSparkContext sc,
MatrixBlock src,
int blen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
SparkExecutionContext.toMatrixJavaPairRDD(org.apache.spark.api.java.JavaSparkContext sc,
MatrixBlock src,
int blen,
int numParts,
boolean inclEmpty) |
Modifier and Type | Method and Description |
---|---|
static MatrixBlock |
SparkExecutionContext.toMatrixBlock(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> rdd,
int rlen,
int clen,
int blen,
long nnz)
Utility method for creating a single matrix block out of a binary block RDD.
|
static MatrixBlock |
SparkExecutionContext.toMatrixBlock(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixCell> rdd,
int rlen,
int clen,
long nnz)
Utility method for creating a single matrix block out of a binary cell RDD.
|
static PartitionedBlock<MatrixBlock> |
SparkExecutionContext.toPartitionedMatrixBlock(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> rdd,
int rlen,
int clen,
int blen,
long nnz) |
Modifier and Type | Method and Description |
---|---|
static org.apache.spark.api.java.JavaPairRDD<Long,scala.Tuple2<MatrixBlock,MatrixBlock>> |
ParamservUtils.assembleTrainingData(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> featuresRDD,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> labelsRDD)
Assemble the matrix of features and labels according to the rowID
|
static org.apache.spark.api.java.JavaPairRDD<Long,scala.Tuple2<MatrixBlock,MatrixBlock>> |
ParamservUtils.assembleTrainingData(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> featuresRDD,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> labelsRDD)
Assemble the matrix of features and labels according to the rowID
|
Modifier and Type | Method and Description |
---|---|
MatrixIndexes |
ResultMergeTaggedMatrixIndexes.getIndexes() |
Modifier and Type | Method and Description |
---|---|
scala.Tuple2<MatrixIndexes,MatrixBlock> |
ResultMergeRemoteSparkWCompare.call(scala.Tuple2<MatrixIndexes,scala.Tuple2<Iterable<MatrixBlock>,MatrixBlock>> arg) |
Modifier and Type | Method and Description |
---|---|
Iterator<scala.Tuple2<Long,org.apache.hadoop.io.Writable>> |
DataPartitionerRemoteSparkMapper.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
scala.Tuple2<MatrixIndexes,MatrixBlock> |
ResultMergeRemoteSparkWCompare.call(scala.Tuple2<MatrixIndexes,scala.Tuple2<Iterable<MatrixBlock>,MatrixBlock>> arg) |
Modifier and Type | Field and Description |
---|---|
MatrixIndexes |
PairWritableCell.indexes |
MatrixIndexes |
PairWritableBlock.indexes |
Constructor and Description |
---|
PairWritableBlock(MatrixIndexes ix,
MatrixBlock mb) |
Modifier and Type | Method and Description |
---|---|
Pair<MatrixIndexes,Double> |
CTable.execute(long row,
double v2,
double w) |
Modifier and Type | Method and Description |
---|---|
void |
ReduceAll.execute(MatrixIndexes in,
MatrixIndexes out) |
void |
SortIndex.execute(MatrixIndexes in,
MatrixIndexes out) |
void |
RevIndex.execute(MatrixIndexes in,
MatrixIndexes out) |
void |
OffsetColumnIndex.execute(MatrixIndexes in,
MatrixIndexes out) |
void |
DiagIndex.execute(MatrixIndexes in,
MatrixIndexes out) |
void |
SwapIndex.execute(MatrixIndexes in,
MatrixIndexes out) |
void |
ReduceRow.execute(MatrixIndexes in,
MatrixIndexes out) |
abstract void |
IndexFunction.execute(MatrixIndexes in,
MatrixIndexes out) |
void |
ReduceDiag.execute(MatrixIndexes in,
MatrixIndexes out) |
void |
ReduceCol.execute(MatrixIndexes in,
MatrixIndexes out) |
Modifier and Type | Method and Description |
---|---|
Iterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
SpoofSPInstruction.ReplicateRightFactorFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
Iterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
ParameterizedBuiltinSPInstruction.RDDRemoveEmptyFunctionInMem.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
Iterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
ParameterizedBuiltinSPInstruction.RDDRExpandFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
Iterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
ParameterizedBuiltinSPInstruction.RDDMapGroupedAggFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
scala.Tuple2<MatrixIndexes,MatrixBlock> |
ParameterizedBuiltinSPInstruction.RDDTransformDecodeExpandFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> in) |
scala.Tuple2<MatrixIndexes,MatrixBlock> |
AppendGAlignedSPInstruction.ShiftColumnIndex.call(scala.Tuple2<MatrixIndexes,MatrixBlock> kv) |
Iterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
AppendGSPInstruction.ShiftMatrix.call(scala.Tuple2<MatrixIndexes,MatrixBlock> kv) |
scala.Tuple2<MatrixIndexes,MatrixBlock> |
BuiltinNarySPInstruction.PadBlocksFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
scala.Tuple2<MatrixIndexes,MatrixBlock> |
AppendGSPInstruction.MergeWithShiftedBlocks.call(scala.Tuple2<MatrixIndexes,scala.Tuple2<Iterable<MatrixBlock>,Iterable<MatrixBlock>>> kv) |
Iterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
ParameterizedBuiltinSPInstruction.RDDRemoveEmptyFunction.call(scala.Tuple2<MatrixIndexes,scala.Tuple2<MatrixBlock,MatrixBlock>> arg0) |
Modifier and Type | Method and Description |
---|---|
Iterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
SpoofSPInstruction.ReplicateRightFactorFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
Iterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
ParameterizedBuiltinSPInstruction.RDDRemoveEmptyFunctionInMem.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
Iterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
ParameterizedBuiltinSPInstruction.RDDRExpandFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
Iterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
ParameterizedBuiltinSPInstruction.RDDMapGroupedAggFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
MatrixBlock |
ParameterizedBuiltinSPInstruction.RDDMapGroupedAggFunction2.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
scala.Tuple2<Long,FrameBlock> |
ParameterizedBuiltinSPInstruction.RDDTransformDecodeFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> in) |
scala.Tuple2<MatrixIndexes,MatrixBlock> |
ParameterizedBuiltinSPInstruction.RDDTransformDecodeExpandFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> in) |
scala.Tuple2<MatrixIndexes,MatrixBlock> |
AppendGAlignedSPInstruction.ShiftColumnIndex.call(scala.Tuple2<MatrixIndexes,MatrixBlock> kv) |
Iterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
AppendGSPInstruction.ShiftMatrix.call(scala.Tuple2<MatrixIndexes,MatrixBlock> kv) |
MatrixBlock |
AggregateUnarySPInstruction.RDDUAggFunction2.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
scala.Tuple2<MatrixIndexes,MatrixBlock> |
BuiltinNarySPInstruction.PadBlocksFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
scala.Tuple2<MatrixIndexes,MatrixBlock> |
AppendGSPInstruction.MergeWithShiftedBlocks.call(scala.Tuple2<MatrixIndexes,scala.Tuple2<Iterable<MatrixBlock>,Iterable<MatrixBlock>>> kv) |
Iterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
ParameterizedBuiltinSPInstruction.RDDRemoveEmptyFunction.call(scala.Tuple2<MatrixIndexes,scala.Tuple2<MatrixBlock,MatrixBlock>> arg0) |
static MatrixBlock |
MatrixIndexingSPInstruction.inmemoryIndexing(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in1,
DataCharacteristics mcIn,
DataCharacteristics mcOut,
IndexRange ixrange) |
void |
ParameterizedBuiltinSPInstruction.setOutputCharacteristicsForGroupedAgg(DataCharacteristics mc1,
DataCharacteristics mcOut,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixCell> out) |
Modifier and Type | Method and Description |
---|---|
MatrixIndexes |
IndexedMatrixValue.getIndexes() |
Modifier and Type | Method and Description |
---|---|
void |
IndexedMatrixValue.set(MatrixIndexes indexes2,
MatrixValue block2) |
Constructor and Description |
---|
IndexedMatrixValue(MatrixIndexes ind,
MatrixValue b) |
Modifier and Type | Method and Description |
---|---|
LazyIterableIterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
MatrixVectorBinaryOpPartitionFunction.call(Iterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> arg0) |
LazyIterableIterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
CopyMatrixBlockPairFunction.call(Iterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> arg0) |
scala.Tuple2<MatrixIndexes,MatrixBlock> |
GetMIMBFromRow.call(org.apache.spark.sql.Row row) |
Iterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
ExtractBlockForBinaryReblock.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
Iterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
ReplicateBlockFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
Iterator<scala.Tuple2<MatrixIndexes,WeightedCell>> |
ExtractGroup.ExtractGroupBroadcast.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg) |
Iterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
ReplicateVectorFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
scala.Tuple2<MatrixIndexes,MatrixBlock> |
ReorgMapFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
scala.Tuple2<MatrixIndexes,MatrixCell> |
CopyBinaryCellFunction.call(scala.Tuple2<MatrixIndexes,MatrixCell> arg0) |
Iterator<scala.Tuple2<MatrixIndexes,WeightedCell>> |
ExtractGroup.ExtractGroupJoin.call(scala.Tuple2<MatrixIndexes,scala.Tuple2<MatrixBlock,MatrixBlock>> arg) |
Iterator<scala.Tuple2<MatrixIndexes,WeightedCell>> |
ExtractGroupNWeights.call(scala.Tuple2<MatrixIndexes,scala.Tuple2<scala.Tuple2<MatrixBlock,MatrixBlock>,MatrixBlock>> arg) |
Modifier and Type | Method and Description |
---|---|
LazyIterableIterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
MatrixVectorBinaryOpPartitionFunction.call(Iterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> arg0) |
LazyIterableIterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
CopyMatrixBlockPairFunction.call(Iterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> arg0) |
Boolean |
FilterNonEmptyBlocksFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
Iterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
ExtractBlockForBinaryReblock.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
Iterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
ReplicateBlockFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
Iterator<scala.Tuple2<MatrixIndexes,WeightedCell>> |
ExtractGroup.ExtractGroupBroadcast.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg) |
Iterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
ReplicateVectorFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
Boolean |
FilterDiagMatrixBlocksFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
scala.Tuple2<MatrixIndexes,MatrixBlock> |
ReorgMapFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
Boolean |
IsBlockInRange.call(scala.Tuple2<MatrixIndexes,MatrixBlock> kv) |
Iterator<String> |
ConvertMatrixBlockToIJVLines.call(scala.Tuple2<MatrixIndexes,MatrixBlock> kv) |
org.apache.sysds.runtime.instructions.spark.functions.OuterVectorBinaryOpFunction.OuterVectorBinaryOpIterator |
OuterVectorBinaryOpFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0) |
Boolean |
IsBlockInList.call(scala.Tuple2<MatrixIndexes,MatrixBlock> kv) |
scala.Tuple2<MatrixIndexes,MatrixCell> |
CopyBinaryCellFunction.call(scala.Tuple2<MatrixIndexes,MatrixCell> arg0) |
Iterator<scala.Tuple2<MatrixIndexes,WeightedCell>> |
ExtractGroup.ExtractGroupJoin.call(scala.Tuple2<MatrixIndexes,scala.Tuple2<MatrixBlock,MatrixBlock>> arg) |
Iterator<scala.Tuple2<MatrixIndexes,WeightedCell>> |
ExtractGroupNWeights.call(scala.Tuple2<MatrixIndexes,scala.Tuple2<scala.Tuple2<MatrixBlock,MatrixBlock>,MatrixBlock>> arg) |
Modifier and Type | Method and Description |
---|---|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.aggByKeyStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
AggregateOperator aop) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.aggByKeyStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
AggregateOperator aop,
boolean deepCopyCombiner) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.aggByKeyStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
AggregateOperator aop,
int numPartitions,
boolean deepCopyCombiner) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
FrameRDDConverterUtils.binaryBlockToMatrixBlock(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> input,
DataCharacteristics mcIn,
DataCharacteristics mcOut) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDConverterUtils.binaryCellToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixCell> input,
DataCharacteristics mcOut,
boolean outputEmptyBlocks) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixCell> |
SparkUtils.cacheBinaryCellRDD(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixCell> input) |
Iterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
RDDConverterUtils.BinaryCellToBinaryBlockFunction.call(Iterator<scala.Tuple2<MatrixIndexes,MatrixCell>> arg0) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDConverterUtilsExt.coordinateMatrixToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.mllib.linalg.distributed.CoordinateMatrix input,
DataCharacteristics mcIn,
boolean outputEmptyBlocks) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDConverterUtilsExt.coordinateMatrixToBinaryBlock(org.apache.spark.SparkContext sc,
org.apache.spark.mllib.linalg.distributed.CoordinateMatrix input,
DataCharacteristics mcIn,
boolean outputEmptyBlocks) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
SparkUtils.copyBinaryBlockMatrix(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in)
Creates a partitioning-preserving deep copy of the input matrix RDD, where
the indexes and values are copied.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
SparkUtils.copyBinaryBlockMatrix(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
boolean deep)
Creates a partitioning-preserving copy of the input matrix RDD.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDConverterUtils.csvToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> input,
DataCharacteristics mc,
boolean hasHeader,
String delim,
boolean fill,
double fillValue,
Set<String> naStrings) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDConverterUtils.csvToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.api.java.JavaRDD<String> input,
DataCharacteristics mcOut,
boolean hasHeader,
String delim,
boolean fill,
double fillValue,
Set<String> naStrings) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDConverterUtils.dataFrameToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> df,
DataCharacteristics mc,
boolean containsID,
boolean isVector) |
static scala.Tuple2<MatrixIndexes,MatrixBlock> |
SparkUtils.fromIndexedMatrixBlock(IndexedMatrixValue in) |
static List<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
SparkUtils.fromIndexedMatrixBlock(List<IndexedMatrixValue> in) |
static Pair<MatrixIndexes,MatrixBlock> |
SparkUtils.fromIndexedMatrixBlockToPair(IndexedMatrixValue in) |
static List<Pair<MatrixIndexes,MatrixBlock>> |
SparkUtils.fromIndexedMatrixBlockToPair(List<IndexedMatrixValue> in) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
SparkUtils.getEmptyBlockRDD(org.apache.spark.api.java.JavaSparkContext sc,
DataCharacteristics mc)
Creates an RDD of empty blocks according to the given matrix characteristics.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.mergeByKey(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in)
Merges disjoint data of all blocks per key.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.mergeByKey(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
boolean deepCopyCombiner)
Merges disjoint data of all blocks per key.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.mergeByKey(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
int numPartitions,
boolean deepCopyCombiner)
Merges disjoint data of all blocks per key.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.mergeRowsByKey(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,RowMatrixBlock> in)
Merges disjoint data of all blocks per key.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortByVal(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in2,
long rlen,
int blen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortByVal(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
long rlen,
int blen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortByVals(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
long rlen,
long clen,
int blen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortDataByIx(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> data,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> ixmap,
long rlen,
long clen,
int blen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortDataByVal(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> val,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> data,
boolean asc,
long rlen,
long clen,
int blen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortDataByValMemSort(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> val,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> data,
boolean asc,
long rlen,
long clen,
int blen,
SparkExecutionContext sec,
ReorgOperator r_op)
This function collects and sorts value column in memory and then broadcasts it.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortDataByVals(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> val,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> data,
boolean asc,
long rlen,
long clen,
long clen2,
int blen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortIndexesByVal(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> val,
boolean asc,
long rlen,
int blen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortIndexesByVals(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
boolean asc,
long rlen,
long clen,
int blen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.sumByKeyStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.sumByKeyStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
boolean deepCopyCombiner) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.sumByKeyStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
int numPartitions,
boolean deepCopyCombiner) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,Double> |
RDDAggregateUtils.sumCellsByKeyStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,Double> in) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,Double> |
RDDAggregateUtils.sumCellsByKeyStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,Double> in,
int numParts) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDConverterUtils.textCellToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> input,
DataCharacteristics mcOut,
boolean outputEmptyBlocks,
FileFormatPropertiesMM mmProps) |
Modifier and Type | Method and Description |
---|---|
static IndexedMatrixValue |
SparkUtils.toIndexedMatrixBlock(MatrixIndexes ix,
MatrixBlock mb) |
Modifier and Type | Method and Description |
---|---|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.aggByKeyStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
AggregateOperator aop) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.aggByKeyStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
AggregateOperator aop,
boolean deepCopyCombiner) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.aggByKeyStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
AggregateOperator aop,
int numPartitions,
boolean deepCopyCombiner) |
static MatrixBlock |
RDDAggregateUtils.aggStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
AggregateOperator aop)
Single block aggregation over pair rdds with corrections for numerical stability.
|
static org.apache.spark.api.java.JavaRDD<String> |
RDDConverterUtils.binaryBlockToCsv(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
DataCharacteristics mcIn,
FileFormatPropertiesCSV props,
boolean strict) |
static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> |
RDDConverterUtils.binaryBlockToDataFrame(org.apache.spark.sql.SparkSession sparkSession,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
DataCharacteristics mc,
boolean toVector) |
static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> |
RDDConverterUtils.binaryBlockToDataFrame(org.apache.spark.sql.SQLContext sqlContext,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
DataCharacteristics mc,
boolean toVector)
Deprecated.
|
static org.apache.spark.api.java.JavaRDD<org.apache.spark.ml.feature.LabeledPoint> |
RDDConverterUtils.binaryBlockToLabeledPoints(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in)
Converter from binary block rdd to rdd of labeled points.
|
static org.apache.spark.api.java.JavaRDD<String> |
RDDConverterUtils.binaryBlockToTextCell(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
DataCharacteristics mc) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDConverterUtils.binaryCellToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixCell> input,
DataCharacteristics mcOut,
boolean outputEmptyBlocks) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixCell> |
SparkUtils.cacheBinaryCellRDD(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixCell> input) |
Iterator<scala.Tuple2<MatrixIndexes,MatrixBlock>> |
RDDConverterUtils.BinaryCellToBinaryBlockFunction.call(Iterator<scala.Tuple2<MatrixIndexes,MatrixCell>> arg0) |
static DataCharacteristics |
SparkUtils.computeDataCharacteristics(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixCell> input)
Utility to compute dimensions and non-zeros in a given RDD of binary cells.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
SparkUtils.copyBinaryBlockMatrix(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in)
Creates a partitioning-preserving deep copy of the input matrix RDD, where
the indexes and values are copied.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
SparkUtils.copyBinaryBlockMatrix(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
boolean deep)
Creates a partitioning-preserving copy of the input matrix RDD.
|
static long |
SparkUtils.getNonZeros(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> input) |
static org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.LongWritable,FrameBlock> |
FrameRDDConverterUtils.matrixBlockToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> input,
DataCharacteristics mcIn) |
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> |
FrameRDDConverterUtils.matrixBlockToBinaryBlockLongIndex(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> input,
DataCharacteristics dcIn) |
static double |
RDDAggregateUtils.max(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.mergeByKey(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in)
Merges disjoint data of all blocks per key.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.mergeByKey(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
boolean deepCopyCombiner)
Merges disjoint data of all blocks per key.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.mergeByKey(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
int numPartitions,
boolean deepCopyCombiner)
Merges disjoint data of all blocks per key.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.mergeRowsByKey(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,RowMatrixBlock> in)
Merges disjoint data of all blocks per key.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortByVal(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in2,
long rlen,
int blen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortByVal(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in2,
long rlen,
int blen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortByVal(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
long rlen,
int blen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortByVals(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
long rlen,
long clen,
int blen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortDataByIx(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> data,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> ixmap,
long rlen,
long clen,
int blen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortDataByIx(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> data,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> ixmap,
long rlen,
long clen,
int blen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortDataByVal(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> val,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> data,
boolean asc,
long rlen,
long clen,
int blen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortDataByVal(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> val,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> data,
boolean asc,
long rlen,
long clen,
int blen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortDataByValMemSort(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> val,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> data,
boolean asc,
long rlen,
long clen,
int blen,
SparkExecutionContext sec,
ReorgOperator r_op)
This function collects and sorts value column in memory and then broadcasts it.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortDataByValMemSort(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> val,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> data,
boolean asc,
long rlen,
long clen,
int blen,
SparkExecutionContext sec,
ReorgOperator r_op)
This function collects and sorts value column in memory and then broadcasts it.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortDataByVals(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> val,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> data,
boolean asc,
long rlen,
long clen,
long clen2,
int blen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortDataByVals(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> val,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> data,
boolean asc,
long rlen,
long clen,
long clen2,
int blen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortIndexesByVal(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> val,
boolean asc,
long rlen,
int blen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDSortUtils.sortIndexesByVals(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
boolean asc,
long rlen,
long clen,
int blen) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.sumByKeyStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.sumByKeyStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
boolean deepCopyCombiner) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
RDDAggregateUtils.sumByKeyStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in,
int numPartitions,
boolean deepCopyCombiner) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,Double> |
RDDAggregateUtils.sumCellsByKeyStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,Double> in) |
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,Double> |
RDDAggregateUtils.sumCellsByKeyStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,Double> in,
int numParts) |
static MatrixBlock |
RDDAggregateUtils.sumStable(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in) |
static IndexedMatrixValue |
SparkUtils.toIndexedMatrixBlock(scala.Tuple2<MatrixIndexes,MatrixBlock> in) |
Modifier and Type | Method and Description |
---|---|
MatrixIndexes |
MatrixIndexes.setIndexes(long r,
long c) |
MatrixIndexes |
MatrixIndexes.setIndexes(MatrixIndexes that) |
Modifier and Type | Method and Description |
---|---|
Pair<MatrixIndexes,MatrixCell> |
TextToBinaryCellConverter.next() |
Modifier and Type | Method and Description |
---|---|
MatrixBlock |
MatrixBlock.aggregateBinaryOperations(MatrixIndexes m1Index,
MatrixBlock m1,
MatrixIndexes m2Index,
MatrixBlock m2,
MatrixBlock ret,
AggregateBinaryOperator op) |
abstract MatrixValue |
MatrixValue.aggregateUnaryOperations(AggregateUnaryOperator op,
MatrixValue result,
int blen,
MatrixIndexes indexesIn) |
MatrixValue |
MatrixCell.aggregateUnaryOperations(AggregateUnaryOperator op,
MatrixValue result,
int blen,
MatrixIndexes indexesIn) |
MatrixBlock |
MatrixBlock.aggregateUnaryOperations(AggregateUnaryOperator op,
MatrixValue result,
int blen,
MatrixIndexes indexesIn) |
MatrixValue |
WeightedCell.aggregateUnaryOperations(AggregateUnaryOperator op,
MatrixValue result,
int blen,
MatrixIndexes indexesIn) |
MatrixValue |
CM_N_COVCell.aggregateUnaryOperations(AggregateUnaryOperator op,
MatrixValue result,
int blen,
MatrixIndexes indexesIn) |
abstract MatrixValue |
MatrixValue.aggregateUnaryOperations(AggregateUnaryOperator op,
MatrixValue result,
int blen,
MatrixIndexes indexesIn,
boolean inCP) |
MatrixValue |
MatrixCell.aggregateUnaryOperations(AggregateUnaryOperator op,
MatrixValue result,
int blen,
MatrixIndexes indexesIn,
boolean inCP) |
MatrixBlock |
MatrixBlock.aggregateUnaryOperations(AggregateUnaryOperator op,
MatrixValue result,
int blen,
MatrixIndexes indexesIn,
boolean inCP) |
MatrixValue |
CM_N_COVCell.aggregateUnaryOperations(AggregateUnaryOperator op,
MatrixValue result,
int blen,
MatrixIndexes indexesIn,
boolean inCP) |
int |
MatrixIndexes.compare(MatrixIndexes m1,
MatrixIndexes m2) |
int |
MatrixIndexes.compareTo(MatrixIndexes other) |
void |
BinaryBlockToTextCellConverter.convert(MatrixIndexes k1,
MatrixBlock v1)
Before calling convert, please make sure to setBlockSize(blen, blen);
|
abstract void |
MatrixValue.ctableOperations(Operator op,
MatrixIndexes ix1,
double scalar_that,
boolean left,
int blen,
CTableMap resultMap,
MatrixBlock resultBlock) |
void |
MatrixCell.ctableOperations(Operator op,
MatrixIndexes ix1,
double scalarThat,
boolean left,
int blen,
CTableMap resultMap,
MatrixBlock resultBlock) |
void |
MatrixBlock.ctableOperations(Operator op,
MatrixIndexes ix1,
double scalarThat,
boolean left,
int blen,
CTableMap resultMap,
MatrixBlock resultBlock)
Specific ctable case of ctable(seq(...),X), where X is the only
matrix input.
|
void |
CM_N_COVCell.ctableOperations(Operator op,
MatrixIndexes ix1,
double scalarThat,
boolean left,
int blen,
CTableMap resultMap,
MatrixBlock resultBlock) |
static MatrixBlock |
OperationsOnMatrixValues.matMult(MatrixIndexes indexes1,
MatrixBlock value1,
MatrixIndexes indexes2,
MatrixBlock value2,
MatrixIndexes indexesOut,
MatrixBlock valueOut,
AggregateBinaryOperator op) |
static void |
OperationsOnMatrixValues.performAggregateUnary(MatrixIndexes indexesIn,
MatrixValue valueIn,
MatrixIndexes indexesOut,
MatrixValue valueOut,
AggregateUnaryOperator op,
int blen) |
static void |
OperationsOnMatrixValues.performCtable(MatrixIndexes indexesIn1,
MatrixValue valueIn1,
double scalarIn2,
boolean left,
int blen,
CTableMap resultMap,
MatrixBlock resultBlock,
Operator op) |
static void |
OperationsOnMatrixValues.performCtable(MatrixIndexes indexesIn1,
MatrixValue valueIn1,
double scalarIn2,
double scalarIn3,
CTableMap resultMap,
MatrixBlock resultBlock,
Operator op) |
static void |
OperationsOnMatrixValues.performCtable(MatrixIndexes indexesIn1,
MatrixValue valueIn1,
double scalarIn2,
MatrixIndexes indexesIn3,
MatrixValue valueIn3,
CTableMap resultMap,
MatrixBlock resultBlock,
Operator op) |
static void |
OperationsOnMatrixValues.performCtable(MatrixIndexes indexesIn1,
MatrixValue valueIn1,
MatrixIndexes indexesIn2,
MatrixValue valueIn2,
double scalarIn3,
CTableMap resultMap,
MatrixBlock resultBlock,
Operator op) |
static void |
OperationsOnMatrixValues.performCtable(MatrixIndexes indexesIn1,
MatrixValue valueIn1,
MatrixIndexes indexesIn2,
MatrixValue valueIn2,
MatrixIndexes indexesIn3,
MatrixValue valueIn3,
CTableMap resultMap,
MatrixBlock resultBlock,
Operator op) |
static void |
OperationsOnMatrixValues.performReorg(MatrixIndexes indexesIn,
MatrixValue valueIn,
MatrixIndexes indexesOut,
MatrixValue valueOut,
ReorgOperator op,
int startRow,
int startColumn,
int length) |
static void |
OperationsOnMatrixValues.performZeroOut(MatrixIndexes indexesIn,
MatrixValue valueIn,
MatrixIndexes indexesOut,
MatrixValue valueOut,
IndexRange range,
boolean complementary) |
static void |
LibMatrixAgg.recomputeIndexes(MatrixBlock out,
AggregateUnaryOperator op,
int blen,
MatrixIndexes ix)
Recompute outputs (e.g., maxindex or minindex) according to block indexes from MR.
|
static void |
LibMatrixOuterAgg.resetOutputMatrix(MatrixIndexes in1Ix,
MatrixBlock in1Val,
MatrixIndexes outIx,
MatrixBlock outVal,
AggregateUnaryOperator uaggOp)
ReSet output matrix
|
MatrixIndexes |
MatrixIndexes.setIndexes(MatrixIndexes that) |
Constructor and Description |
---|
MatrixIndexes(MatrixIndexes indexes) |
Modifier and Type | Method and Description |
---|---|
org.apache.hadoop.mapred.RecordReader<MatrixIndexes,MatrixBlock> |
BinaryBlockInputFormat.getRecordReader(org.apache.hadoop.mapred.InputSplit split,
org.apache.hadoop.mapred.JobConf job,
org.apache.hadoop.mapred.Reporter reporter) |
Modifier and Type | Method and Description |
---|---|
static boolean |
UtilFunctions.isInBlockRange(MatrixIndexes ix,
int blen,
IndexRange ixrange) |
static boolean |
UtilFunctions.isInBlockRange(MatrixIndexes ix,
int blen,
long rl,
long ru,
long cl,
long cu) |
boolean |
BinaryBlockRecordReader.next(MatrixIndexes key,
MatrixBlock value) |
Modifier and Type | Method and Description |
---|---|
static MatrixBlock |
DataConverter.convertToMatrixBlock(HashMap<MatrixIndexes,Double> map) |
static MatrixBlock |
DataConverter.convertToMatrixBlock(HashMap<MatrixIndexes,Double> map,
int rlen,
int clen)
NOTE: this method also ensures the specified matrix dimensions
|
static int |
LocalFileUtils.readBlockSequenceFromLocal(String fname,
Pair<MatrixIndexes,MatrixValue>[] outValues,
HashMap<MatrixIndexes,Integer> outMap) |
Copyright © 2020 The Apache Software Foundation. All rights reserved.