Modifier and Type | Method and Description |
---|---|
TensorBlock |
ExecutionContext.getTensorInput(String varName)
Pins a matrix variable into memory and returns the internal matrix block.
|
static TensorBlock |
SparkExecutionContext.toTensorBlock(org.apache.spark.api.java.JavaPairRDD<TensorIndexes,TensorBlock> rdd,
DataCharacteristics dc) |
Modifier and Type | Method and Description |
---|---|
org.apache.spark.api.java.JavaPairRDD<TensorIndexes,TensorBlock> |
SparkExecutionContext.getBinaryTensorBlockRDDHandleForVariable(String varname)
Spark instructions should call this for all tensor inputs except broadcast
variables.
|
org.apache.spark.api.java.JavaPairRDD<TensorIndexes,TensorBlock> |
SparkExecutionContext.getBinaryTensorBlockRDDHandleForVariable(String varname,
int numParts,
boolean inclEmpty) |
PartitionedBroadcast<TensorBlock> |
SparkExecutionContext.getBroadcastForTensorObject(TensorObject to) |
PartitionedBroadcast<TensorBlock> |
SparkExecutionContext.getBroadcastForTensorVariable(String varname) |
static org.apache.spark.api.java.JavaPairRDD<TensorIndexes,TensorBlock> |
SparkExecutionContext.toTensorJavaPairRDD(org.apache.spark.api.java.JavaSparkContext sc,
TensorBlock src,
int blen) |
static org.apache.spark.api.java.JavaPairRDD<TensorIndexes,TensorBlock> |
SparkExecutionContext.toTensorJavaPairRDD(org.apache.spark.api.java.JavaSparkContext sc,
TensorBlock src,
int blen,
int numParts,
boolean inclEmpty) |
Modifier and Type | Method and Description |
---|---|
void |
ExecutionContext.setTensorOutput(String varName,
TensorBlock outputData) |
static org.apache.spark.api.java.JavaPairRDD<TensorIndexes,TensorBlock> |
SparkExecutionContext.toTensorJavaPairRDD(org.apache.spark.api.java.JavaSparkContext sc,
TensorBlock src,
int blen) |
static org.apache.spark.api.java.JavaPairRDD<TensorIndexes,TensorBlock> |
SparkExecutionContext.toTensorJavaPairRDD(org.apache.spark.api.java.JavaSparkContext sc,
TensorBlock src,
int blen,
int numParts,
boolean inclEmpty) |
Modifier and Type | Method and Description |
---|---|
static TensorBlock |
SparkExecutionContext.toTensorBlock(org.apache.spark.api.java.JavaPairRDD<TensorIndexes,TensorBlock> rdd,
DataCharacteristics dc) |
Modifier and Type | Method and Description |
---|---|
TensorBlock |
TensorBlock.allocateBlock()
If data is not yet allocated, allocate.
|
TensorBlock |
TensorBlock.binaryOperations(BinaryOperator op,
TensorBlock thatValue,
TensorBlock result) |
TensorBlock |
TensorBlock.copy(int[] lower,
int[] upper,
TensorBlock src)
Copy a part of another
TensorBlock |
TensorBlock |
TensorBlock.copy(TensorBlock src) |
TensorBlock |
TensorBlock.copyExact(int[] lower,
int[] upper,
TensorBlock src)
Copy a part of another
TensorBlock . |
TensorBlock |
IndexedTensorBlock.getValue() |
TensorBlock |
TensorBlock.slice(int[] offsets,
TensorBlock outBlock)
Slice the current block and write into the outBlock.
|
Modifier and Type | Method and Description |
---|---|
TensorBlock |
TensorBlock.binaryOperations(BinaryOperator op,
TensorBlock thatValue,
TensorBlock result) |
static void |
LibTensorBincell.bincellOp(TensorBlock m1,
TensorBlock m2,
TensorBlock ret,
BinaryOperator op)
tensor-tensor binary operations
|
TensorBlock |
TensorBlock.copy(int[] lower,
int[] upper,
TensorBlock src)
Copy a part of another
TensorBlock |
TensorBlock |
TensorBlock.copy(TensorBlock src) |
TensorBlock |
TensorBlock.copyExact(int[] lower,
int[] upper,
TensorBlock src)
Copy a part of another
TensorBlock . |
static boolean |
LibTensorBincell.isValidDimensionsBinary(TensorBlock m1,
TensorBlock m2) |
void |
IndexedTensorBlock.set(TensorIndexes indexes2,
TensorBlock block2) |
TensorBlock |
TensorBlock.slice(int[] offsets,
TensorBlock outBlock)
Slice the current block and write into the outBlock.
|
Constructor and Description |
---|
IndexedTensorBlock(TensorIndexes ind,
TensorBlock b) |
TensorBlock(TensorBlock that)
Copy constructor
|
Modifier and Type | Method and Description |
---|---|
TensorBlock |
AggregateUnarySPInstruction.RDDUTensorAggFunction2.call(scala.Tuple2<TensorIndexes,TensorBlock> arg0) |
Modifier and Type | Method and Description |
---|---|
TensorBlock |
AggregateUnarySPInstruction.RDDUTensorAggFunction2.call(scala.Tuple2<TensorIndexes,TensorBlock> arg0) |
Modifier and Type | Method and Description |
---|---|
TensorBlock |
TensorTensorBinaryOpFunction.call(scala.Tuple2<TensorBlock,TensorBlock> arg0) |
Modifier and Type | Method and Description |
---|---|
LazyIterableIterator<scala.Tuple2<TensorIndexes,TensorBlock>> |
TensorTensorBinaryOpPartitionFunction.call(Iterator<scala.Tuple2<TensorIndexes,TensorBlock>> arg0) |
Iterator<scala.Tuple2<TensorIndexes,TensorBlock>> |
ReplicateTensorFunction.call(scala.Tuple2<TensorIndexes,TensorBlock> arg0) |
Iterator<scala.Tuple2<TensorIndexes,TensorBlock>> |
ReblockTensorFunction.call(scala.Tuple2<TensorIndexes,TensorBlock> arg0) |
Modifier and Type | Method and Description |
---|---|
LazyIterableIterator<scala.Tuple2<TensorIndexes,TensorBlock>> |
TensorTensorBinaryOpPartitionFunction.call(Iterator<scala.Tuple2<TensorIndexes,TensorBlock>> arg0) |
TensorBlock |
TensorTensorBinaryOpFunction.call(scala.Tuple2<TensorBlock,TensorBlock> arg0) |
TensorBlock |
TensorTensorBinaryOpFunction.call(scala.Tuple2<TensorBlock,TensorBlock> arg0) |
Iterator<scala.Tuple2<TensorIndexes,TensorBlock>> |
ReplicateTensorFunction.call(scala.Tuple2<TensorIndexes,TensorBlock> arg0) |
Iterator<scala.Tuple2<TensorIndexes,TensorBlock>> |
ReblockTensorFunction.call(scala.Tuple2<TensorIndexes,TensorBlock> arg0) |
Constructor and Description |
---|
TensorTensorBinaryOpPartitionFunction(BinaryOperator op,
PartitionedBroadcast<TensorBlock> binput,
boolean[] replicateDim) |
Modifier and Type | Method and Description |
---|---|
static TensorBlock |
RDDAggregateUtils.aggStableTensor(org.apache.spark.api.java.JavaPairRDD<TensorIndexes,TensorBlock> in,
AggregateOperator aop)
Single block aggregation over pair rdds with corrections for numerical stability.
|
static TensorBlock |
RDDAggregateUtils.aggStableTensor(org.apache.spark.api.java.JavaRDD<TensorBlock> in,
AggregateOperator aop)
Single block aggregation over rdds with corrections for numerical stability.
|
Modifier and Type | Method and Description |
---|---|
static IndexedTensorBlock |
SparkUtils.toIndexedTensorBlock(TensorIndexes ix,
TensorBlock mb) |
Modifier and Type | Method and Description |
---|---|
static TensorBlock |
RDDAggregateUtils.aggStableTensor(org.apache.spark.api.java.JavaPairRDD<TensorIndexes,TensorBlock> in,
AggregateOperator aop)
Single block aggregation over pair rdds with corrections for numerical stability.
|
static TensorBlock |
RDDAggregateUtils.aggStableTensor(org.apache.spark.api.java.JavaRDD<TensorBlock> in,
AggregateOperator aop)
Single block aggregation over rdds with corrections for numerical stability.
|
static IndexedTensorBlock |
SparkUtils.toIndexedTensorBlock(scala.Tuple2<TensorIndexes,TensorBlock> in) |
Modifier and Type | Method and Description |
---|---|
abstract TensorBlock |
TensorReader.readTensorFromHDFS(String fname,
long[] dims,
int blen,
Types.ValueType[] schema) |
TensorBlock |
TensorReaderTextCell.readTensorFromHDFS(String fname,
long[] dims,
int blen,
Types.ValueType[] schema) |
TensorBlock |
TensorReaderBinaryBlock.readTensorFromHDFS(String fname,
long[] dims,
int blen,
Types.ValueType[] schema) |
Modifier and Type | Method and Description |
---|---|
void |
TensorWriterBinaryBlock.writeTensorToHDFS(TensorBlock src,
String fname,
int blen) |
void |
TensorWriterTextCell.writeTensorToHDFS(TensorBlock src,
String fname,
int blen) |
abstract void |
TensorWriter.writeTensorToHDFS(TensorBlock src,
String fname,
int blen) |
Constructor and Description |
---|
WriteFileTask(org.apache.hadoop.fs.Path path,
org.apache.hadoop.mapred.JobConf job,
org.apache.hadoop.fs.FileSystem fs,
TensorBlock src,
long rl,
long ru,
int blen) |
Modifier and Type | Method and Description |
---|---|
static TensorBlock |
DataConverter.convertToTensorBlock(MatrixBlock mb,
Types.ValueType vt,
boolean toBasicTensor) |
static TensorBlock |
DataConverter.readTensorFromHDFS(String dir,
Types.FileFormat fmt,
long[] dims,
int blen,
Types.ValueType[] schema) |
Modifier and Type | Method and Description |
---|---|
static String |
DataConverter.toString(TensorBlock tb) |
static String |
DataConverter.toString(TensorBlock tb,
boolean sparse,
String separator,
String lineseparator,
String leftBorder,
String rightBorder,
int rowsToPrint,
int colsToPrint,
int decimal)
Returns a string representation of a tensor
|
static void |
DataConverter.writeTensorToHDFS(TensorBlock tensor,
String dir,
Types.FileFormat fmt,
DataCharacteristics dc) |
Copyright © 2020 The Apache Software Foundation. All rights reserved.