Modifier and Type | Class and Description |
---|---|
class |
CompressedMatrixBlock |
Modifier and Type | Method and Description |
---|---|
MatrixBlock |
CompressedMatrixBlock.slice(int rl,
int ru,
int cl,
int cu,
boolean deep,
CacheBlock ret) |
Modifier and Type | Class and Description |
---|---|
class |
CacheableData<T extends CacheBlock>
Each object of this class is a cache envelope for some large piece of data
called "cache block".
|
Modifier and Type | Method and Description |
---|---|
CacheBlock |
ByteBuffer.deserializeBlock() |
static CacheBlock |
CacheBlockFactory.newInstance(int code) |
CacheBlock |
UnifiedMemoryManager.pin(String key,
CacheBlock block,
boolean dirty)
Pins a cache block into operation memory.
|
static CacheBlock |
LazyWriteBuffer.readBlock(String fname,
boolean matrix) |
CacheBlock |
CacheBlock.slice(int rl,
int ru,
int cl,
int cu,
boolean deep,
CacheBlock block)
Slice a sub block out of the current block and write into the given output block.
|
CacheBlock |
CacheBlock.slice(int rl,
int ru,
int cl,
int cu,
CacheBlock block)
Slice a sub block out of the current block and write into the given output block.
|
Modifier and Type | Method and Description |
---|---|
static long |
LazyWriteBuffer.getCacheBlockSize(CacheBlock cb) |
static int |
CacheBlockFactory.getCode(CacheBlock block) |
static ArrayList<?> |
CacheBlockFactory.getPairList(CacheBlock block) |
static boolean |
CacheableData.isBelowCachingThreshold(CacheBlock data) |
static boolean |
ByteBuffer.isValidCapacity(long size,
CacheBlock cb)
Determines if byte buffer can hold the given size given this specific cache block.
|
void |
CacheBlock.merge(CacheBlock that,
boolean appendOnly)
Merge the given block into the current block.
|
CacheBlock |
UnifiedMemoryManager.pin(String key,
CacheBlock block,
boolean dirty)
Pins a cache block into operation memory.
|
void |
ByteBuffer.serializeBlock(CacheBlock cb) |
CacheBlock |
CacheBlock.slice(int rl,
int ru,
int cl,
int cu,
boolean deep,
CacheBlock block)
Slice a sub block out of the current block and write into the given output block.
|
CacheBlock |
CacheBlock.slice(int rl,
int ru,
int cl,
int cu,
CacheBlock block)
Slice a sub block out of the current block and write into the given output block.
|
void |
UnifiedMemoryManager.unpin(String key,
CacheBlock block)
Unpins (releases) a cache block from operation memory.
|
static int |
LazyWriteBuffer.writeBlock(String fname,
CacheBlock cb) |
Modifier and Type | Method and Description |
---|---|
org.apache.spark.broadcast.Broadcast<CacheBlock> |
SparkExecutionContext.broadcastVariable(CacheableData<CacheBlock> cd) |
Modifier and Type | Method and Description |
---|---|
static CacheableData<?> |
ExecutionContext.createCacheableData(CacheBlock cb) |
Modifier and Type | Method and Description |
---|---|
org.apache.spark.broadcast.Broadcast<CacheBlock> |
SparkExecutionContext.broadcastVariable(CacheableData<CacheBlock> cd) |
Modifier and Type | Method and Description |
---|---|
void |
CachedReuseVariables.reuseVariables(long pfid,
LocalVariableMap vars,
Collection<String> excludeList,
Map<String,org.apache.spark.broadcast.Broadcast<CacheBlock>> _brInputs,
boolean cleanCache) |
Constructor and Description |
---|
RemoteParForSparkWorker(long jobid,
String program,
HashMap<String,byte[]> clsMap,
boolean cpCaching,
org.apache.spark.util.LongAccumulator atasks,
org.apache.spark.util.LongAccumulator aiters,
Map<String,org.apache.spark.broadcast.Broadcast<CacheBlock>> brInputs,
boolean cleanCache,
Map<String,String> lineage) |
Modifier and Type | Class and Description |
---|---|
class |
TensorBlock
A
TensorBlock is the most top level representation of a tensor. |
Modifier and Type | Method and Description |
---|---|
CacheBlock |
TensorBlock.slice(int rl,
int ru,
int cl,
int cu,
boolean deep,
CacheBlock block) |
CacheBlock |
TensorBlock.slice(int rl,
int ru,
int cl,
int cu,
CacheBlock block) |
Modifier and Type | Method and Description |
---|---|
void |
TensorBlock.merge(CacheBlock that,
boolean appendOnly) |
CacheBlock |
TensorBlock.slice(int rl,
int ru,
int cl,
int cu,
boolean deep,
CacheBlock block) |
CacheBlock |
TensorBlock.slice(int rl,
int ru,
int cl,
int cu,
CacheBlock block) |
Modifier and Type | Class and Description |
---|---|
class |
BroadcastObject<T extends CacheBlock> |
class |
PartitionedBlock<T extends CacheBlock>
This class is for partitioned matrix/frame blocks, to be used as broadcasts.
|
class |
PartitionedBroadcast<T extends CacheBlock>
This class is a wrapper around an array of broadcasts of partitioned matrix/frame blocks,
which is required due to 2GB limitations of Spark's broadcast handling.
|
Modifier and Type | Class and Description |
---|---|
class |
FrameBlock |
class |
MatrixBlock |
Modifier and Type | Method and Description |
---|---|
void |
FrameBlock.merge(CacheBlock that,
boolean bDummy) |
void |
MatrixBlock.merge(CacheBlock that,
boolean appendOnly) |
static List |
OperationsOnMatrixValues.performSlice(IndexRange ixrange,
int blen,
int iix,
int jix,
CacheBlock in)
Slice used in broadcasting matrix blocks for spark, since this slice up a given matrix
into blocks.
|
FrameBlock |
FrameBlock.slice(int rl,
int ru,
int cl,
int cu,
boolean deep,
CacheBlock retCache)
Right indexing operations to slice a subframe out of this frame block.
|
MatrixBlock |
MatrixBlock.slice(int rl,
int ru,
int cl,
int cu,
boolean deep,
CacheBlock ret)
Method to perform rightIndex operation for a given lower and upper bounds in row and column dimensions.
|
FrameBlock |
FrameBlock.slice(int rl,
int ru,
int cl,
int cu,
CacheBlock retCache) |
MatrixBlock |
MatrixBlock.slice(int rl,
int ru,
int cl,
int cu,
CacheBlock ret) |
Modifier and Type | Method and Description |
---|---|
MatrixBlock |
MultiColumnEncoder.apply(CacheBlock in) |
MatrixBlock |
MultiColumnEncoder.apply(CacheBlock in,
int k) |
MatrixBlock |
MultiColumnEncoder.apply(CacheBlock in,
MatrixBlock out,
int outputCol) |
MatrixBlock |
Encoder.apply(CacheBlock in,
MatrixBlock out,
int outputCol)
Apply the generated metadata to the FrameBlock and saved the result in out.
|
MatrixBlock |
ColumnEncoder.apply(CacheBlock in,
MatrixBlock out,
int outputCol)
Apply Functions are only used in Single Threaded or Multi-Threaded Dense context.
|
MatrixBlock |
MultiColumnEncoder.apply(CacheBlock in,
MatrixBlock out,
int outputCol,
int k) |
MatrixBlock |
ColumnEncoderComposite.apply(CacheBlock in,
MatrixBlock out,
int outputCol,
int rowStart,
int blk) |
MatrixBlock |
ColumnEncoder.apply(CacheBlock in,
MatrixBlock out,
int outputCol,
int rowStart,
int blk) |
void |
ColumnEncoderBin.build(CacheBlock in) |
void |
MultiColumnEncoder.build(CacheBlock in) |
void |
ColumnEncoderComposite.build(CacheBlock in) |
void |
Encoder.build(CacheBlock in)
Build the transform meta data for the given block input.
|
void |
ColumnEncoderFeatureHash.build(CacheBlock in) |
void |
ColumnEncoderPassThrough.build(CacheBlock in) |
void |
ColumnEncoderDummycode.build(CacheBlock in) |
void |
ColumnEncoderRecode.build(CacheBlock in) |
void |
MultiColumnEncoder.build(CacheBlock in,
int k) |
MatrixBlock |
MultiColumnEncoder.encode(CacheBlock in) |
MatrixBlock |
MultiColumnEncoder.encode(CacheBlock in,
int k) |
List<DependencyTask<?>> |
ColumnEncoderComposite.getApplyTasks(CacheBlock in,
MatrixBlock out,
int outputCol) |
List<DependencyTask<?>> |
ColumnEncoder.getApplyTasks(CacheBlock in,
MatrixBlock out,
int outputCol) |
Callable<Object> |
ColumnEncoderBin.getBuildTask(CacheBlock in) |
Callable<Object> |
ColumnEncoder.getBuildTask(CacheBlock in) |
Callable<Object> |
ColumnEncoderRecode.getBuildTask(CacheBlock in) |
List<DependencyTask<?>> |
ColumnEncoderComposite.getBuildTasks(CacheBlock in) |
List<DependencyTask<?>> |
ColumnEncoderFeatureHash.getBuildTasks(CacheBlock in) |
List<DependencyTask<?>> |
ColumnEncoderPassThrough.getBuildTasks(CacheBlock in) |
List<DependencyTask<?>> |
ColumnEncoderDummycode.getBuildTasks(CacheBlock in) |
List<DependencyTask<?>> |
ColumnEncoder.getBuildTasks(CacheBlock in) |
Callable<Object> |
ColumnEncoderBin.getPartialBuildTask(CacheBlock in,
int startRow,
int blockSize,
HashMap<Integer,Object> ret) |
Callable<Object> |
ColumnEncoder.getPartialBuildTask(CacheBlock in,
int startRow,
int blockSize,
HashMap<Integer,Object> ret) |
Callable<Object> |
ColumnEncoderRecode.getPartialBuildTask(CacheBlock in,
int startRow,
int blockSize,
HashMap<Integer,Object> ret) |
Constructor and Description |
---|
FeatureHashSparseApplyTask(ColumnEncoderFeatureHash encoder,
CacheBlock input,
MatrixBlock out,
int outputCol) |
FeatureHashSparseApplyTask(ColumnEncoderFeatureHash encoder,
CacheBlock input,
MatrixBlock out,
int outputCol,
int startRow,
int blk) |
Modifier and Type | Method and Description |
---|---|
static CacheBlock |
LocalFileUtils.readCacheBlockFromLocal(String fname,
boolean matrix)
Reads a matrix/frame block from local file system.
|
Modifier and Type | Method and Description |
---|---|
static void |
LocalFileUtils.writeCacheBlockToLocal(String fname,
CacheBlock cb)
Writes a matrix/frame block to local file system.
|
Copyright © 2021 The Apache Software Foundation. All rights reserved.