| CachedReuseVariables |
|
| DataPartitioner |
This is the base class for all data partitioner.
|
| DataPartitionerLocal |
Partitions a given matrix into row or column partitions with a two pass-approach.
|
| DataPartitionerRemoteSpark |
MR job class for submitting parfor remote partitioning MR jobs.
|
| DataPartitionerRemoteSparkMapper |
NOTE: for the moment we only support binary block here
TODO extend impl for binarycell and textcell
Interface of Writable output in order to support both PairWritableBlock and PairWritableCell.
|
| DataPartitionerRemoteSparkReducer |
|
| LocalParWorker |
Instances of this class can be used to execute tasks in parallel.
|
| LocalTaskQueue<T> |
This class provides a way of dynamic task distribution to multiple workers
in local multi-threaded environments.
|
| ParForBody |
Wrapper for exchanging parfor body data structures.
|
| ParWorker |
Super class for master/worker pattern implementations.
|
| RemoteDPParForSpark |
TODO heavy hitter maintenance
TODO data partitioning with binarycell
|
| RemoteDPParForSparkWorker |
|
| RemoteParForJobReturn |
Wrapper for job return of ParFor REMOTE for transferring statistics and result symbol table.
|
| RemoteParForSpark |
This class serves two purposes: (1) isolating Spark imports to enable running in
environments where no Spark libraries are available, and (2) to follow the same
structure as the parfor remote_mr job submission.
|
| RemoteParForSparkWorker |
|
| RemoteParForUtils |
Common functionalities for parfor workers in MR jobs.
|
| ResultMerge<T extends CacheableData<?>> |
|
| ResultMergeFrameLocalMemory |
|
| ResultMergeLocalAutomatic |
|
| ResultMergeLocalFile |
TODO potential extension: parallel merge (create individual staging files concurrently)
NOTE: file merge typically used due to memory constraints - parallel merge would increase the memory
consumption again.
|
| ResultMergeLocalMemory |
Local in-memory realization of result merge.
|
| ResultMergeMatrix |
Due to independence of all iterations, any result has the following properties:
(1) non local var, (2) matrix object, and (3) completely independent.
|
| ResultMergeRemoteGrouping |
|
| ResultMergeRemoteSorting |
|
| ResultMergeRemoteSpark |
|
| ResultMergeRemoteSparkWCompare |
|
| ResultMergeTaggedMatrixIndexes |
This class serves as composite key for the remote result merge job
(for any data format) in order to sort on both matrix indexes and tag
but group all blocks according to matrix indexes only.
|
| Task |
A task is a logical group of one or multiple iterations (each iteration is assigned to exactly one task).
|
| TaskPartitioner |
This is the base class for all task partitioner.
|
| TaskPartitionerFactoring |
This factoring task partitioner virtually iterates over the given FOR loop (from, to, incr),
creates iterations and group them to tasks.
|
| TaskPartitionerFactoringCmax |
Factoring with maximum constraint (e.g., if LIX matrix out-of-core and we need
to bound the maximum number of iterations per map task -> memory bounds)
|
| TaskPartitionerFactoringCmin |
Factoring with minimum constraint (e.g., if communication is expensive)
|
| TaskPartitionerFixedsize |
This naive task partitioner virtually iterates over the given FOR loop (from, to, incr),
creates iterations and group them to tasks according to the given task size.
|
| TaskPartitionerNaive |
This static task partitioner virtually iterates over the given FOR loop (from, to, incr),
creates iterations and group them to tasks according to a task size of numIterations/numWorkers.
|
| TaskPartitionerStatic |
This static task partitioner virtually iterates over the given FOR loop (from, to, incr),
creates iterations and group them to tasks according to a task size of numIterations/numWorkers.
|