org.apache.sysml.runtime.matrix

Class MMCJMR

java.lang.Object

org.apache.sysml.runtime.matrix.MMCJMR

public class MMCJMR
extends Object

Method Summary

Modifier and Type	Method and Description
protected static int	determineNumReducers(long[] rlen, long[] clen, int defaultNumRed, long numRedGroups) Determine number of reducers based on configured number of reducers, number of results groups and input data divided by blocksize (as heuristic for useful degree of parallelism).
static JobReturn	runJob(MRJobInstruction inst, String[] inputs, InputInfo[] inputInfos, long[] rlens, long[] clens, int[] brlens, int[] bclens, String instructionsInMapper, String aggInstructionsInReducer, String aggBinInstrction, int numReducers, int replication, String output, OutputInfo outputinfo)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

runJob

public static JobReturn runJob(MRJobInstruction inst,
                               String[] inputs,
                               InputInfo[] inputInfos,
                               long[] rlens,
                               long[] clens,
                               int[] brlens,
                               int[] bclens,
                               String instructionsInMapper,
                               String aggInstructionsInReducer,
                               String aggBinInstrction,
                               int numReducers,
                               int replication,
                               String output,
                               OutputInfo outputinfo)
                        throws Exception

Throws:

Exception

determineNumReducers

protected static int determineNumReducers(long[] rlen,
                                          long[] clen,
                                          int defaultNumRed,
                                          long numRedGroups)

Determine number of reducers based on configured number of reducers, number of results groups and input data divided by blocksize (as heuristic for useful degree of parallelism).

Parameters:

rlen - array of numbers of rows

clen - array of numbers of columns

defaultNumRed - default number of reducers

numRedGroups - number of reducer groups

Returns:

number of reducers