org.apache.sysds.runtime.instructions.spark.data

Class RDDObject

java.lang.Object

org.apache.sysds.runtime.instructions.spark.data.LineageObject

org.apache.sysds.runtime.instructions.spark.data.RDDObject

public class RDDObject
extends LineageObject

Constructor Summary

Constructors

Constructor and Description
RDDObject(org.apache.spark.api.java.JavaPairRDD<?,?> rddvar)

Method Summary

Modifier and Type	Method and Description
boolean	allowsShortCircuitCollect()
boolean	allowsShortCircuitRead() Indicates if rdd is an hdfs file or a checkpoint over an hdfs file; in both cases, we can directly read the file instead of collecting the given rdd.
String	getHDFSFilename()
org.apache.spark.api.java.JavaPairRDD<?,?>	getRDD()
boolean	isCheckpointRDD()
boolean	isHDFSFile()
boolean	isParallelizedRDD()
boolean	isPending()
boolean	rHasCheckpointRDDChilds()
void	setCheckpointRDD(boolean flag)
void	setHDFSFile(boolean flag)
void	setHDFSFilename(String fname)
void	setParallelizedRDD(boolean flag)
void	setPending(boolean flag)

Methods inherited from class org.apache.sysds.runtime.instructions.spark.data.LineageObject

addLineageChild, decrementNumReferences, getLineageChilds, getNumReferences, hasBackReference, incrementNumReferences, setBackReference

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RDDObject

public RDDObject(org.apache.spark.api.java.JavaPairRDD<?,?> rddvar)

Method Detail

getRDD

public org.apache.spark.api.java.JavaPairRDD<?,?> getRDD()

setCheckpointRDD

public void setCheckpointRDD(boolean flag)

isCheckpointRDD

public boolean isCheckpointRDD()

setHDFSFile

public void setHDFSFile(boolean flag)

setHDFSFilename

public void setHDFSFilename(String fname)

isHDFSFile

public boolean isHDFSFile()

getHDFSFilename

public String getHDFSFilename()

setParallelizedRDD

public void setParallelizedRDD(boolean flag)

isParallelizedRDD

public boolean isParallelizedRDD()

setPending

public void setPending(boolean flag)

isPending

public boolean isPending()

allowsShortCircuitRead

public boolean allowsShortCircuitRead()

Indicates if rdd is an hdfs file or a checkpoint over an hdfs file; in both cases, we can directly read the file instead of collecting the given rdd.

Returns:

true if rdd is an hdfs file or a checkpoint over an hdfs file

allowsShortCircuitCollect

public boolean allowsShortCircuitCollect()

rHasCheckpointRDDChilds

public boolean rHasCheckpointRDDChilds()