org.apache.sysds.api.mlcontext

Class Frame

java.lang.Object

org.apache.sysds.api.mlcontext.Frame

public class Frame
extends Object

Frame encapsulates a SystemDS frame.

Constructor Summary

Constructors

Constructor and Description
Frame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame) Convert a Spark DataFrame to a SystemDS binary-block representation.
Frame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, FrameMetadata frameMetadata) Convert a Spark DataFrame to a SystemDS binary-block representation.
Frame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, long numRows, long numCols) Convert a Spark DataFrame to a SystemDS binary-block representation, specifying the number of rows and columns.
Frame(FrameObject frameObject, SparkExecutionContext sparkExecutionContext)
Frame(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> binaryBlocks, FrameMetadata frameMetadata) Create a Frame, specifying the SystemDS binary-block frame and its metadata.

Method Summary

Modifier and Type	Method and Description
FrameMetadata	getFrameMetadata() Obtain the frame metadata
boolean	hasBinaryBlocks() Whether or not this frame contains data as binary blocks
boolean	hasFrameObject() Whether or not this frame contains data as a FrameObject
String[][]	to2DStringArray() Obtain the frame as a two-dimensional String array
org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>	toBinaryBlocks() Obtain the frame as a JavaPairRDD<Long, FrameBlock>
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	toDF() Obtain the frame as a DataFrame
FrameObject	toFrameObject() Obtain the frame as a SystemDS FrameObject.
org.apache.spark.api.java.JavaRDD<String>	toJavaRDDStringCSV() Obtain the frame as a JavaRDD<String> in CSV format
org.apache.spark.api.java.JavaRDD<String>	toJavaRDDStringIJV() Obtain the frame as a JavaRDD<String> in IJV format
org.apache.spark.rdd.RDD<String>	toRDDStringCSV() Obtain the frame as a RDD<String> in CSV format
org.apache.spark.rdd.RDD<String>	toRDDStringIJV() Obtain the frame as a RDD<String> in IJV format
String	toString()

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Frame

public Frame(FrameObject frameObject,
             SparkExecutionContext sparkExecutionContext)

Frame

public Frame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame,
             FrameMetadata frameMetadata)

Convert a Spark DataFrame to a SystemDS binary-block representation.

Parameters:

dataFrame - the Spark DataFrame

frameMetadata - frame metadata, such as number of rows and columns

Frame

public Frame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame,
             long numRows,
             long numCols)

Convert a Spark DataFrame to a SystemDS binary-block representation, specifying the number of rows and columns.

Parameters:

dataFrame - the Spark DataFrame

numRows - the number of rows

numCols - the number of columns

Frame

public Frame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame)

Convert a Spark DataFrame to a SystemDS binary-block representation.

Parameters:

dataFrame - the Spark DataFrame

Frame

public Frame(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> binaryBlocks,
             FrameMetadata frameMetadata)

Create a Frame, specifying the SystemDS binary-block frame and its metadata.

Parameters:

binaryBlocks - the JavaPairRDD<Long, FrameBlock> frame

frameMetadata - frame metadata, such as number of rows and columnss

Method Detail

toFrameObject

public FrameObject toFrameObject()

Obtain the frame as a SystemDS FrameObject.

Returns:

the frame as a SystemDS FrameObject

to2DStringArray

public String[][] to2DStringArray()

Obtain the frame as a two-dimensional String array

Returns:

the frame as a two-dimensional String array

toJavaRDDStringIJV

public org.apache.spark.api.java.JavaRDD<String> toJavaRDDStringIJV()

Obtain the frame as a JavaRDD<String> in IJV format

Returns:

the frame as a JavaRDD<String> in IJV format

toJavaRDDStringCSV

public org.apache.spark.api.java.JavaRDD<String> toJavaRDDStringCSV()

Obtain the frame as a JavaRDD<String> in CSV format

Returns:

the frame as a JavaRDD<String> in CSV format

toRDDStringCSV

public org.apache.spark.rdd.RDD<String> toRDDStringCSV()

Obtain the frame as a RDD<String> in CSV format

Returns:

the frame as a RDD<String> in CSV format

toRDDStringIJV

public org.apache.spark.rdd.RDD<String> toRDDStringIJV()

Obtain the frame as a RDD<String> in IJV format

Returns:

the frame as a RDD<String> in IJV format

toDF

public org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> toDF()

Obtain the frame as a DataFrame

Returns:

the frame as a DataFrame

toBinaryBlocks

public org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> toBinaryBlocks()

Obtain the frame as a JavaPairRDD<Long, FrameBlock>

Returns:

the frame as a JavaPairRDD<Long, FrameBlock>

getFrameMetadata

public FrameMetadata getFrameMetadata()

Obtain the frame metadata

Returns:

the frame metadata

toString

public String toString()

Overrides:

toString in class Object

hasBinaryBlocks

public boolean hasBinaryBlocks()

Whether or not this frame contains data as binary blocks

Returns:

true if data as binary blocks are present, false otherwise.

hasFrameObject

public boolean hasFrameObject()

Whether or not this frame contains data as a FrameObject

Returns:

true if data as binary blocks are present, false otherwise.