Package org.apache.sysds.api.mlcontext
Class Frame
- java.lang.Object
-
- org.apache.sysds.api.mlcontext.Frame
-
public class Frame extends Object
Frame encapsulates a SystemDS frame.
-
-
Constructor Summary
Constructors Constructor Description Frame(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> binaryBlocks, FrameMetadata frameMetadata)
Create a Frame, specifying the SystemDS binary-block frame and its metadata.Frame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame)
Convert a Spark DataFrame to a SystemDS binary-block representation.Frame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, long numRows, long numCols)
Convert a Spark DataFrame to a SystemDS binary-block representation, specifying the number of rows and columns.Frame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, FrameMetadata frameMetadata)
Convert a Spark DataFrame to a SystemDS binary-block representation.Frame(FrameObject frameObject, SparkExecutionContext sparkExecutionContext)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description FrameMetadata
getFrameMetadata()
Obtain the frame metadataboolean
hasBinaryBlocks()
Whether or not this frame contains data as binary blocksboolean
hasFrameObject()
Whether or not this frame contains data as a FrameObjectString[][]
to2DStringArray()
Obtain the frame as a two-dimensional String arrayorg.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>
toBinaryBlocks()
Obtain the frame as aJavaPairRDD<Long, FrameBlock>
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>
toDF()
Obtain the frame as aDataFrame
FrameObject
toFrameObject()
Obtain the frame as a SystemDS FrameObject.org.apache.spark.api.java.JavaRDD<String>
toJavaRDDStringCSV()
Obtain the frame as aJavaRDD<String>
in CSV formatorg.apache.spark.api.java.JavaRDD<String>
toJavaRDDStringIJV()
Obtain the frame as aJavaRDD<String>
in IJV formatorg.apache.spark.rdd.RDD<String>
toRDDStringCSV()
Obtain the frame as aRDD<String>
in CSV formatorg.apache.spark.rdd.RDD<String>
toRDDStringIJV()
Obtain the frame as aRDD<String>
in IJV formatString
toString()
-
-
-
Constructor Detail
-
Frame
public Frame(FrameObject frameObject, SparkExecutionContext sparkExecutionContext)
-
Frame
public Frame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, FrameMetadata frameMetadata)
Convert a Spark DataFrame to a SystemDS binary-block representation.- Parameters:
dataFrame
- the Spark DataFrameframeMetadata
- frame metadata, such as number of rows and columns
-
Frame
public Frame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, long numRows, long numCols)
Convert a Spark DataFrame to a SystemDS binary-block representation, specifying the number of rows and columns.- Parameters:
dataFrame
- the Spark DataFramenumRows
- the number of rowsnumCols
- the number of columns
-
Frame
public Frame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame)
Convert a Spark DataFrame to a SystemDS binary-block representation.- Parameters:
dataFrame
- the Spark DataFrame
-
Frame
public Frame(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> binaryBlocks, FrameMetadata frameMetadata)
Create a Frame, specifying the SystemDS binary-block frame and its metadata.- Parameters:
binaryBlocks
- theJavaPairRDD<Long, FrameBlock>
frameframeMetadata
- frame metadata, such as number of rows and columnss
-
-
Method Detail
-
toFrameObject
public FrameObject toFrameObject()
Obtain the frame as a SystemDS FrameObject.- Returns:
- the frame as a SystemDS FrameObject
-
to2DStringArray
public String[][] to2DStringArray()
Obtain the frame as a two-dimensional String array- Returns:
- the frame as a two-dimensional String array
-
toJavaRDDStringIJV
public org.apache.spark.api.java.JavaRDD<String> toJavaRDDStringIJV()
Obtain the frame as aJavaRDD<String>
in IJV format- Returns:
- the frame as a
JavaRDD<String>
in IJV format
-
toJavaRDDStringCSV
public org.apache.spark.api.java.JavaRDD<String> toJavaRDDStringCSV()
Obtain the frame as aJavaRDD<String>
in CSV format- Returns:
- the frame as a
JavaRDD<String>
in CSV format
-
toRDDStringCSV
public org.apache.spark.rdd.RDD<String> toRDDStringCSV()
Obtain the frame as aRDD<String>
in CSV format- Returns:
- the frame as a
RDD<String>
in CSV format
-
toRDDStringIJV
public org.apache.spark.rdd.RDD<String> toRDDStringIJV()
Obtain the frame as aRDD<String>
in IJV format- Returns:
- the frame as a
RDD<String>
in IJV format
-
toDF
public org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> toDF()
Obtain the frame as aDataFrame
- Returns:
- the frame as a
DataFrame
-
toBinaryBlocks
public org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> toBinaryBlocks()
Obtain the frame as aJavaPairRDD<Long, FrameBlock>
- Returns:
- the frame as a
JavaPairRDD<Long, FrameBlock>
-
getFrameMetadata
public FrameMetadata getFrameMetadata()
Obtain the frame metadata- Returns:
- the frame metadata
-
hasBinaryBlocks
public boolean hasBinaryBlocks()
Whether or not this frame contains data as binary blocks- Returns:
true
if data as binary blocks are present,false
otherwise.
-
hasFrameObject
public boolean hasFrameObject()
Whether or not this frame contains data as a FrameObject- Returns:
true
if data as binary blocks are present,false
otherwise.
-
-