Modifier and Type | Method and Description |
---|---|
static FrameObject |
MLContextConversionUtil.binaryBlocksToFrameObject(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> binaryBlocks)
Convert a
JavaPairRDD<Long, FrameBlock> to a FrameObject . |
static FrameObject |
MLContextConversionUtil.binaryBlocksToFrameObject(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> binaryBlocks,
FrameMetadata frameMetadata)
Convert a
JavaPairRDD<Long, FrameBlock> to a FrameObject . |
static FrameObject |
MLContextConversionUtil.dataFrameToFrameObject(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame)
Convert a
DataFrame to a FrameObject . |
static FrameObject |
MLContextConversionUtil.dataFrameToFrameObject(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame,
FrameMetadata frameMetadata)
Convert a
DataFrame to a FrameObject . |
static FrameObject |
MLContextConversionUtil.frameBlockToFrameObject(String variableName,
FrameBlock frameBlock,
FrameMetadata frameMetadata)
Convert a
FrameBlock to a FrameObject . |
FrameObject |
MLResults.getFrameObject(String outputName)
Obtain an output as a
FrameObject |
static FrameObject |
MLContextConversionUtil.javaRDDStringCSVToFrameObject(org.apache.spark.api.java.JavaRDD<String> javaRDD)
Convert a
JavaRDD<String> in CSV format to a FrameObject |
static FrameObject |
MLContextConversionUtil.javaRDDStringCSVToFrameObject(org.apache.spark.api.java.JavaRDD<String> javaRDD,
FrameMetadata frameMetadata)
Convert a
JavaRDD<String> in CSV format to a FrameObject |
static FrameObject |
MLContextConversionUtil.javaRDDStringIJVToFrameObject(org.apache.spark.api.java.JavaRDD<String> javaRDD,
FrameMetadata frameMetadata)
Convert a
JavaRDD<String> in IJV format to a FrameObject
. |
static FrameObject |
MLContextConversionUtil.rddStringCSVToFrameObject(org.apache.spark.rdd.RDD<String> rdd)
Convert a
RDD<String> in CSV format to a FrameObject |
static FrameObject |
MLContextConversionUtil.rddStringCSVToFrameObject(org.apache.spark.rdd.RDD<String> rdd,
FrameMetadata frameMetadata)
Convert a
RDD<String> in CSV format to a FrameObject |
static FrameObject |
MLContextConversionUtil.rddStringIJVToFrameObject(org.apache.spark.rdd.RDD<String> rdd,
FrameMetadata frameMetadata)
Convert a
RDD<String> in IJV format to a FrameObject . |
FrameObject |
Frame.toFrameObject()
Obtain the frame as a SystemDS FrameObject.
|
Modifier and Type | Method and Description |
---|---|
static String[][] |
MLContextConversionUtil.frameObjectTo2DStringArray(FrameObject frameObject)
Convert a
FrameObject to a two-dimensional string array. |
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> |
MLContextConversionUtil.frameObjectToBinaryBlocks(FrameObject frameObject,
SparkExecutionContext sparkExecutionContext)
Convert a
FrameObject to a JavaPairRDD<Long, FrameBlock> . |
static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> |
MLContextConversionUtil.frameObjectToDataFrame(FrameObject frameObject,
SparkExecutionContext sparkExecutionContext)
Convert a
FrameObject to a DataFrame . |
static org.apache.spark.api.java.JavaRDD<String> |
MLContextConversionUtil.frameObjectToJavaRDDStringCSV(FrameObject frameObject,
String delimiter)
Convert a
FrameObject to a JavaRDD<String> in CSV format. |
static org.apache.spark.api.java.JavaRDD<String> |
MLContextConversionUtil.frameObjectToJavaRDDStringIJV(FrameObject frameObject)
Convert a
FrameObject to a JavaRDD<String> in IJV format. |
static List<String> |
MLContextConversionUtil.frameObjectToListStringCSV(FrameObject frameObject,
String delimiter)
Convert a
FrameObject to a List<String> in CSV format. |
static List<String> |
MLContextConversionUtil.frameObjectToListStringIJV(FrameObject frameObject)
Convert a
FrameObject to a List<String> in IJV format. |
static org.apache.spark.rdd.RDD<String> |
MLContextConversionUtil.frameObjectToRDDStringCSV(FrameObject frameObject,
String delimiter)
Convert a
FrameObject to a RDD<String> in CSV format. |
static org.apache.spark.rdd.RDD<String> |
MLContextConversionUtil.frameObjectToRDDStringIJV(FrameObject frameObject)
Convert a
FrameObject to a RDD<String> in IJV format. |
Constructor and Description |
---|
Frame(FrameObject frameObject,
SparkExecutionContext sparkExecutionContext) |
Modifier and Type | Method and Description |
---|---|
Types.ValueType[] |
FrameObject.mergeSchemas(FrameObject fo)
Creates a new collection which contains the schema of the current
frame object concatenated with the schema of the passed frame object.
|
Constructor and Description |
---|
FrameObject(FrameObject fo)
Copy constructor that copies meta data but NO data.
|
Modifier and Type | Method and Description |
---|---|
static FrameObject |
ExecutionContext.createFrameObject(FrameBlock fb) |
FrameObject |
ExecutionContext.getFrameObject(CPOperand input) |
FrameObject |
ExecutionContext.getFrameObject(String varname) |
Modifier and Type | Method and Description |
---|---|
org.apache.spark.api.java.JavaPairRDD<?,?> |
SparkExecutionContext.getRDDHandleForFrameObject(FrameObject fo,
Types.FileFormat fmt)
FIXME: currently this implementation assumes matrix representations but frame signature
in order to support the old transform implementation.
|
Modifier and Type | Method and Description |
---|---|
void |
InitFEDInstruction.federateFrame(FrameObject output,
List<org.apache.commons.lang3.tuple.Pair<FederatedRange,FederatedData>> workers) |
Copyright © 2020 The Apache Software Foundation. All rights reserved.