Modifier and Type | Method and Description |
---|---|
FrameBlock |
Connection.convertToFrame(InputStream input,
int rows,
int cols)
Converts an input stream of a string frame in textcell format
into a frame block.
|
FrameBlock |
Connection.convertToFrame(InputStream input,
int rows,
int cols,
String format)
Converts an input stream of a frame in csv or textcell format
into a frame block.
|
FrameBlock |
Connection.convertToFrame(InputStream input,
String meta)
Converts an input stream of a string frame in csv or textcell format
into a frame block.
|
FrameBlock |
Connection.convertToFrame(String input,
int rows,
int cols)
Converts an input string representation of a frame in textcell format
into a frame block.
|
FrameBlock |
Connection.convertToFrame(String input,
String meta)
Converts an input string representation of a frame in csv or textcell format
into a frame block.
|
FrameBlock |
ResultVariables.getFrameBlock(String varname)
Obtain the frame represented by the given output variable.
|
FrameBlock |
Connection.readTransformMetaDataFromFile(String metapath)
Reads transform meta data from an HDFS file path and converts it into an in-memory
FrameBlock object.
|
FrameBlock |
Connection.readTransformMetaDataFromFile(String spec,
String metapath)
Reads transform meta data from an HDFS file path and converts it into an in-memory
FrameBlock object.
|
FrameBlock |
Connection.readTransformMetaDataFromFile(String spec,
String metapath,
String colDelim)
Reads transform meta data from an HDFS file path and converts it into an in-memory
FrameBlock object.
|
FrameBlock |
Connection.readTransformMetaDataFromPath(String metapath)
Reads transform meta data from the class path and converts it into an in-memory
FrameBlock object.
|
FrameBlock |
Connection.readTransformMetaDataFromPath(String spec,
String metapath)
Reads transform meta data from the class path and converts it into an in-memory
FrameBlock object.
|
FrameBlock |
Connection.readTransformMetaDataFromPath(String spec,
String metapath,
String colDelim)
Reads transform meta data from the class path and converts it into an in-memory
FrameBlock object.
|
Modifier and Type | Method and Description |
---|---|
void |
PreparedScript.setFrame(String varname,
FrameBlock frame,
boolean reuse)
Binds a frame object to a registered input variable.
|
Modifier and Type | Method and Description |
---|---|
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> |
MLContextConversionUtil.dataFrameToFrameBinaryBlocks(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame,
FrameMetadata frameMetadata)
Convert a
DataFrame to a JavaPairRDD<Long, FrameBlock>
binary-block frame. |
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> |
MLContextConversionUtil.frameObjectToBinaryBlocks(org.apache.sysml.runtime.controlprogram.caching.FrameObject frameObject,
org.apache.sysml.runtime.controlprogram.context.SparkExecutionContext sparkExecutionContext)
Convert a
FrameObject to a JavaPairRDD<Long, FrameBlock> . |
org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> |
Frame.toBinaryBlocks()
Obtain the frame as a
JavaPairRDD<Long, FrameBlock> |
Modifier and Type | Method and Description |
---|---|
static org.apache.sysml.runtime.controlprogram.caching.FrameObject |
MLContextConversionUtil.frameBlockToFrameObject(String variableName,
FrameBlock frameBlock,
FrameMetadata frameMetadata)
Convert a
FrameBlock to a FrameObject . |
Modifier and Type | Method and Description |
---|---|
static org.apache.sysml.runtime.controlprogram.caching.FrameObject |
MLContextConversionUtil.binaryBlocksToFrameObject(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> binaryBlocks)
Convert a
JavaPairRDD<Long, FrameBlock> to a FrameObject . |
static org.apache.sysml.runtime.controlprogram.caching.FrameObject |
MLContextConversionUtil.binaryBlocksToFrameObject(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> binaryBlocks,
FrameMetadata frameMetadata)
Convert a
JavaPairRDD<Long, FrameBlock> to a FrameObject . |
Constructor and Description |
---|
Frame(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> binaryBlocks,
FrameMetadata frameMetadata)
Create a Frame, specifying the SystemML binary-block frame and its
metadata.
|
Modifier and Type | Method and Description |
---|---|
scala.Tuple2<Long,FrameBlock> |
ParameterizedBuiltinSPInstruction.RDDTransformApplyFunction.call(scala.Tuple2<Long,FrameBlock> in) |
scala.Tuple2<Long,FrameBlock> |
ParameterizedBuiltinSPInstruction.RDDTransformDecodeFunction.call(scala.Tuple2<MatrixIndexes,MatrixBlock> in) |
protected org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> |
CSVReblockSPInstruction.processFrameCSVReblockInstruction(org.apache.sysml.runtime.controlprogram.context.SparkExecutionContext sec,
MatrixCharacteristics mcOut,
org.apache.sysml.parser.Expression.ValueType[] schema) |
Modifier and Type | Method and Description |
---|---|
Iterator<scala.Tuple2<Integer,Object>> |
MultiReturnParameterizedBuiltinSPInstruction.TransformEncodeBuildFunction.call(Iterator<scala.Tuple2<Long,FrameBlock>> iter) |
Iterator<scala.Tuple2<Integer,FrameBlock.ColumnMetadata>> |
MultiReturnParameterizedBuiltinSPInstruction.TransformEncodeBuild2Function.call(Iterator<scala.Tuple2<Long,FrameBlock>> iter) |
scala.Tuple2<Long,FrameBlock> |
ParameterizedBuiltinSPInstruction.RDDTransformApplyFunction.call(scala.Tuple2<Long,FrameBlock> in) |
scala.Tuple2<Long,Long> |
ParameterizedBuiltinSPInstruction.RDDTransformApplyOffsetFunction.call(scala.Tuple2<Long,FrameBlock> in) |
Modifier and Type | Method and Description |
---|---|
scala.Tuple2<org.apache.hadoop.io.LongWritable,FrameBlock> |
FrameRDDConverterUtils.LongFrameToLongWritableFrameFunction.call(scala.Tuple2<Long,FrameBlock> arg0) |
scala.Tuple2<org.apache.hadoop.io.LongWritable,FrameBlock> |
FrameRDDConverterUtils.LongWritableToSerFunction.call(scala.Tuple2<org.apache.hadoop.io.LongWritable,FrameBlock> arg0) |
scala.Tuple2<Long,FrameBlock> |
FrameRDDConverterUtils.LongWritableFrameToLongFrameFunction.call(scala.Tuple2<org.apache.hadoop.io.LongWritable,FrameBlock> arg0) |
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> |
FrameRDDConverterUtils.csvToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> input,
MatrixCharacteristics mc,
org.apache.sysml.parser.Expression.ValueType[] schema,
boolean hasHeader,
String delim,
boolean fill,
double fillValue) |
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> |
FrameRDDConverterUtils.csvToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.api.java.JavaRDD<String> input,
MatrixCharacteristics mcOut,
org.apache.sysml.parser.Expression.ValueType[] schema,
boolean hasHeader,
String delim,
boolean fill,
double fillValue) |
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> |
FrameRDDConverterUtils.dataFrameToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> df,
MatrixCharacteristics mc,
boolean containsID) |
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> |
FrameRDDConverterUtils.dataFrameToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> df,
MatrixCharacteristics mc,
boolean containsID,
Pair<String[],org.apache.sysml.parser.Expression.ValueType[]> out) |
static List<scala.Tuple2<Long,FrameBlock>> |
SparkUtils.fromIndexedFrameBlock(List<Pair<Long,FrameBlock>> in) |
static scala.Tuple2<Long,FrameBlock> |
SparkUtils.fromIndexedFrameBlock(Pair<Long,FrameBlock> in) |
static org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.LongWritable,FrameBlock> |
FrameRDDConverterUtils.matrixBlockToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> input,
MatrixCharacteristics mcIn) |
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> |
FrameRDDConverterUtils.matrixBlockToBinaryBlockLongIndex(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> input,
MatrixCharacteristics mcIn) |
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> |
FrameRDDAggregateUtils.mergeByKey(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> in) |
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> |
FrameRDDConverterUtils.textCellToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> in,
MatrixCharacteristics mcOut,
org.apache.sysml.parser.Expression.ValueType[] schema) |
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> |
FrameRDDConverterUtils.textCellToBinaryBlockLongIndex(org.apache.spark.api.java.JavaSparkContext sc,
org.apache.spark.api.java.JavaPairRDD<Long,org.apache.hadoop.io.Text> input,
MatrixCharacteristics mc,
org.apache.sysml.parser.Expression.ValueType[] schema) |
static Pair<Long,FrameBlock> |
SparkUtils.toIndexedFrameBlock(scala.Tuple2<Long,FrameBlock> in) |
Modifier and Type | Method and Description |
---|---|
static org.apache.spark.api.java.JavaRDD<String> |
FrameRDDConverterUtils.binaryBlockToCsv(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> in,
MatrixCharacteristics mcIn,
org.apache.sysml.runtime.io.FileFormatPropertiesCSV props,
boolean strict) |
static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> |
FrameRDDConverterUtils.binaryBlockToDataFrame(org.apache.spark.sql.SparkSession sparkSession,
org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> in,
MatrixCharacteristics mc,
org.apache.sysml.parser.Expression.ValueType[] schema) |
static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> |
FrameRDDConverterUtils.binaryBlockToDataFrame(org.apache.spark.sql.SQLContext sqlContext,
org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> in,
MatrixCharacteristics mc,
org.apache.sysml.parser.Expression.ValueType[] schema)
Deprecated.
|
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> |
FrameRDDConverterUtils.binaryBlockToMatrixBlock(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> input,
MatrixCharacteristics mcIn,
MatrixCharacteristics mcOut) |
static org.apache.spark.api.java.JavaRDD<String> |
FrameRDDConverterUtils.binaryBlockToTextCell(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> input,
MatrixCharacteristics mcIn) |
scala.Tuple2<org.apache.hadoop.io.LongWritable,FrameBlock> |
FrameRDDConverterUtils.LongFrameToLongWritableFrameFunction.call(scala.Tuple2<Long,FrameBlock> arg0) |
scala.Tuple2<org.apache.hadoop.io.LongWritable,FrameBlock> |
FrameRDDConverterUtils.LongWritableToSerFunction.call(scala.Tuple2<org.apache.hadoop.io.LongWritable,FrameBlock> arg0) |
scala.Tuple2<Long,FrameBlock> |
FrameRDDConverterUtils.LongWritableFrameToLongFrameFunction.call(scala.Tuple2<org.apache.hadoop.io.LongWritable,FrameBlock> arg0) |
static List<scala.Tuple2<Long,FrameBlock>> |
SparkUtils.fromIndexedFrameBlock(List<Pair<Long,FrameBlock>> in) |
static scala.Tuple2<Long,FrameBlock> |
SparkUtils.fromIndexedFrameBlock(Pair<Long,FrameBlock> in) |
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> |
FrameRDDAggregateUtils.mergeByKey(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> in) |
static Pair<Long,FrameBlock> |
SparkUtils.toIndexedFrameBlock(scala.Tuple2<Long,FrameBlock> in) |
Modifier and Type | Method and Description |
---|---|
FrameBlock |
FrameBlock.append(FrameBlock that,
FrameBlock ret,
boolean cbind)
Appends the given argument frameblock 'that' to this frameblock by
creating a deep copy to prevent side effects.
|
FrameBlock |
FrameBlock.leftIndexingOperations(FrameBlock rhsFrame,
org.apache.sysml.runtime.util.IndexRange ixrange,
FrameBlock ret) |
FrameBlock |
FrameBlock.leftIndexingOperations(FrameBlock rhsFrame,
int rl,
int ru,
int cl,
int cu,
FrameBlock ret) |
FrameBlock |
FrameBlock.slice(org.apache.sysml.runtime.util.IndexRange ixrange,
FrameBlock ret) |
FrameBlock |
FrameBlock.slice(int rl,
int ru,
int cl,
int cu,
org.apache.sysml.runtime.controlprogram.caching.CacheBlock retCache)
Right indexing operations to slice a subframe out of this frame block.
|
FrameBlock |
FrameBlock.zeroOutOperations(FrameBlock result,
org.apache.sysml.runtime.util.IndexRange range,
boolean complementary,
int iRowStartSrc,
int iRowStartDest,
int brlen,
int iMaxRowsToCopy)
This function ZERO OUT the data in the slicing window applicable for this block.
|
Modifier and Type | Method and Description |
---|---|
FrameBlock |
FrameBlock.append(FrameBlock that,
FrameBlock ret,
boolean cbind)
Appends the given argument frameblock 'that' to this frameblock by
creating a deep copy to prevent side effects.
|
void |
FrameBlock.copy(FrameBlock src) |
void |
FrameBlock.copy(int rl,
int ru,
int cl,
int cu,
FrameBlock src) |
FrameBlock |
FrameBlock.leftIndexingOperations(FrameBlock rhsFrame,
org.apache.sysml.runtime.util.IndexRange ixrange,
FrameBlock ret) |
FrameBlock |
FrameBlock.leftIndexingOperations(FrameBlock rhsFrame,
int rl,
int ru,
int cl,
int cu,
FrameBlock ret) |
void |
FrameBlock.merge(FrameBlock that) |
static ArrayList |
OperationsOnMatrixValues.performSlice(org.apache.sysml.runtime.util.IndexRange ixrange,
int brlen,
int bclen,
int iix,
int jix,
FrameBlock in) |
FrameBlock |
FrameBlock.slice(org.apache.sysml.runtime.util.IndexRange ixrange,
FrameBlock ret) |
FrameBlock |
FrameBlock.zeroOutOperations(FrameBlock result,
org.apache.sysml.runtime.util.IndexRange range,
boolean complementary,
int iRowStartSrc,
int iRowStartDest,
int brlen,
int iMaxRowsToCopy)
This function ZERO OUT the data in the slicing window applicable for this block.
|
Modifier and Type | Method and Description |
---|---|
static void |
OperationsOnMatrixValues.performShift(Pair<Long,FrameBlock> in,
org.apache.sysml.runtime.util.IndexRange ixrange,
int brlenLeft,
int clenLeft,
long rlen,
long clen,
ArrayList<Pair<Long,FrameBlock>> outlist) |
static void |
OperationsOnMatrixValues.performShift(Pair<Long,FrameBlock> in,
org.apache.sysml.runtime.util.IndexRange ixrange,
int brlenLeft,
int clenLeft,
long rlen,
long clen,
ArrayList<Pair<Long,FrameBlock>> outlist) |
static void |
OperationsOnMatrixValues.performSlice(Pair<Long,FrameBlock> in,
org.apache.sysml.runtime.util.IndexRange ixrange,
int brlen,
int bclen,
ArrayList<Pair<Long,FrameBlock>> outlist)
This function will get slice of the input frame block overlapping in overall slice(Range), slice has requested for.
|
static void |
OperationsOnMatrixValues.performSlice(Pair<Long,FrameBlock> in,
org.apache.sysml.runtime.util.IndexRange ixrange,
int brlen,
int bclen,
ArrayList<Pair<Long,FrameBlock>> outlist)
This function will get slice of the input frame block overlapping in overall slice(Range), slice has requested for.
|
void |
FrameBlock.slice(ArrayList<Pair<Long,FrameBlock>> outlist,
org.apache.sysml.runtime.util.IndexRange range,
int rowCut) |
Constructor and Description |
---|
FrameBlock(FrameBlock that)
Copy constructor for frame blocks, which uses a shallow copy for
the schema (column types and names) but a deep copy for meta data
and actual column data.
|
Copyright © 2018 The Apache Software Foundation. All rights reserved.