Uses of Class
org.apache.sysds.runtime.frame.data.FrameBlock
-
-
Uses of FrameBlock in org.apache.sysds.api.jmlc
Methods in org.apache.sysds.api.jmlc that return FrameBlock Modifier and Type Method Description FrameBlock
Connection. convertToFrame(InputStream input, int rows, int cols)
Converts an input stream of a string frame in textcell format into a frame block.FrameBlock
Connection. convertToFrame(InputStream input, int rows, int cols, String format)
Converts an input stream of a frame in csv or textcell format into a frame block.FrameBlock
Connection. convertToFrame(InputStream input, String meta)
Converts an input stream of a string frame in csv or textcell format into a frame block.FrameBlock
Connection. convertToFrame(String input, int rows, int cols)
Converts an input string representation of a frame in textcell format into a frame block.FrameBlock
Connection. convertToFrame(String input, String meta)
Converts an input string representation of a frame in csv or textcell format into a frame block.FrameBlock
ResultVariables. getFrameBlock(String varname)
Obtain the frame represented by the given output variable.FrameBlock
Connection. readTransformMetaDataFromFile(String metapath)
Reads transform meta data from an HDFS file path and converts it into an in-memory FrameBlock object.FrameBlock
Connection. readTransformMetaDataFromFile(String spec, String metapath)
Reads transform meta data from an HDFS file path and converts it into an in-memory FrameBlock object.FrameBlock
Connection. readTransformMetaDataFromFile(String spec, String metapath, String colDelim)
Reads transform meta data from an HDFS file path and converts it into an in-memory FrameBlock object.FrameBlock
Connection. readTransformMetaDataFromPath(String metapath)
Reads transform meta data from the class path and converts it into an in-memory FrameBlock object.FrameBlock
Connection. readTransformMetaDataFromPath(String spec, String metapath)
Reads transform meta data from the class path and converts it into an in-memory FrameBlock object.FrameBlock
Connection. readTransformMetaDataFromPath(String spec, String metapath, String colDelim)
Reads transform meta data from the class path and converts it into an in-memory FrameBlock object.Methods in org.apache.sysds.api.jmlc with parameters of type FrameBlock Modifier and Type Method Description void
PreparedScript. setFrame(String varname, FrameBlock frame, boolean reuse)
Binds a frame object to a registered input variable. -
Uses of FrameBlock in org.apache.sysds.api.mlcontext
Methods in org.apache.sysds.api.mlcontext that return types with arguments of type FrameBlock Modifier and Type Method Description static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>
MLContextConversionUtil. dataFrameToFrameBinaryBlocks(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, FrameMetadata frameMetadata)
Convert aDataFrame
to aJavaPairRDD<Long, FrameBlock>
binary-block frame.static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>
MLContextConversionUtil. frameObjectToBinaryBlocks(FrameObject frameObject, SparkExecutionContext sparkExecutionContext)
Convert aFrameObject
to aJavaPairRDD<Long, FrameBlock>
.org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>
Frame. toBinaryBlocks()
Obtain the frame as aJavaPairRDD<Long, FrameBlock>
Methods in org.apache.sysds.api.mlcontext with parameters of type FrameBlock Modifier and Type Method Description static FrameObject
MLContextConversionUtil. frameBlockToFrameObject(String variableName, FrameBlock frameBlock, FrameMetadata frameMetadata)
Convert aFrameBlock
to aFrameObject
.Method parameters in org.apache.sysds.api.mlcontext with type arguments of type FrameBlock Modifier and Type Method Description static FrameObject
MLContextConversionUtil. binaryBlocksToFrameObject(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> binaryBlocks)
Convert aJavaPairRDD<Long, FrameBlock>
to aFrameObject
.static FrameObject
MLContextConversionUtil. binaryBlocksToFrameObject(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> binaryBlocks, FrameMetadata frameMetadata)
Convert aJavaPairRDD<Long, FrameBlock>
to aFrameObject
.Constructor parameters in org.apache.sysds.api.mlcontext with type arguments of type FrameBlock Constructor Description Frame(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> binaryBlocks, FrameMetadata frameMetadata)
Create a Frame, specifying the SystemDS binary-block frame and its metadata. -
Uses of FrameBlock in org.apache.sysds.runtime.compress.lib
Methods in org.apache.sysds.runtime.compress.lib that return types with arguments of type FrameBlock Modifier and Type Method Description static org.apache.commons.lang3.tuple.Pair<MatrixBlock,FrameBlock>
CLALibBinCompress. binCompress(CacheBlock<?> X, MatrixBlock d, int k)
-
Uses of FrameBlock in org.apache.sysds.runtime.controlprogram.context
Methods in org.apache.sysds.runtime.controlprogram.context that return FrameBlock Modifier and Type Method Description FrameBlock
ExecutionContext. getFrameInput(String varName)
Pins a frame variable into memory and returns the internal frame block.static FrameBlock
SparkExecutionContext. toFrameBlock(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> rdd, Types.ValueType[] schema, int rlen, int clen)
static FrameBlock
SparkExecutionContext. toFrameBlock(RDDObject rdd, Types.ValueType[] schema, int rlen, int clen)
Methods in org.apache.sysds.runtime.controlprogram.context that return types with arguments of type FrameBlock Modifier and Type Method Description PartitionedBroadcast<FrameBlock>
SparkExecutionContext. getBroadcastForFrameVariable(String varname)
org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>
SparkExecutionContext. getFrameBinaryBlockRDDHandleForVariable(String varname)
Spark instructions should call this for all frame inputs except broadcast variables.static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>
SparkExecutionContext. toFrameJavaPairRDD(org.apache.spark.api.java.JavaSparkContext sc, FrameBlock src)
Methods in org.apache.sysds.runtime.controlprogram.context with parameters of type FrameBlock Modifier and Type Method Description static FrameObject
ExecutionContext. createFrameObject(FrameBlock fb)
void
ExecutionContext. setFrameOutput(String varName, FrameBlock outputData)
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>
SparkExecutionContext. toFrameJavaPairRDD(org.apache.spark.api.java.JavaSparkContext sc, FrameBlock src)
Method parameters in org.apache.sysds.runtime.controlprogram.context with type arguments of type FrameBlock Modifier and Type Method Description static FrameBlock
SparkExecutionContext. toFrameBlock(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> rdd, Types.ValueType[] schema, int rlen, int clen)
-
Uses of FrameBlock in org.apache.sysds.runtime.frame.data
Methods in org.apache.sysds.runtime.frame.data that return FrameBlock Modifier and Type Method Description FrameBlock
FrameBlock. append(FrameBlock that, boolean cbind)
Appends the given argument FrameBlock 'that' to this FrameBlock by creating a deep copy to prevent side effects.FrameBlock
FrameBlock. applySchema(FrameBlock schema)
FrameBlock
FrameBlock. applySchema(FrameBlock schema, int k)
FrameBlock
FrameBlock. binaryOperations(BinaryOperator bop, FrameBlock that, FrameBlock out)
This method performs the value comparison on two frames if the values in both frames are equal, not equal, less than, greater than, less than/greater than and equal to the output frame will store boolean value for each each comparisonstatic FrameBlock
FrameBlock. convertToFrameBlock(MatrixBlock mb, Types.ValueType[] schema, int k)
FrameBlock
FrameBlock. copy()
FrameBlock
FrameBlock. detectSchema(double sampleFraction, int k)
FrameBlock
FrameBlock. detectSchema(int k)
FrameBlock
FrameBlock. dropInvalidType(FrameBlock schema)
Drop the cell value which does not confirms to the data type of its columnFrameBlock
FrameBlock. frameRowReplication(FrameBlock rowToreplicate)
FrameBlock
FrameBlock. getColumnNamesAsFrame()
FrameBlock
FrameBlock. getSchemaTypeOf()
FrameBlock
FrameBlock. invalidByLength(MatrixBlock feaLen)
This method validates the frame data against an attribute length constrain if data value in any cell is greater than the specified threshold of that attribute the output frame will store a null on that cell position, thus removing the length-violating values.FrameBlock
FrameBlock. leftIndexingOperations(FrameBlock rhsFrame, int rl, int ru, int cl, int cu, FrameBlock ret)
FrameBlock
FrameBlock. leftIndexingOperations(FrameBlock rhsFrame, IndexRange ixrange, FrameBlock ret)
FrameBlock
FrameBlock. map(String lambdaExpr, long margin)
FrameBlock
FrameBlock. map(FrameBlock.FrameMapFunction lambdaExpr, long margin)
FrameBlock
FrameBlock. mapDist(FrameBlock.FrameMapFunction lambdaExpr)
FrameBlock
FrameBlock. merge(FrameBlock that)
FrameBlock
FrameBlock. merge(FrameBlock that, boolean appendOnly)
FrameBlock
FrameBlock. removeEmptyOperations(boolean rows, boolean emptyReturn, MatrixBlock select)
<T> FrameBlock
FrameBlock. replaceOperations(String pattern, String replacement)
FrameBlock
FrameBlock. slice(int rl, int ru)
FrameBlock
FrameBlock. slice(int rl, int ru, boolean deep)
FrameBlock
FrameBlock. slice(int rl, int ru, int cl, int cu)
FrameBlock
FrameBlock. slice(int rl, int ru, int cl, int cu, boolean deep)
FrameBlock
FrameBlock. slice(int rl, int ru, int cl, int cu, boolean deep, FrameBlock ret)
FrameBlock
FrameBlock. slice(int rl, int ru, int cl, int cu, FrameBlock ret)
FrameBlock
FrameBlock. slice(IndexRange ixrange, FrameBlock ret)
FrameBlock
FrameBlock. valueSwap(FrameBlock schema)
FrameBlock
FrameBlock. zeroOutOperations(FrameBlock result, IndexRange range, boolean complementary, int iRowStartSrc, int iRowStartDest, int blen, int iMaxRowsToCopy)
This function ZERO OUT the data in the slicing window applicable for this block.Methods in org.apache.sysds.runtime.frame.data with parameters of type FrameBlock Modifier and Type Method Description FrameBlock
FrameBlock. append(FrameBlock that, boolean cbind)
Appends the given argument FrameBlock 'that' to this FrameBlock by creating a deep copy to prevent side effects.FrameBlock
FrameBlock. applySchema(FrameBlock schema)
FrameBlock
FrameBlock. applySchema(FrameBlock schema, int k)
FrameBlock
FrameBlock. binaryOperations(BinaryOperator bop, FrameBlock that, FrameBlock out)
This method performs the value comparison on two frames if the values in both frames are equal, not equal, less than, greater than, less than/greater than and equal to the output frame will store boolean value for each each comparisonvoid
FrameBlock. copy(int rl, int ru, int cl, int cu, FrameBlock src)
Copy src matrix into the index range of the existing current matrix.void
FrameBlock. copy(FrameBlock src)
FrameBlock
FrameBlock. dropInvalidType(FrameBlock schema)
Drop the cell value which does not confirms to the data type of its columnFrameBlock
FrameBlock. frameRowReplication(FrameBlock rowToreplicate)
FrameBlock
FrameBlock. leftIndexingOperations(FrameBlock rhsFrame, int rl, int ru, int cl, int cu, FrameBlock ret)
FrameBlock
FrameBlock. leftIndexingOperations(FrameBlock rhsFrame, IndexRange ixrange, FrameBlock ret)
FrameBlock
FrameBlock. merge(FrameBlock that)
FrameBlock
FrameBlock. merge(FrameBlock that, boolean appendOnly)
FrameBlock
FrameBlock. slice(int rl, int ru, int cl, int cu, boolean deep, FrameBlock ret)
FrameBlock
FrameBlock. slice(int rl, int ru, int cl, int cu, FrameBlock ret)
FrameBlock
FrameBlock. slice(IndexRange ixrange, FrameBlock ret)
FrameBlock
FrameBlock. valueSwap(FrameBlock schema)
FrameBlock
FrameBlock. zeroOutOperations(FrameBlock result, IndexRange range, boolean complementary, int iRowStartSrc, int iRowStartDest, int blen, int iMaxRowsToCopy)
This function ZERO OUT the data in the slicing window applicable for this block.Method parameters in org.apache.sysds.runtime.frame.data with type arguments of type FrameBlock Modifier and Type Method Description void
FrameBlock. slice(ArrayList<Pair<Long,FrameBlock>> outList, IndexRange range, int rowCut)
Constructors in org.apache.sysds.runtime.frame.data with parameters of type FrameBlock Constructor Description FrameBlock(FrameBlock that)
Copy constructor for frame blocks, which uses a shallow copy for the schema (column types and names) but a deep copy for meta data and actual column data. -
Uses of FrameBlock in org.apache.sysds.runtime.frame.data.compress
Methods in org.apache.sysds.runtime.frame.data.compress that return FrameBlock Modifier and Type Method Description static FrameBlock
CompressedFrameBlockFactory. compress(FrameBlock fb)
static FrameBlock
CompressedFrameBlockFactory. compress(FrameBlock fb, int k, WTreeRoot root)
static FrameBlock
CompressedFrameBlockFactory. compress(FrameBlock fb, FrameCompressionSettings cs)
static FrameBlock
CompressedFrameBlockFactory. compress(FrameBlock fb, FrameCompressionSettingsBuilder csb)
Methods in org.apache.sysds.runtime.frame.data.compress with parameters of type FrameBlock Modifier and Type Method Description static FrameBlock
CompressedFrameBlockFactory. compress(FrameBlock fb)
static FrameBlock
CompressedFrameBlockFactory. compress(FrameBlock fb, int k, WTreeRoot root)
static FrameBlock
CompressedFrameBlockFactory. compress(FrameBlock fb, FrameCompressionSettings cs)
static FrameBlock
CompressedFrameBlockFactory. compress(FrameBlock fb, FrameCompressionSettingsBuilder csb)
-
Uses of FrameBlock in org.apache.sysds.runtime.frame.data.iterators
Methods in org.apache.sysds.runtime.frame.data.iterators with parameters of type FrameBlock Modifier and Type Method Description static RowIterator<Object>
IteratorFactory. getObjectRowIterator(FrameBlock fb)
Get a row iterator over the frame where all fields are encoded as boxed objects according to their value types.static RowIterator<Object>
IteratorFactory. getObjectRowIterator(FrameBlock fb, int colID)
Get a row iterator over the frame where all selected fields are encoded as objects according to their value types.static RowIterator<Object>
IteratorFactory. getObjectRowIterator(FrameBlock fb, int[] cols)
Get a row iterator over the frame where all selected fields are encoded as boxed objects according to their value types.static RowIterator<Object>
IteratorFactory. getObjectRowIterator(FrameBlock fb, int rl, int ru)
Get a row iterator over the frame where all fields are encoded as boxed objects according to their value types.static RowIterator<Object>
IteratorFactory. getObjectRowIterator(FrameBlock fb, int rl, int ru, int colID)
Get a row iterator over the frame where all selected fields are encoded as boxed objects according to their value types.static RowIterator<Object>
IteratorFactory. getObjectRowIterator(FrameBlock fb, int rl, int ru, int[] cols)
Get a row iterator over the frame where all selected fields are encoded as boxed objects according to their value types.static RowIterator<Object>
IteratorFactory. getObjectRowIterator(FrameBlock fb, Types.ValueType[] schema)
Get a row iterator over the frame where all fields are encoded as boxed objects according to the value types of the provided target schema.static RowIterator<String>
IteratorFactory. getStringRowIterator(FrameBlock fb)
Get a row iterator over the frame where all fields are encoded as strings independent of their value types.static RowIterator<String>
IteratorFactory. getStringRowIterator(FrameBlock fb, int colID)
Get a row iterator over the frame where all selected fields are encoded as strings independent of their value types.static RowIterator<String>
IteratorFactory. getStringRowIterator(FrameBlock fb, int[] cols)
Get a row iterator over the frame where all selected fields are encoded as strings independent of their value types.static RowIterator<String>
IteratorFactory. getStringRowIterator(FrameBlock fb, int rl, int ru)
Get a row iterator over the frame where all fields are encoded as strings independent of their value types.static RowIterator<String>
IteratorFactory. getStringRowIterator(FrameBlock fb, int rl, int ru, int colID)
Get a row iterator over the frame where all selected fields are encoded as strings independent of their value types.static RowIterator<String>
IteratorFactory. getStringRowIterator(FrameBlock fb, int rl, int ru, int[] cols)
Get a row iterator over the frame where all selected fields are encoded as strings independent of their value types. -
Uses of FrameBlock in org.apache.sysds.runtime.frame.data.lib
Methods in org.apache.sysds.runtime.frame.data.lib that return FrameBlock Modifier and Type Method Description static FrameBlock
FrameLibAppend. append(FrameBlock a, FrameBlock b, boolean cbind)
Appends the given argument FrameBlock 'that' to this FrameBlock by creating a deep copy to prevent side effects.static FrameBlock
FrameLibAppend. appendCbind(FrameBlock a, FrameBlock b)
static FrameBlock
FrameLibAppend. appendRbind(FrameBlock a, FrameBlock b)
static FrameBlock
FrameLibApplySchema. applySchema(FrameBlock fb, Types.ValueType[] schema)
Method to create a new FrameBlock where the given schema is applied.static FrameBlock
FrameLibApplySchema. applySchema(FrameBlock fb, Types.ValueType[] schema, int k)
Method to create a new FrameBlock where the given schema is applied, k is the parallelization degree.static FrameBlock
FrameLibApplySchema. applySchema(FrameBlock fb, FrameBlock schema)
static FrameBlock
FrameLibApplySchema. applySchema(FrameBlock fb, FrameBlock schema, int k)
Method to create a new FrameBlock where the given schema is applied, k is the parallelization degree.static FrameBlock
FrameLibCompress. compress(FrameBlock in, int k)
static FrameBlock
FrameLibCompress. compress(FrameBlock in, int k, WTreeRoot root)
static FrameBlock
FrameFromMatrixBlock. convertToFrameBlock(MatrixBlock mb, int k)
static FrameBlock
FrameFromMatrixBlock. convertToFrameBlock(MatrixBlock mb, Types.ValueType[] schema, int k)
static FrameBlock
FrameFromMatrixBlock. convertToFrameBlock(MatrixBlock mb, Types.ValueType vt, int k)
static FrameBlock
FrameLibDetectSchema. detectSchema(FrameBlock in, double sampleFraction, int k)
static FrameBlock
FrameLibDetectSchema. detectSchema(FrameBlock in, int k)
static FrameBlock
FrameUtil. mergeSchema(FrameBlock temp1, FrameBlock temp2)
static FrameBlock
FrameLibRemoveEmpty. removeEmpty(FrameBlock fb, boolean rows, boolean emptyReturn, MatrixBlock select)
Methods in org.apache.sysds.runtime.frame.data.lib with parameters of type FrameBlock Modifier and Type Method Description static FrameBlock
FrameLibAppend. append(FrameBlock a, FrameBlock b, boolean cbind)
Appends the given argument FrameBlock 'that' to this FrameBlock by creating a deep copy to prevent side effects.static FrameBlock
FrameLibAppend. appendCbind(FrameBlock a, FrameBlock b)
static FrameBlock
FrameLibAppend. appendRbind(FrameBlock a, FrameBlock b)
static FrameBlock
FrameLibApplySchema. applySchema(FrameBlock fb, Types.ValueType[] schema)
Method to create a new FrameBlock where the given schema is applied.static FrameBlock
FrameLibApplySchema. applySchema(FrameBlock fb, Types.ValueType[] schema, int k)
Method to create a new FrameBlock where the given schema is applied, k is the parallelization degree.static FrameBlock
FrameLibApplySchema. applySchema(FrameBlock fb, FrameBlock schema)
static FrameBlock
FrameLibApplySchema. applySchema(FrameBlock fb, FrameBlock schema, int k)
Method to create a new FrameBlock where the given schema is applied, k is the parallelization degree.static FrameBlock
FrameLibCompress. compress(FrameBlock in, int k)
static FrameBlock
FrameLibCompress. compress(FrameBlock in, int k, WTreeRoot root)
static MatrixBlock
MatrixBlockFromFrame. convertToMatrixBlock(FrameBlock frame, int k)
Converts a frame block with arbitrary schema into a matrix block.static FrameBlock
FrameLibDetectSchema. detectSchema(FrameBlock in, double sampleFraction, int k)
static FrameBlock
FrameLibDetectSchema. detectSchema(FrameBlock in, int k)
static FrameBlock
FrameUtil. mergeSchema(FrameBlock temp1, FrameBlock temp2)
static FrameBlock
FrameLibRemoveEmpty. removeEmpty(FrameBlock fb, boolean rows, boolean emptyReturn, MatrixBlock select)
-
Uses of FrameBlock in org.apache.sysds.runtime.instructions.fed
Constructors in org.apache.sysds.runtime.instructions.fed with parameters of type FrameBlock Constructor Description DecodeMatrix(long input, long outputID, FrameBlock meta, Decoder decoder)
-
Uses of FrameBlock in org.apache.sysds.runtime.instructions.spark
Methods in org.apache.sysds.runtime.instructions.spark that return FrameBlock Modifier and Type Method Description FrameBlock
ParameterizedBuiltinSPInstruction.RDDFrameReplaceFunction. call(FrameBlock arg0)
Methods in org.apache.sysds.runtime.instructions.spark that return types with arguments of type FrameBlock Modifier and Type Method Description scala.Tuple2<Long,FrameBlock>
ParameterizedBuiltinSPInstruction.RDDTokenizeFunction. call(scala.Tuple2<Long,FrameBlock> in)
scala.Tuple2<Long,FrameBlock>
ParameterizedBuiltinSPInstruction.RDDTransformApplyFunction. call(scala.Tuple2<Long,FrameBlock> in)
scala.Tuple2<Long,FrameBlock>
ParameterizedBuiltinSPInstruction.RDDTransformDecodeFunction. call(scala.Tuple2<MatrixIndexes,MatrixBlock> in)
Methods in org.apache.sysds.runtime.instructions.spark with parameters of type FrameBlock Modifier and Type Method Description FrameBlock
ParameterizedBuiltinSPInstruction.RDDFrameReplaceFunction. call(FrameBlock arg0)
Method parameters in org.apache.sysds.runtime.instructions.spark with type arguments of type FrameBlock Modifier and Type Method Description Iterator<scala.Tuple2<Integer,ColumnMetadata>>
MultiReturnParameterizedBuiltinSPInstruction.TransformEncodeBuild2Function. call(Iterator<scala.Tuple2<Long,FrameBlock>> iter)
Iterator<scala.Tuple2<Integer,Object>>
MultiReturnParameterizedBuiltinSPInstruction.TransformEncodeBuildFunction. call(Iterator<scala.Tuple2<Long,FrameBlock>> iter)
scala.Tuple2<Long,FrameBlock>
ParameterizedBuiltinSPInstruction.RDDTokenizeFunction. call(scala.Tuple2<Long,FrameBlock> in)
scala.Tuple2<Long,FrameBlock>
ParameterizedBuiltinSPInstruction.RDDTransformApplyFunction. call(scala.Tuple2<Long,FrameBlock> in)
scala.Tuple2<Long,MatrixBlock>
ParameterizedBuiltinSPInstruction.RDDTransformApplyFunction2. call(scala.Tuple2<Long,FrameBlock> in)
scala.Tuple2<Long,Long>
ParameterizedBuiltinSPInstruction.RDDTransformApplyOffsetFunction. call(scala.Tuple2<Long,FrameBlock> in)
-
Uses of FrameBlock in org.apache.sysds.runtime.instructions.spark.data
Method parameters in org.apache.sysds.runtime.instructions.spark.data with type arguments of type FrameBlock Modifier and Type Method Description void
FrameReblockBuffer. flushBufferToBinaryBlocks(ArrayList<Pair<Long,FrameBlock>> outList)
-
Uses of FrameBlock in org.apache.sysds.runtime.instructions.spark.functions
Methods in org.apache.sysds.runtime.instructions.spark.functions that return FrameBlock Modifier and Type Method Description FrameBlock
CopyFrameBlockFunction. call(FrameBlock arg0)
Methods in org.apache.sysds.runtime.instructions.spark.functions that return types with arguments of type FrameBlock Modifier and Type Method Description scala.Tuple2<Long,FrameBlock>
CopyFrameBlockPairFunction. call(scala.Tuple2<org.apache.hadoop.io.LongWritable,FrameBlock> arg0)
Methods in org.apache.sysds.runtime.instructions.spark.functions with parameters of type FrameBlock Modifier and Type Method Description FrameBlock
CopyFrameBlockFunction. call(FrameBlock arg0)
Method parameters in org.apache.sysds.runtime.instructions.spark.functions with type arguments of type FrameBlock Modifier and Type Method Description Iterator<String>
ConvertFrameBlockToIJVLines. call(scala.Tuple2<Long,FrameBlock> kv)
scala.Tuple2<Long,FrameBlock>
CopyFrameBlockPairFunction. call(scala.Tuple2<org.apache.hadoop.io.LongWritable,FrameBlock> arg0)
Boolean
IsFrameBlockInRange. call(scala.Tuple2<Long,FrameBlock> kv)
-
Uses of FrameBlock in org.apache.sysds.runtime.instructions.spark.utils
Methods in org.apache.sysds.runtime.instructions.spark.utils that return types with arguments of type FrameBlock Modifier and Type Method Description scala.Tuple2<org.apache.hadoop.io.LongWritable,FrameBlock>
FrameRDDConverterUtils.LongFrameToLongWritableFrameFunction. call(scala.Tuple2<Long,FrameBlock> arg0)
scala.Tuple2<Long,FrameBlock>
FrameRDDConverterUtils.LongWritableFrameToLongFrameFunction. call(scala.Tuple2<org.apache.hadoop.io.LongWritable,FrameBlock> arg0)
scala.Tuple2<org.apache.hadoop.io.LongWritable,FrameBlock>
FrameRDDConverterUtils.LongWritableToSerFunction. call(scala.Tuple2<org.apache.hadoop.io.LongWritable,FrameBlock> arg0)
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>
FrameRDDConverterUtils. csvToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> input, DataCharacteristics mc, Types.ValueType[] schema, boolean hasHeader, String delim, boolean fill, double fillValue, Set<String> naStrings)
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>
FrameRDDConverterUtils. csvToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaRDD<String> input, DataCharacteristics mcOut, Types.ValueType[] schema, boolean hasHeader, String delim, boolean fill, double fillValue, Set<String> naStrings)
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>
FrameRDDConverterUtils. dataFrameToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> df, DataCharacteristics mc, boolean containsID)
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>
FrameRDDConverterUtils. dataFrameToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> df, DataCharacteristics mc, boolean containsID, Pair<String[],Types.ValueType[]> out)
static List<scala.Tuple2<Long,FrameBlock>>
SparkUtils. fromIndexedFrameBlock(List<Pair<Long,FrameBlock>> in)
static scala.Tuple2<Long,FrameBlock>
SparkUtils. fromIndexedFrameBlock(Pair<Long,FrameBlock> in)
static org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.LongWritable,FrameBlock>
FrameRDDConverterUtils. matrixBlockToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> input, DataCharacteristics mcIn)
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>
FrameRDDConverterUtils. matrixBlockToBinaryBlockLongIndex(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> input, DataCharacteristics dcIn)
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>
FrameRDDAggregateUtils. mergeByKey(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> in)
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>
FrameRDDConverterUtils. textCellToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> in, DataCharacteristics mcOut, Types.ValueType[] schema)
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>
FrameRDDConverterUtils. textCellToBinaryBlockLongIndex(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaPairRDD<Long,org.apache.hadoop.io.Text> input, DataCharacteristics mc, Types.ValueType[] schema)
static Pair<Long,FrameBlock>
SparkUtils. toIndexedFrameBlock(scala.Tuple2<Long,FrameBlock> in)
Method parameters in org.apache.sysds.runtime.instructions.spark.utils with type arguments of type FrameBlock Modifier and Type Method Description static org.apache.spark.api.java.JavaRDD<String>
FrameRDDConverterUtils. binaryBlockToCsv(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> in, DataCharacteristics mcIn, FileFormatPropertiesCSV props, boolean strict)
static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>
FrameRDDConverterUtils. binaryBlockToDataFrame(org.apache.spark.sql.SparkSession sparkSession, org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> in, DataCharacteristics mc, Types.ValueType[] schema)
static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>
FrameRDDConverterUtils. binaryBlockToDataFrame(org.apache.spark.sql.SQLContext sqlContext, org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> in, DataCharacteristics mc, Types.ValueType[] schema)
Deprecated.static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>
FrameRDDConverterUtils. binaryBlockToMatrixBlock(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> input, DataCharacteristics mcIn, DataCharacteristics mcOut)
static org.apache.spark.api.java.JavaRDD<String>
FrameRDDConverterUtils. binaryBlockToTextCell(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> input, DataCharacteristics mcIn)
scala.Tuple2<org.apache.hadoop.io.LongWritable,FrameBlock>
FrameRDDConverterUtils.LongFrameToLongWritableFrameFunction. call(scala.Tuple2<Long,FrameBlock> arg0)
scala.Tuple2<Long,FrameBlock>
FrameRDDConverterUtils.LongWritableFrameToLongFrameFunction. call(scala.Tuple2<org.apache.hadoop.io.LongWritable,FrameBlock> arg0)
scala.Tuple2<org.apache.hadoop.io.LongWritable,FrameBlock>
FrameRDDConverterUtils.LongWritableToSerFunction. call(scala.Tuple2<org.apache.hadoop.io.LongWritable,FrameBlock> arg0)
static scala.Tuple2<Boolean,Integer>
FrameRDDAggregateUtils. checkRowAlignment(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> in, int blen)
static List<scala.Tuple2<Long,FrameBlock>>
SparkUtils. fromIndexedFrameBlock(List<Pair<Long,FrameBlock>> in)
static scala.Tuple2<Long,FrameBlock>
SparkUtils. fromIndexedFrameBlock(Pair<Long,FrameBlock> in)
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>
FrameRDDAggregateUtils. mergeByKey(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> in)
static Pair<Long,FrameBlock>
SparkUtils. toIndexedFrameBlock(scala.Tuple2<Long,FrameBlock> in)
-
Uses of FrameBlock in org.apache.sysds.runtime.io
Methods in org.apache.sysds.runtime.io that return FrameBlock Modifier and Type Method Description FrameBlock
FrameReaderBinaryBlock. readFirstBlock(String fname)
Specific functionality of FrameReaderBinaryBlock, mostly used for testing.FrameBlock
FrameReader. readFrameFromHDFS(String fname, long rlen, long clen)
FrameBlock
FrameReader. readFrameFromHDFS(String fname, Types.ValueType[] schema, long rlen, long clen)
abstract FrameBlock
FrameReader. readFrameFromHDFS(String fname, Types.ValueType[] schema, String[] names, long rlen, long clen)
FrameBlock
FrameReaderBinaryBlock. readFrameFromHDFS(String fname, Types.ValueType[] schema, String[] names, long rlen, long clen)
FrameBlock
FrameReaderJSONL. readFrameFromHDFS(String fname, Types.ValueType[] schema, Map<String,Integer> schemaMap, long rlen, long clen)
FrameBlock
FrameReaderProto. readFrameFromHDFS(String fname, Types.ValueType[] schema, String[] names, long rlen, long clen)
FrameBlock
FrameReaderTextCell. readFrameFromHDFS(String fname, Types.ValueType[] schema, String[] names, long rlen, long clen)
FrameBlock
FrameReaderTextCSV. readFrameFromHDFS(String fname, Types.ValueType[] schema, String[] names, long rlen, long clen)
FrameBlock
FrameReader. readFrameFromInputStream(InputStream is, long rlen, long clen)
FrameBlock
FrameReader. readFrameFromInputStream(InputStream is, Types.ValueType[] schema, long rlen, long clen)
abstract FrameBlock
FrameReader. readFrameFromInputStream(InputStream is, Types.ValueType[] schema, String[] names, long rlen, long clen)
FrameBlock
FrameReaderBinaryBlock. readFrameFromInputStream(InputStream is, Types.ValueType[] schema, String[] names, long rlen, long clen)
FrameBlock
FrameReaderProto. readFrameFromInputStream(InputStream is, Types.ValueType[] schema, String[] names, long rlen, long clen)
FrameBlock
FrameReaderTextCell. readFrameFromInputStream(InputStream is, Types.ValueType[] schema, String[] names, long rlen, long clen)
FrameBlock
FrameReaderTextCSV. readFrameFromInputStream(InputStream is, Types.ValueType[] schema, String[] names, long rlen, long clen)
Methods in org.apache.sysds.runtime.io with parameters of type FrameBlock Modifier and Type Method Description abstract void
FrameWriter. writeFrameToHDFS(FrameBlock src, String fname, long rlen, long clen)
void
FrameWriterBinaryBlock. writeFrameToHDFS(FrameBlock src, String fname, long rlen, long clen)
void
FrameWriterJSONL. writeFrameToHDFS(FrameBlock src, String fname, Map<String,Integer> schemaMap, long rlen, long clen)
void
FrameWriterProto. writeFrameToHDFS(FrameBlock src, String fname, long rlen, long clen)
void
FrameWriterTextCell. writeFrameToHDFS(FrameBlock src, String fname, long rlen, long clen)
void
FrameWriterTextCSV. writeFrameToHDFS(FrameBlock src, String fname, long rlen, long clen)
Constructors in org.apache.sysds.runtime.io with parameters of type FrameBlock Constructor Description ReadTask(org.apache.hadoop.mapred.InputSplit split, org.apache.hadoop.mapred.TextInputFormat informat, org.apache.hadoop.mapred.JobConf job, FrameBlock dest)
-
Uses of FrameBlock in org.apache.sysds.runtime.iogen
Methods in org.apache.sysds.runtime.iogen that return FrameBlock Modifier and Type Method Description FrameBlock
SampleProperties. getSampleFrame()
Methods in org.apache.sysds.runtime.iogen with parameters of type FrameBlock Modifier and Type Method Description void
SampleProperties. setSampleFrame(FrameBlock sampleFrame)
Constructors in org.apache.sysds.runtime.iogen with parameters of type FrameBlock Constructor Description FormatIdentifyer(String raw, FrameBlock frame)
GenerateReaderFrame(String sampleRaw, FrameBlock sampleFrame, boolean parallel)
ReaderMapping(String raw, FrameBlock frame)
ReaderMappingIndex(String raw, FrameBlock frame)
SampleProperties(String sampleRaw, FrameBlock sampleFrame)
-
Uses of FrameBlock in org.apache.sysds.runtime.iogen.template
Methods in org.apache.sysds.runtime.iogen.template that return FrameBlock Modifier and Type Method Description FrameBlock
FrameGenerateReader. readFrameFromHDFS(String fname, Types.ValueType[] schema, String[] names, long rlen, long clen)
FrameBlock
FrameGenerateReaderParallel. readFrameFromHDFS(String fname, Types.ValueType[] schema, String[] names, long rlen, long clen)
FrameBlock
FrameGenerateReader. readFrameFromInputStream(InputStream is, Types.ValueType[] schema, String[] names, long rlen, long clen)
FrameBlock
FrameGenerateReaderParallel. readFrameFromInputStream(InputStream is, Types.ValueType[] schema, String[] names, long rlen, long clen)
-
Uses of FrameBlock in org.apache.sysds.runtime.matrix.data
Methods in org.apache.sysds.runtime.matrix.data that return types with arguments of type FrameBlock Modifier and Type Method Description static ArrayList<Pair<Long,FrameBlock>>
OperationsOnMatrixValues. performSlice(Pair<Long,FrameBlock> in, IndexRange ixrange, int blen)
This function will get slice of the input frame block overlapping in overall slice(Range), slice has requested for.Methods in org.apache.sysds.runtime.matrix.data with parameters of type FrameBlock Modifier and Type Method Description static ArrayList
OperationsOnMatrixValues. performSlice(IndexRange ixrange, int blen, int iix, int jix, FrameBlock in)
Method parameters in org.apache.sysds.runtime.matrix.data with type arguments of type FrameBlock Modifier and Type Method Description static void
OperationsOnMatrixValues. performShift(Pair<Long,FrameBlock> in, IndexRange ixrange, int blenLeft, long rlen, long clen, ArrayList<Pair<Long,FrameBlock>> outlist)
static void
OperationsOnMatrixValues. performShift(Pair<Long,FrameBlock> in, IndexRange ixrange, int blenLeft, long rlen, long clen, ArrayList<Pair<Long,FrameBlock>> outlist)
static ArrayList<Pair<Long,FrameBlock>>
OperationsOnMatrixValues. performSlice(Pair<Long,FrameBlock> in, IndexRange ixrange, int blen)
This function will get slice of the input frame block overlapping in overall slice(Range), slice has requested for. -
Uses of FrameBlock in org.apache.sysds.runtime.transform.decode
Methods in org.apache.sysds.runtime.transform.decode that return FrameBlock Modifier and Type Method Description abstract FrameBlock
Decoder. decode(MatrixBlock in, FrameBlock out)
Block decode API converting a matrix block into a frame block.FrameBlock
Decoder. decode(MatrixBlock in, FrameBlock out, int k)
Block decode API converting a matrix block into a frame block in parallel.FrameBlock
DecoderBin. decode(MatrixBlock in, FrameBlock out)
FrameBlock
DecoderComposite. decode(MatrixBlock in, FrameBlock out)
FrameBlock
DecoderComposite. decode(MatrixBlock in, FrameBlock out, int k)
FrameBlock
DecoderDummycode. decode(MatrixBlock in, FrameBlock out)
FrameBlock
DecoderPassThrough. decode(MatrixBlock in, FrameBlock out)
FrameBlock
DecoderRecode. decode(MatrixBlock in, FrameBlock out)
Methods in org.apache.sysds.runtime.transform.decode with parameters of type FrameBlock Modifier and Type Method Description static Decoder
DecoderFactory. createDecoder(String spec, String[] colnames, Types.ValueType[] schema, FrameBlock meta)
static Decoder
DecoderFactory. createDecoder(String spec, String[] colnames, Types.ValueType[] schema, FrameBlock meta, int clen)
static Decoder
DecoderFactory. createDecoder(String spec, String[] colnames, Types.ValueType[] schema, FrameBlock meta, int minCol, int maxCol)
static Decoder
DecoderFactory. createDecoder(String spec, String[] colnames, Types.ValueType[] schema, FrameBlock meta, int clen, int minCol, int maxCol)
abstract FrameBlock
Decoder. decode(MatrixBlock in, FrameBlock out)
Block decode API converting a matrix block into a frame block.FrameBlock
Decoder. decode(MatrixBlock in, FrameBlock out, int k)
Block decode API converting a matrix block into a frame block in parallel.abstract void
Decoder. decode(MatrixBlock in, FrameBlock out, int rl, int ru)
Block decode row blockFrameBlock
DecoderBin. decode(MatrixBlock in, FrameBlock out)
void
DecoderBin. decode(MatrixBlock in, FrameBlock out, int rl, int ru)
FrameBlock
DecoderComposite. decode(MatrixBlock in, FrameBlock out)
FrameBlock
DecoderComposite. decode(MatrixBlock in, FrameBlock out, int k)
void
DecoderComposite. decode(MatrixBlock in, FrameBlock out, int rl, int ru)
FrameBlock
DecoderDummycode. decode(MatrixBlock in, FrameBlock out)
void
DecoderDummycode. decode(MatrixBlock in, FrameBlock out, int rl, int ru)
FrameBlock
DecoderPassThrough. decode(MatrixBlock in, FrameBlock out)
void
DecoderPassThrough. decode(MatrixBlock in, FrameBlock out, int rl, int ru)
FrameBlock
DecoderRecode. decode(MatrixBlock in, FrameBlock out)
void
DecoderRecode. decode(MatrixBlock in, FrameBlock out, int rl, int ru)
abstract void
Decoder. initMetaData(FrameBlock meta)
void
DecoderBin. initMetaData(FrameBlock meta)
void
DecoderComposite. initMetaData(FrameBlock meta)
void
DecoderDummycode. initMetaData(FrameBlock meta)
void
DecoderPassThrough. initMetaData(FrameBlock meta)
void
DecoderRecode. initMetaData(FrameBlock meta)
-
Uses of FrameBlock in org.apache.sysds.runtime.transform.encode
Methods in org.apache.sysds.runtime.transform.encode that return FrameBlock Modifier and Type Method Description FrameBlock
ColumnEncoderBin. getMetaData(FrameBlock meta)
FrameBlock
ColumnEncoderComposite. getMetaData(FrameBlock out)
FrameBlock
ColumnEncoderDummycode. getMetaData(FrameBlock meta)
FrameBlock
ColumnEncoderFeatureHash. getMetaData(FrameBlock meta)
FrameBlock
ColumnEncoderPassThrough. getMetaData(FrameBlock meta)
FrameBlock
ColumnEncoderRecode. getMetaData(FrameBlock meta)
FrameBlock
ColumnEncoderUDF. getMetaData(FrameBlock meta)
FrameBlock
ColumnEncoderWordEmbedding. getMetaData(FrameBlock out)
FrameBlock
Encoder. getMetaData(FrameBlock out)
Construct a frame block out of the transform meta data.FrameBlock
EncoderMVImpute. getMetaData(FrameBlock out)
FrameBlock
EncoderOmit. getMetaData(FrameBlock out)
abstract FrameBlock
LegacyEncoder. getMetaData(FrameBlock out)
Construct a frame block out of the transform meta data.FrameBlock
MultiColumnEncoder. getMetaData(FrameBlock meta)
FrameBlock
MultiColumnEncoder. getMetaData(FrameBlock meta, int k)
Methods in org.apache.sysds.runtime.transform.encode with parameters of type FrameBlock Modifier and Type Method Description void
ColumnEncoderBin. allocateMetaData(FrameBlock meta)
void
ColumnEncoderComposite. allocateMetaData(FrameBlock meta)
void
ColumnEncoderDummycode. allocateMetaData(FrameBlock meta)
void
ColumnEncoderFeatureHash. allocateMetaData(FrameBlock meta)
void
ColumnEncoderPassThrough. allocateMetaData(FrameBlock meta)
void
ColumnEncoderRecode. allocateMetaData(FrameBlock meta)
void
ColumnEncoderUDF. allocateMetaData(FrameBlock meta)
void
ColumnEncoderWordEmbedding. allocateMetaData(FrameBlock meta)
void
Encoder. allocateMetaData(FrameBlock meta)
Pre-allocate a FrameBlock for metadata collection.void
MultiColumnEncoder. allocateMetaData(FrameBlock meta)
MatrixBlock
EncoderMVImpute. apply(FrameBlock in, MatrixBlock out)
MatrixBlock
EncoderOmit. apply(FrameBlock in, MatrixBlock out)
abstract MatrixBlock
LegacyEncoder. apply(FrameBlock in, MatrixBlock out)
Encode input data blockwise according to existing transform meta data (transform apply).void
EncoderMVImpute. build(FrameBlock in)
void
EncoderOmit. build(FrameBlock in)
abstract void
LegacyEncoder. build(FrameBlock in)
Build the transform meta data for the given block input.void
ColumnEncoder. buildPartial(FrameBlock in)
Partial build of internal data structures (e.g., in distributed spark operations).void
ColumnEncoderBin. buildPartial(FrameBlock in)
void
ColumnEncoderComposite. buildPartial(FrameBlock in)
void
ColumnEncoderRecode. buildPartial(FrameBlock in)
void
Encoder. buildPartial(FrameBlock in)
Partial build of internal data structures (e.g., in distributed spark operations).void
LegacyEncoder. buildPartial(FrameBlock in)
Partial build of internal data structures (e.g., in distributed spark operations).void
MultiColumnEncoder. buildPartial(FrameBlock in)
static MultiColumnEncoder
EncoderFactory. createEncoder(String spec, String[] colnames, int clen, FrameBlock meta)
static MultiColumnEncoder
EncoderFactory. createEncoder(String spec, String[] colnames, int clen, FrameBlock meta, int minCol, int maxCol)
static MultiColumnEncoder
EncoderFactory. createEncoder(String spec, String[] colnames, int clen, FrameBlock meta, MatrixBlock embeddings)
static MultiColumnEncoder
EncoderFactory. createEncoder(String spec, String[] colnames, int clen, FrameBlock meta, MatrixBlock embeddings, int minCol, int maxCol)
static MultiColumnEncoder
EncoderFactory. createEncoder(String spec, String[] colnames, Types.ValueType[] schema, int clen, FrameBlock meta)
static MultiColumnEncoder
EncoderFactory. createEncoder(String spec, String[] colnames, Types.ValueType[] schema, int clen, FrameBlock meta, MatrixBlock embeddings)
static MultiColumnEncoder
EncoderFactory. createEncoder(String spec, String[] colnames, Types.ValueType[] schema, FrameBlock meta)
static MultiColumnEncoder
EncoderFactory. createEncoder(String spec, String[] colnames, Types.ValueType[] schema, FrameBlock meta, int minCol, int maxCol)
static MultiColumnEncoder
EncoderFactory. createEncoder(String spec, String[] colnames, Types.ValueType[] schema, FrameBlock meta, MatrixBlock embeddings)
static MatrixBlock
CompressedEncode. encode(MultiColumnEncoder enc, FrameBlock in, int k)
MatrixBlock
EncoderMVImpute. encode(FrameBlock in, MatrixBlock out)
MatrixBlock
EncoderOmit. encode(FrameBlock in, MatrixBlock out)
abstract MatrixBlock
LegacyEncoder. encode(FrameBlock in, MatrixBlock out)
Block encode: build and apply (transform encode).MatrixBlock
ColumnEncoder. getColMapping(FrameBlock meta)
Obtain the column mapping of encoded frames based on the passed meta data frame.MatrixBlock
LegacyEncoder. getColMapping(FrameBlock meta, MatrixBlock out)
Obtain the column mapping of encoded frames based on the passed meta data frame.MatrixBlock
MultiColumnEncoder. getColMapping(FrameBlock meta)
Obtain the column mapping of encoded frames based on the passed meta data frame.FrameBlock
ColumnEncoderBin. getMetaData(FrameBlock meta)
FrameBlock
ColumnEncoderComposite. getMetaData(FrameBlock out)
FrameBlock
ColumnEncoderDummycode. getMetaData(FrameBlock meta)
FrameBlock
ColumnEncoderFeatureHash. getMetaData(FrameBlock meta)
FrameBlock
ColumnEncoderPassThrough. getMetaData(FrameBlock meta)
FrameBlock
ColumnEncoderRecode. getMetaData(FrameBlock meta)
FrameBlock
ColumnEncoderUDF. getMetaData(FrameBlock meta)
FrameBlock
ColumnEncoderWordEmbedding. getMetaData(FrameBlock out)
FrameBlock
Encoder. getMetaData(FrameBlock out)
Construct a frame block out of the transform meta data.FrameBlock
EncoderMVImpute. getMetaData(FrameBlock out)
FrameBlock
EncoderOmit. getMetaData(FrameBlock out)
abstract FrameBlock
LegacyEncoder. getMetaData(FrameBlock out)
Construct a frame block out of the transform meta data.FrameBlock
MultiColumnEncoder. getMetaData(FrameBlock meta)
FrameBlock
MultiColumnEncoder. getMetaData(FrameBlock meta, int k)
void
ColumnEncoderBin. initMetaData(FrameBlock meta)
void
ColumnEncoderComposite. initMetaData(FrameBlock out)
void
ColumnEncoderDummycode. initMetaData(FrameBlock meta)
void
ColumnEncoderFeatureHash. initMetaData(FrameBlock meta)
void
ColumnEncoderPassThrough. initMetaData(FrameBlock meta)
void
ColumnEncoderRecode. initMetaData(FrameBlock meta)
Construct the recodemaps from the given input frame for all columns registered for recode.void
ColumnEncoderUDF. initMetaData(FrameBlock meta)
void
ColumnEncoderWordEmbedding. initMetaData(FrameBlock meta)
void
Encoder. initMetaData(FrameBlock meta)
Sets up the required meta data for a subsequent call to apply.void
EncoderMVImpute. initMetaData(FrameBlock meta)
void
EncoderOmit. initMetaData(FrameBlock meta)
abstract void
LegacyEncoder. initMetaData(FrameBlock meta)
Sets up the required meta data for a subsequent call to apply.void
MultiColumnEncoder. initMetaData(FrameBlock meta)
void
MultiColumnEncoder. legacyBuild(FrameBlock in)
Constructors in org.apache.sysds.runtime.transform.encode with parameters of type FrameBlock Constructor Description ColumnEncoderComposite(List<ColumnEncoder> columnEncoders, FrameBlock meta)
-
Uses of FrameBlock in org.apache.sysds.runtime.transform.meta
Methods in org.apache.sysds.runtime.transform.meta that return FrameBlock Modifier and Type Method Description static FrameBlock
TfMetaUtils. readTransformMetaDataFromFile(String spec, String metapath, String colDelim)
Reads transform meta data from an HDFS file path and converts it into an in-memory FrameBlock object.static FrameBlock
TfMetaUtils. readTransformMetaDataFromPath(String spec, String metapath, String colDelim)
Reads transform meta data from the class path and converts it into an in-memory FrameBlock object. -
Uses of FrameBlock in org.apache.sysds.runtime.transform.tokenize
Methods in org.apache.sysds.runtime.transform.tokenize that return FrameBlock Modifier and Type Method Description FrameBlock
Tokenizer. apply(FrameBlock out, int k)
FrameBlock
Tokenizer. tokenize(FrameBlock in)
FrameBlock
Tokenizer. tokenize(FrameBlock in, int k)
Methods in org.apache.sysds.runtime.transform.tokenize with parameters of type FrameBlock Modifier and Type Method Description FrameBlock
Tokenizer. apply(FrameBlock out, int k)
void
Tokenizer. build(FrameBlock in, int k)
List<DependencyTask<?>>
Tokenizer. getBuildTasks(FrameBlock in)
FrameBlock
Tokenizer. tokenize(FrameBlock in)
FrameBlock
Tokenizer. tokenize(FrameBlock in, int k)
-
Uses of FrameBlock in org.apache.sysds.runtime.transform.tokenize.applier
Methods in org.apache.sysds.runtime.transform.tokenize.applier with parameters of type FrameBlock Modifier and Type Method Description int
TokenizerApplierPosition. appendTokensLong(int row, List<Object> keys, List<Token> tokenList, FrameBlock out)
int
TokenizerApplierPosition. appendTokensWide(int row, List<Object> keys, List<Token> tokenList, FrameBlock out)
int
TokenizerApplier. applyInternalRepresentation(DocumentRepresentation[] internalRepresentation, FrameBlock out)
int
TokenizerApplierCount. applyInternalRepresentation(DocumentRepresentation[] internalRepresentation, FrameBlock out, int inputRowStart, int blk)
int
TokenizerApplierHash. applyInternalRepresentation(DocumentRepresentation[] internalRepresentation, FrameBlock out, int inputRowStart, int blk)
int
TokenizerApplierPosition. applyInternalRepresentation(DocumentRepresentation[] internalRepresentation, FrameBlock out, int inputRowStart, int blk)
List<DependencyTask<?>>
TokenizerApplier. getApplyTasks(DocumentRepresentation[] internalRepresentation, FrameBlock out)
-
Uses of FrameBlock in org.apache.sysds.runtime.transform.tokenize.builder
Methods in org.apache.sysds.runtime.transform.tokenize.builder with parameters of type FrameBlock Modifier and Type Method Description void
TokenizerBuilder. createInternalRepresentation(FrameBlock in, DocumentRepresentation[] internalRepresentation)
abstract void
TokenizerBuilder. createInternalRepresentation(FrameBlock in, DocumentRepresentation[] internalRepresentation, int rowStart, int blk)
void
TokenizerBuilderNgram. createInternalRepresentation(FrameBlock in, DocumentRepresentation[] internalRepresentation, int rowStart, int blk)
void
TokenizerBuilderWhitespaceSplit. createInternalRepresentation(FrameBlock in, DocumentRepresentation[] internalRepresentation, int rowStart, int blk)
List<DependencyTask<?>>
TokenizerBuilder. getTasks(FrameBlock in, DocumentRepresentation[] internalRepresentation)
-
Uses of FrameBlock in org.apache.sysds.runtime.util
Methods in org.apache.sysds.runtime.util that return FrameBlock Modifier and Type Method Description static FrameBlock
DataConverter. convertToFrameBlock(String[][] data)
Converts a two dimensions string array into a frame block of value type string.static FrameBlock
DataConverter. convertToFrameBlock(String[][] data, Types.ValueType[] schema)
static FrameBlock
DataConverter. convertToFrameBlock(String[][] data, Types.ValueType[] schema, String[] colnames)
static FrameBlock
DataConverter. convertToFrameBlock(MatrixBlock mb)
Converts a matrix block into a frame block of value type double.static FrameBlock
DataConverter. convertToFrameBlock(MatrixBlock mb, int k)
Converts a matrix block into a frame block of value type double.static FrameBlock
DataConverter. convertToFrameBlock(MatrixBlock mb, Types.ValueType vt)
Converts a matrix block into a frame block of value type given.static FrameBlock
DataConverter. convertToFrameBlock(MatrixBlock mb, Types.ValueType[] schema)
Converts a matrix block into a frame block of with the given schemastatic FrameBlock
DataConverter. convertToFrameBlock(MatrixBlock mb, Types.ValueType[] schema, int k)
Converts a matrix block into a frame block of with the given schemastatic FrameBlock
DataConverter. convertToFrameBlock(MatrixBlock mb, Types.ValueType vt, int k)
Converts a matrix block into a frame block of a given value type.static FrameBlock
EMAUtils. exponentialMovingAverageImputation(FrameBlock block, int search_iterations, String mode, int freq, Double alpha, Double beta, Double gamma)
static FrameBlock
UtilFunctions. generateRandomFrameBlock(int rows, int cols, Types.ValueType[] schema, Random random)
Generates a random FrameBlock with given parameters.static FrameBlock
LocalFileUtils. readFrameBlockFromLocal(String fname)
Reads a frame block from local file system.static FrameBlock
LocalFileUtils. readFrameBlockFromStream(InputStream is)
Reads a frame block from an input stream, using a fused buffered reader with special support for matrix blocks.static FrameBlock
DMVUtils. syntacticalPatternDiscovery(FrameBlock frame, double threshold, String disguised_value)
Methods in org.apache.sysds.runtime.util with parameters of type FrameBlock Modifier and Type Method Description static MatrixBlock
DataConverter. convertToMatrixBlock(FrameBlock frame)
Converts a frame block with arbitrary schema into a matrix block.static String[][]
DataConverter. convertToStringFrame(FrameBlock frame)
Converts a frame block with arbitrary schema into a two dimensional string array.static FrameBlock
EMAUtils. exponentialMovingAverageImputation(FrameBlock block, int search_iterations, String mode, int freq, Double alpha, Double beta, Double gamma)
static FrameBlock
DMVUtils. syntacticalPatternDiscovery(FrameBlock frame, double threshold, String disguised_value)
static String
DataConverter. toString(FrameBlock fb)
static String
DataConverter. toString(FrameBlock fb, boolean sparse, String separator, String lineseparator, int rowsToPrint, int colsToPrint, int decimal)
static void
LocalFileUtils. writeFrameBlockToLocal(String fname, FrameBlock fb)
Writes a frame block to local file system.Method parameters in org.apache.sysds.runtime.util with type arguments of type FrameBlock Modifier and Type Method Description static IndexRange
UtilFunctions. getSelectedRangeForZeroOut(Pair<Long,FrameBlock> in, int blen, IndexRange indexRange, long lSrcRowIndex, long lDestRowIndex)
-
Uses of FrameBlock in org.apache.sysds.utils
Methods in org.apache.sysds.utils that return FrameBlock Modifier and Type Method Description static FrameBlock
DataAugmentation. dataCorruption(FrameBlock input, double pTypo, double pMiss, double pDrop, double pOut, double pSwap)
This function returns a new frame block with error introduced in the data: Typos in string values, null values, outliers in numeric data and swapped elements.static FrameBlock
DataAugmentation. miss(FrameBlock frame, double pMiss, double pDrop)
This function modifies the given, preprocessed frame block to add missing values to some of the rows, marking them with the label missing.static FrameBlock
DataAugmentation. outlier(FrameBlock frame, List<Integer> numerics, double pOut, double pPos, int times)
This function modifies the given, preprocessed frame block to add outliers to some of the numeric data of the frame, adding or several times the standard deviation, and marking them with the label outlier.static FrameBlock
DataAugmentation. preprocessing(FrameBlock frame, List<Integer> numerics, List<Integer> strings, List<Integer> swappable)
This function returns a new frame block with a labels column added, and build the lists with column index of the different types of data.static FrameBlock
DataAugmentation. swap(FrameBlock frame, List<Integer> swappable, double pSwap)
This function modifies the given, preprocessed frame block to add swapped fields of the same ValueType that are consecutive, marking them with the label swap.static FrameBlock
DataAugmentation. typos(FrameBlock frame, List<Integer> strings, double pTypo)
This function modifies the given, preprocessed frame block to add typos to the string values, marking them with the label typos.Methods in org.apache.sysds.utils with parameters of type FrameBlock Modifier and Type Method Description static FrameBlock
DataAugmentation. dataCorruption(FrameBlock input, double pTypo, double pMiss, double pDrop, double pOut, double pSwap)
This function returns a new frame block with error introduced in the data: Typos in string values, null values, outliers in numeric data and swapped elements.static FrameBlock
DataAugmentation. miss(FrameBlock frame, double pMiss, double pDrop)
This function modifies the given, preprocessed frame block to add missing values to some of the rows, marking them with the label missing.static FrameBlock
DataAugmentation. outlier(FrameBlock frame, List<Integer> numerics, double pOut, double pPos, int times)
This function modifies the given, preprocessed frame block to add outliers to some of the numeric data of the frame, adding or several times the standard deviation, and marking them with the label outlier.static FrameBlock
DataAugmentation. preprocessing(FrameBlock frame, List<Integer> numerics, List<Integer> strings, List<Integer> swappable)
This function returns a new frame block with a labels column added, and build the lists with column index of the different types of data.static FrameBlock
DataAugmentation. swap(FrameBlock frame, List<Integer> swappable, double pSwap)
This function modifies the given, preprocessed frame block to add swapped fields of the same ValueType that are consecutive, marking them with the label swap.static FrameBlock
DataAugmentation. typos(FrameBlock frame, List<Integer> strings, double pTypo)
This function modifies the given, preprocessed frame block to add typos to the string values, marking them with the label typos.
-