public class FrameBlock extends Object implements org.apache.hadoop.io.Writable, org.apache.sysml.runtime.controlprogram.caching.CacheBlock, Externalizable
Modifier and Type | Class and Description |
---|---|
static class |
FrameBlock.ColumnMetadata |
Modifier and Type | Field and Description |
---|---|
static int |
BUFFER_SIZE |
Constructor and Description |
---|
FrameBlock() |
FrameBlock(org.apache.sysml.parser.Expression.ValueType[] schema) |
FrameBlock(org.apache.sysml.parser.Expression.ValueType[] schema,
String[] names) |
FrameBlock(org.apache.sysml.parser.Expression.ValueType[] schema,
String[][] data) |
FrameBlock(org.apache.sysml.parser.Expression.ValueType[] schema,
String[] names,
String[][] data) |
FrameBlock(FrameBlock that)
Copy constructor for frame blocks, which uses a shallow copy for
the schema (column types and names) but a deep copy for meta data
and actual column data.
|
FrameBlock(int ncols,
org.apache.sysml.parser.Expression.ValueType vt) |
Modifier and Type | Method and Description |
---|---|
FrameBlock |
append(FrameBlock that,
FrameBlock ret,
boolean cbind)
Appends the given argument frameblock 'that' to this frameblock by
creating a deep copy to prevent side effects.
|
void |
appendColumn(boolean[] col)
Append a column of value type BOOLEAN as the last column of
the data frame.
|
void |
appendColumn(double[] col)
Append a column of value type DOUBLE as the last column of
the data frame.
|
void |
appendColumn(long[] col)
Append a column of value type INT as the last column of
the data frame.
|
void |
appendColumn(String[] col)
Append a column of value type STRING as the last column of
the data frame.
|
void |
appendColumns(double[][] cols)
Append a set of column of value type DOUBLE at the end of the frame
in order to avoid repeated allocation with appendColumns.
|
void |
appendRow(Object[] row)
Append a row to the end of the data frame, where all row fields
are boxed objects according to the schema.
|
void |
appendRow(String[] row)
Append a row to the end of the data frame, where all row fields
are string encoded.
|
void |
compactEmptyBlock()
Free unnecessarily allocated empty block.
|
void |
copy(FrameBlock src) |
void |
copy(int rl,
int ru,
int cl,
int cu,
FrameBlock src) |
static String |
createColName(int i) |
static String[] |
createColNames(int size) |
static String[] |
createColNames(int off,
int size) |
void |
ensureAllocatedColumns(int numRows)
Allocate column data structures if necessary, i.e., if schema specified
but not all column data structures created yet.
|
void |
ensureColumnCompatibility(int newlen)
Checks for matching column sizes in case of existing columns.
|
Object |
get(int r,
int c)
Gets a boxed object of the value in position (r,c).
|
org.apache.sysml.runtime.matrix.data.FrameBlock.Array |
getColumn(int c) |
Object |
getColumnData(int c) |
FrameBlock.ColumnMetadata[] |
getColumnMetadata() |
FrameBlock.ColumnMetadata |
getColumnMetadata(int c) |
String |
getColumnName(int c)
Returns the column name for the requested column.
|
Map<String,Integer> |
getColumnNameIDMap()
Creates a mapping from column names to column IDs, i.e.,
1-based column indexes
|
String[] |
getColumnNames()
Returns the column names of the frame block.
|
String[] |
getColumnNames(boolean alloc)
Returns the column names of the frame block.
|
long |
getExactSerializedSize()
Get the exact serialized size in bytes of the cache block.
|
long |
getInMemorySize()
Get the in-memory size in bytes of the cache block.
|
int |
getNumColumns()
Get the number of columns of the frame block, that is
the number of columns defined in the schema.
|
int |
getNumRows()
Get the number of rows of the frame block.
|
Iterator<Object[]> |
getObjectRowIterator()
Get a row iterator over the frame where all fields are encoded
as boxed objects according to their value types.
|
Iterator<Object[]> |
getObjectRowIterator(org.apache.sysml.parser.Expression.ValueType[] schema)
Get a row iterator over the frame where all fields are encoded
as boxed objects according to the value types of the provided
target schema.
|
Iterator<Object[]> |
getObjectRowIterator(int[] cols)
Get a row iterator over the frame where all selected fields are
encoded as boxed objects according to their value types.
|
Iterator<Object[]> |
getObjectRowIterator(int rl,
int ru)
Get a row iterator over the frame where all fields are encoded
as boxed objects according to their value types.
|
Iterator<Object[]> |
getObjectRowIterator(int rl,
int ru,
int[] cols)
Get a row iterator over the frame where all selected fields are
encoded as boxed objects according to their value types.
|
HashMap<String,Long> |
getRecodeMap(int col)
This function will split every Recode map in the column using delimiter Lop.DATATYPE_PREFIX,
as Recode map generated earlier in the form of Code+Lop.DATATYPE_PREFIX+Token and store it in a map
which contains token and code for every unique tokens.
|
org.apache.sysml.parser.Expression.ValueType[] |
getSchema()
Returns the schema of the frame block.
|
Iterator<String[]> |
getStringRowIterator()
Get a row iterator over the frame where all fields are encoded
as strings independent of their value types.
|
Iterator<String[]> |
getStringRowIterator(int[] cols)
Get a row iterator over the frame where all selected fields are
encoded as strings independent of their value types.
|
Iterator<String[]> |
getStringRowIterator(int rl,
int ru)
Get a row iterator over the frame where all fields are encoded
as strings independent of their value types.
|
Iterator<String[]> |
getStringRowIterator(int rl,
int ru,
int[] cols)
Get a row iterator over the frame where all selected fields are
encoded as strings independent of their value types.
|
boolean |
isColNameDefault(int i) |
boolean |
isColNamesDefault() |
boolean |
isColumnMetadataDefault() |
boolean |
isColumnMetadataDefault(int c) |
boolean |
isShallowSerialize()
Indicates if the cache block is subject to shallow serialized,
which is generally true if in-memory size and serialized size
are almost identical allowing to avoid unnecessary deep serialize.
|
boolean |
isShallowSerialize(boolean inclConvert)
Indicates if the cache block is subject to shallow serialized,
which is generally true if in-memory size and serialized size
are almost identical allowing to avoid unnecessary deep serialize.
|
FrameBlock |
leftIndexingOperations(FrameBlock rhsFrame,
org.apache.sysml.runtime.util.IndexRange ixrange,
FrameBlock ret) |
FrameBlock |
leftIndexingOperations(FrameBlock rhsFrame,
int rl,
int ru,
int cl,
int cu,
FrameBlock ret) |
void |
merge(org.apache.sysml.runtime.controlprogram.caching.CacheBlock that,
boolean bDummy)
Merge the given block into the current block.
|
void |
merge(FrameBlock that) |
void |
readExternal(ObjectInput in) |
void |
readFields(DataInput in) |
void |
recomputeColumnCardinality() |
void |
reset() |
void |
reset(int nrow,
boolean clearMeta) |
void |
set(int r,
int c,
Object val)
Sets the value in position (r,c), where the input is assumed
to be a boxed object consistent with the schema definition.
|
void |
setColumn(int c,
org.apache.sysml.runtime.matrix.data.FrameBlock.Array column) |
void |
setColumnMetadata(FrameBlock.ColumnMetadata[] colmeta) |
void |
setColumnMetadata(int c,
FrameBlock.ColumnMetadata colmeta) |
void |
setColumnNames(String[] colnames) |
void |
setNumRows(int numRows) |
void |
setSchema(org.apache.sysml.parser.Expression.ValueType[] schema)
Sets the schema of the frame block.
|
void |
slice(ArrayList<Pair<Long,FrameBlock>> outlist,
org.apache.sysml.runtime.util.IndexRange range,
int rowCut) |
FrameBlock |
slice(org.apache.sysml.runtime.util.IndexRange ixrange,
FrameBlock ret) |
FrameBlock |
slice(int rl,
int ru,
int cl,
int cu,
org.apache.sysml.runtime.controlprogram.caching.CacheBlock retCache)
Right indexing operations to slice a subframe out of this frame block.
|
void |
toShallowSerializeBlock()
Converts a cache block that is not shallow serializable into
a form that is shallow serializable.
|
void |
write(DataOutput out) |
void |
writeExternal(ObjectOutput out) |
FrameBlock |
zeroOutOperations(FrameBlock result,
org.apache.sysml.runtime.util.IndexRange range,
boolean complementary,
int iRowStartSrc,
int iRowStartDest,
int brlen,
int iMaxRowsToCopy)
This function ZERO OUT the data in the slicing window applicable for this block.
|
public static final int BUFFER_SIZE
public FrameBlock()
public FrameBlock(FrameBlock that)
that
- frame blockpublic FrameBlock(int ncols, org.apache.sysml.parser.Expression.ValueType vt)
public FrameBlock(org.apache.sysml.parser.Expression.ValueType[] schema)
public FrameBlock(org.apache.sysml.parser.Expression.ValueType[] schema, String[] names)
public FrameBlock(org.apache.sysml.parser.Expression.ValueType[] schema, String[][] data)
public int getNumRows()
getNumRows
in interface org.apache.sysml.runtime.controlprogram.caching.CacheBlock
public void setNumRows(int numRows)
public int getNumColumns()
getNumColumns
in interface org.apache.sysml.runtime.controlprogram.caching.CacheBlock
public org.apache.sysml.parser.Expression.ValueType[] getSchema()
public void setSchema(org.apache.sysml.parser.Expression.ValueType[] schema)
schema
- schema as array of ValueTypespublic String[] getColumnNames()
public String[] getColumnNames(boolean alloc)
alloc
- if true, create column namespublic String getColumnName(int c)
c
- column indexpublic void setColumnNames(String[] colnames)
public FrameBlock.ColumnMetadata[] getColumnMetadata()
public FrameBlock.ColumnMetadata getColumnMetadata(int c)
public boolean isColumnMetadataDefault()
public boolean isColumnMetadataDefault(int c)
public void setColumnMetadata(FrameBlock.ColumnMetadata[] colmeta)
public void setColumnMetadata(int c, FrameBlock.ColumnMetadata colmeta)
public Map<String,Integer> getColumnNameIDMap()
public void ensureAllocatedColumns(int numRows)
numRows
- number of rowspublic void ensureColumnCompatibility(int newlen)
newlen
- number of rows to compare with existing number of rowspublic static String[] createColNames(int size)
public static String[] createColNames(int off, int size)
public static String createColName(int i)
public boolean isColNamesDefault()
public boolean isColNameDefault(int i)
public void recomputeColumnCardinality()
public Object get(int r, int c)
r
- row index, 0-basedc
- column index, 0-basedpublic void set(int r, int c, Object val)
r
- row indexc
- column indexval
- value to set at specified positionpublic void reset(int nrow, boolean clearMeta)
public void reset()
public void appendRow(Object[] row)
row
- array of objectspublic void appendRow(String[] row)
row
- array of stringspublic void appendColumn(String[] col)
col
- array of stringspublic void appendColumn(boolean[] col)
col
- array of booleanspublic void appendColumn(long[] col)
col
- array of longspublic void appendColumn(double[] col)
col
- array of doublespublic void appendColumns(double[][] cols)
cols
- 2d array of doublespublic Object getColumnData(int c)
public org.apache.sysml.runtime.matrix.data.FrameBlock.Array getColumn(int c)
public void setColumn(int c, org.apache.sysml.runtime.matrix.data.FrameBlock.Array column)
public Iterator<String[]> getStringRowIterator()
public Iterator<String[]> getStringRowIterator(int[] cols)
cols
- column selection, 1-basedpublic Iterator<String[]> getStringRowIterator(int rl, int ru)
rl
- lower row indexru
- upper row indexpublic Iterator<String[]> getStringRowIterator(int rl, int ru, int[] cols)
rl
- lower row indexru
- upper row indexcols
- column selection, 1-basedpublic Iterator<Object[]> getObjectRowIterator()
public Iterator<Object[]> getObjectRowIterator(org.apache.sysml.parser.Expression.ValueType[] schema)
schema
- target schema of objectspublic Iterator<Object[]> getObjectRowIterator(int[] cols)
cols
- column selection, 1-basedpublic Iterator<Object[]> getObjectRowIterator(int rl, int ru)
rl
- lower row indexru
- upper row indexpublic Iterator<Object[]> getObjectRowIterator(int rl, int ru, int[] cols)
rl
- lower row indexru
- upper row indexcols
- column selection, 1-basedpublic void write(DataOutput out) throws IOException
write
in interface org.apache.hadoop.io.Writable
IOException
public void readFields(DataInput in) throws IOException
readFields
in interface org.apache.hadoop.io.Writable
IOException
public void writeExternal(ObjectOutput out) throws IOException
writeExternal
in interface Externalizable
IOException
public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException
readExternal
in interface Externalizable
IOException
ClassNotFoundException
public long getInMemorySize()
org.apache.sysml.runtime.controlprogram.caching.CacheBlock
getInMemorySize
in interface org.apache.sysml.runtime.controlprogram.caching.CacheBlock
public long getExactSerializedSize()
org.apache.sysml.runtime.controlprogram.caching.CacheBlock
getExactSerializedSize
in interface org.apache.sysml.runtime.controlprogram.caching.CacheBlock
public boolean isShallowSerialize()
org.apache.sysml.runtime.controlprogram.caching.CacheBlock
isShallowSerialize
in interface org.apache.sysml.runtime.controlprogram.caching.CacheBlock
public boolean isShallowSerialize(boolean inclConvert)
org.apache.sysml.runtime.controlprogram.caching.CacheBlock
isShallowSerialize
in interface org.apache.sysml.runtime.controlprogram.caching.CacheBlock
inclConvert
- if true report blocks as shallow serialize that are
currently not amenable but can be brought into an amenable form
via toShallowSerializeBlock
.public void toShallowSerializeBlock()
org.apache.sysml.runtime.controlprogram.caching.CacheBlock
toShallowSerializeBlock
in interface org.apache.sysml.runtime.controlprogram.caching.CacheBlock
public void compactEmptyBlock()
org.apache.sysml.runtime.controlprogram.caching.CacheBlock
compactEmptyBlock
in interface org.apache.sysml.runtime.controlprogram.caching.CacheBlock
public FrameBlock leftIndexingOperations(FrameBlock rhsFrame, org.apache.sysml.runtime.util.IndexRange ixrange, FrameBlock ret)
public FrameBlock leftIndexingOperations(FrameBlock rhsFrame, int rl, int ru, int cl, int cu, FrameBlock ret)
public FrameBlock slice(org.apache.sysml.runtime.util.IndexRange ixrange, FrameBlock ret)
public FrameBlock slice(int rl, int ru, int cl, int cu, org.apache.sysml.runtime.controlprogram.caching.CacheBlock retCache)
slice
in interface org.apache.sysml.runtime.controlprogram.caching.CacheBlock
rl
- row lower index, inclusive, 0-basedru
- row upper index, inclusive, 0-basedcl
- column lower index, inclusive, 0-basedcu
- column upper index, inclusive, 0-basedretCache
- cache blockpublic void slice(ArrayList<Pair<Long,FrameBlock>> outlist, org.apache.sysml.runtime.util.IndexRange range, int rowCut)
public FrameBlock append(FrameBlock that, FrameBlock ret, boolean cbind)
that
- frame block to append to current frame blockret
- frame block to return, can be nullcbind
- if true, column appendpublic void copy(FrameBlock src)
public void copy(int rl, int ru, int cl, int cu, FrameBlock src)
public HashMap<String,Long> getRecodeMap(int col)
col
- is the column # from frame data which contains Recode map generated earlier.public void merge(org.apache.sysml.runtime.controlprogram.caching.CacheBlock that, boolean bDummy)
org.apache.sysml.runtime.controlprogram.caching.CacheBlock
merge
in interface org.apache.sysml.runtime.controlprogram.caching.CacheBlock
that
- cache blockbDummy
- ?public void merge(FrameBlock that)
public FrameBlock zeroOutOperations(FrameBlock result, org.apache.sysml.runtime.util.IndexRange range, boolean complementary, int iRowStartSrc, int iRowStartDest, int brlen, int iMaxRowsToCopy)
result
- frame blockrange
- index rangecomplementary
- ?iRowStartSrc
- ?iRowStartDest
- ?brlen
- ?iMaxRowsToCopy
- ?Copyright © 2018 The Apache Software Foundation. All rights reserved.