Package org.apache.sysds.runtime.data

Class TensorBlock

java.lang.Object

org.apache.sysds.runtime.data.TensorBlock

All Implemented Interfaces:

Externalizable, Serializable, org.apache.hadoop.io.Writable, CacheBlock

public class TensorBlock
extends Object
implements CacheBlock, Externalizable

A TensorBlock is the most top level representation of a tensor. There are two types of data representation which can be used: Basic/Homogeneous and Data/Heterogeneous Basic supports only one ValueType, while Data supports multiple ValueTypes along the column axis. The format determines if the TensorBlock uses a BasicTensorBlock or a DataTensorBlock for storing the data.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field	Description
static int[]	DEFAULT_DIMS
static Types.ValueType	DEFAULT_VTYPE

Constructor Summary

Constructors

Constructor	Description
TensorBlock()	Create a TensorBlock with [0,0] dimension and homogeneous representation (aka.
TensorBlock(double value)	Create a [1,1] basic FP64 TensorBlock containing the given value.
TensorBlock(int[] dims, boolean basic)	Create a TensorBlock with the given dimensions and the given data representation (basic/data).
TensorBlock(Types.ValueType[] schema, int[] dims)	Create a data TensorBlock with the given schema and the given dimensions.
TensorBlock(Types.ValueType vt, int[] dims)	Create a basic TensorBlock with the given ValueType and the given dimensions.
TensorBlock(BasicTensorBlock basicTensor)	Wrap the given BasicTensorBlock inside a TensorBlock.
TensorBlock(DataTensorBlock dataTensor)	Wrap the given DataTensorBlock inside a TensorBlock.
TensorBlock(TensorBlock that)	Copy constructor

Method Summary

Modifier and Type	Method	Description
TensorBlock	allocateBlock()	If data is not yet allocated, allocate.
TensorBlock	binaryOperations(BinaryOperator op, TensorBlock thatValue, TensorBlock result)
void	compactEmptyBlock()	Free unnecessarily allocated empty block.
TensorBlock	copy(int[] lower, int[] upper, TensorBlock src)	Copy a part of another TensorBlock
TensorBlock	copy(TensorBlock src)
TensorBlock	copyExact(int[] lower, int[] upper, TensorBlock src)	Copy a part of another TensorBlock.
Object	get(int[] ix)
double	get(int r, int c)
BasicTensorBlock	getBasicTensor()
DataCharacteristics	getDataCharacteristics()
DataTensorBlock	getDataTensor()
int	getDim(int i)
int[]	getDims()
double	getDouble(int r, int c)	Returns the double value at the passed row and column.
double	getDoubleNaN(int r, int c)	Returns the double value at the passed row and column.
long	getExactBlockDataSerializedSize(BasicTensorBlock bt)	Get the exact serialized size of a BasicTensorBlock if written by TensorBlock.writeBlockData(DataOutput,BasicTensorBlock).
long	getExactSerializedSize()	Get the exact serialized size in bytes of the cache block.
long	getInMemorySize()	Get the in-memory size in bytes of the cache block.
long	getLength()
long[]	getLongDims()
void	getNextIndexes(int[] ix)	Calculates the next index array.
static void	getNextIndexes(int[] dims, int[] ix)	Calculates the next index array.
long	getNonZeros()
int	getNumColumns()
int	getNumDims()
int	getNumRows()
Types.ValueType[]	getSchema()	Get the schema if this TensorBlock is heterogeneous.
String	getString(int r, int c)	Returns the string of the value at the passed row and column.
Types.ValueType	getValueType()	Get the ValueType if this TensorBlock is homogeneous.
boolean	isAllocated()
boolean	isBasic()
boolean	isEmpty()
boolean	isEmpty(boolean safe)
boolean	isMatrix()
boolean	isShallowSerialize()	Indicates if the cache block is subject to shallow serialized, which is generally true if in-memory size and serialized size are almost identical allowing to avoid unnecessary deep serialize.
boolean	isShallowSerialize(boolean inclConvert)	Indicates if the cache block is subject to shallow serialized, which is generally true if in-memory size and serialized size are almost identical allowing to avoid unnecessary deep serialize.
boolean	isVector()
void	merge(CacheBlock that, boolean appendOnly)	Merge the given block into the current block.
void	readExternal(ObjectInput in)
void	readFields(DataInput in)
void	reset()	Reset all cells to 0.
void	reset(int[] dims)	Reset data with new dimensions.
static Types.ValueType	resultValueType(Types.ValueType in1, Types.ValueType in2)
void	set(int[] ix, Object v)	Set a cell to the value given as an `Object`.
void	set(int r, int c, double v)	Set a cell in a 2-dimensional tensor.
void	set(Object v)
void	set(MatrixBlock other)
TensorBlock	slice(int[] offsets, TensorBlock outBlock)	Slice the current block and write into the outBlock.
CacheBlock	slice(int rl, int ru, int cl, int cu, boolean deep, CacheBlock block)	Slice a sub block out of the current block and write into the given output block.
CacheBlock	slice(int rl, int ru, int cl, int cu, CacheBlock block)	Slice a sub block out of the current block and write into the given output block.
void	toShallowSerializeBlock()	Converts a cache block that is not shallow serializable into a form that is shallow serializable.
void	write(DataOutput out)
void	writeBlockData(DataOutput out, BasicTensorBlock bt)	Write a BasicTensorBlock.
void	writeExternal(ObjectOutput out)

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_DIMS

public static final int[] DEFAULT_DIMS

DEFAULT_VTYPE

public static final Types.ValueType DEFAULT_VTYPE

Constructor Detail

TensorBlock

public TensorBlock()

Create a TensorBlock with [0,0] dimension and homogeneous representation (aka. basic).

TensorBlock

public TensorBlock(int[] dims,
                   boolean basic)

Create a TensorBlock with the given dimensions and the given data representation (basic/data).

Parameters:

dims - dimensions

basic - if true then basic TensorBlock else a data type of TensorBlock.

TensorBlock

public TensorBlock(Types.ValueType vt,
                   int[] dims)

Create a basic TensorBlock with the given ValueType and the given dimensions.

Parameters:

vt - value type

dims - dimensions

TensorBlock

public TensorBlock(Types.ValueType[] schema,
                   int[] dims)

Create a data TensorBlock with the given schema and the given dimensions.

Parameters:

schema - schema of the columns

dims - dimensions

TensorBlock

public TensorBlock(double value)

Create a [1,1] basic FP64 TensorBlock containing the given value.

Parameters:

value - value to put inside

TensorBlock

public TensorBlock(BasicTensorBlock basicTensor)

Wrap the given BasicTensorBlock inside a TensorBlock.

Parameters:

basicTensor - basic tensor block

TensorBlock

public TensorBlock(DataTensorBlock dataTensor)

Wrap the given DataTensorBlock inside a TensorBlock.

Parameters:

dataTensor - basic tensor block

TensorBlock

public TensorBlock(TensorBlock that)

Copy constructor

Parameters:

that - TensorBlock to copy

Method Detail

reset

public void reset()

Reset all cells to 0.

reset

public void reset(int[] dims)

Reset data with new dimensions.

Parameters:

dims - new dimensions

isBasic

public boolean isBasic()

isAllocated

public boolean isAllocated()

allocateBlock

public TensorBlock allocateBlock()

If data is not yet allocated, allocate.

Returns:

this TensorBlock

getBasicTensor

public BasicTensorBlock getBasicTensor()

getDataTensor

public DataTensorBlock getDataTensor()

getValueType

public Types.ValueType getValueType()

Get the ValueType if this TensorBlock is homogeneous.

Returns:

ValueType if homogeneous, null otherwise

getSchema

public Types.ValueType[] getSchema()

Get the schema if this TensorBlock is heterogeneous.

Returns:

value type if heterogeneous, null otherwise

getNumDims

public int getNumDims()

getNumRows

public int getNumRows()

Specified by:

getNumRows in interface CacheBlock

getNumColumns

public int getNumColumns()

Specified by:

getNumColumns in interface CacheBlock

getDataCharacteristics

public DataCharacteristics getDataCharacteristics()

Specified by:

getDataCharacteristics in interface CacheBlock

getInMemorySize

public long getInMemorySize()

Description copied from interface: CacheBlock

Get the in-memory size in bytes of the cache block.

Specified by:

getInMemorySize in interface CacheBlock

Returns:

in-memory size in bytes of cache block

isShallowSerialize

public boolean isShallowSerialize()

Description copied from interface: CacheBlock

Indicates if the cache block is subject to shallow serialized, which is generally true if in-memory size and serialized size are almost identical allowing to avoid unnecessary deep serialize.

Specified by:

isShallowSerialize in interface CacheBlock

Returns:

true if shallow serialized

isShallowSerialize

public boolean isShallowSerialize(boolean inclConvert)

Description copied from interface: CacheBlock

Indicates if the cache block is subject to shallow serialized, which is generally true if in-memory size and serialized size are almost identical allowing to avoid unnecessary deep serialize.

Specified by:

isShallowSerialize in interface CacheBlock

Parameters:

inclConvert - if true report blocks as shallow serialize that are currently not amenable but can be brought into an amenable form via toShallowSerializeBlock.

Returns:

true if shallow serialized

toShallowSerializeBlock

public void toShallowSerializeBlock()

Description copied from interface: CacheBlock

Converts a cache block that is not shallow serializable into a form that is shallow serializable. This methods has no affect if the given cache block is not amenable.

Specified by:

toShallowSerializeBlock in interface CacheBlock

compactEmptyBlock

public void compactEmptyBlock()

Description copied from interface: CacheBlock

Free unnecessarily allocated empty block.

Specified by:

compactEmptyBlock in interface CacheBlock

slice

public CacheBlock slice(int rl,
                        int ru,
                        int cl,
                        int cu,
                        CacheBlock block)

Description copied from interface: CacheBlock

Slice a sub block out of the current block and write into the given output block. This method returns the passed instance if not null.

Specified by:

slice in interface CacheBlock

Parameters:

rl - row lower

ru - row upper

cl - column lower

cu - column upper

block - cache block

Returns:

sub-block of cache block

slice

public CacheBlock slice(int rl,
                        int ru,
                        int cl,
                        int cu,
                        boolean deep,
                        CacheBlock block)

Description copied from interface: CacheBlock

Slice a sub block out of the current block and write into the given output block. This method returns the passed instance if not null.

Specified by:

slice in interface CacheBlock

Parameters:

rl - row lower

ru - row upper

cl - column lower

cu - column upper

deep - enforce deep-copy

block - cache block

Returns:

sub-block of cache block

merge

public void merge(CacheBlock that,
                  boolean appendOnly)

Description copied from interface: CacheBlock

Merge the given block into the current block. Both blocks needs to be of equal dimensions and contain disjoint non-zero cells.

Specified by:

merge in interface CacheBlock

Parameters:

that - cache block

appendOnly - ?

getDouble

public double getDouble(int r,
                        int c)

Description copied from interface: CacheBlock

Returns the double value at the passed row and column. If the value is missing 0 is returned.

Specified by:

getDouble in interface CacheBlock

Parameters:

r - row of the value

c - column of the value

Returns:

double value at the passed row and column

getDoubleNaN

public double getDoubleNaN(int r,
                           int c)

Description copied from interface: CacheBlock

Returns the double value at the passed row and column. If the value is missing NaN is returned.

Specified by:

getDoubleNaN in interface CacheBlock

Parameters:

r - row of the value

c - column of the value

Returns:

double value at the passed row and column

getString

public String getString(int r,
                        int c)

Description copied from interface: CacheBlock

Returns the string of the value at the passed row and column. If the value is missing or NaN, null is returned.

Specified by:

getString in interface CacheBlock

Parameters:

r - row of the value

c - column of the value

Returns:

string of the value at the passed row and column

getDim

public int getDim(int i)

getDims

public int[] getDims()

getLongDims

public long[] getLongDims()

getNextIndexes

public static void getNextIndexes(int[] dims,
                                  int[] ix)

Calculates the next index array. Note that if the given index array was the last element, the next index will be the first one.

Parameters:

dims - the dims array for which we have to decide the next index

ix - the index array which will be incremented to the next index array

getNextIndexes

public void getNextIndexes(int[] ix)

Calculates the next index array. Note that if the given index array was the last element, the next index will be the first one.

Parameters:

ix - the index array which will be incremented to the next index array

isVector

public boolean isVector()

isMatrix

public boolean isMatrix()

getLength

public long getLength()

isEmpty

public boolean isEmpty()

isEmpty

public boolean isEmpty(boolean safe)

getNonZeros

public long getNonZeros()

get

public Object get(int[] ix)

get

public double get(int r,
                  int c)

set

public void set(Object v)

set

public void set(MatrixBlock other)

set

public void set(int[] ix,
                Object v)

Set a cell to the value given as an `Object`.

Parameters:

ix - indexes in each dimension, starting with 0

v - value to set

set

public void set(int r,
                int c,
                double v)

Set a cell in a 2-dimensional tensor.

Parameters:

r - row of the cell

c - column of the cell

v - value to set

slice

public TensorBlock slice(int[] offsets,
                         TensorBlock outBlock)

Slice the current block and write into the outBlock. The offsets determines where the slice starts, the length of the blocks is given by the outBlock dimensions.

Parameters:

offsets - offsets where the slice starts

outBlock - sliced result block

Returns:

the sliced result block

copy

public TensorBlock copy(TensorBlock src)

copy

public TensorBlock copy(int[] lower,
                        int[] upper,
                        TensorBlock src)

Copy a part of another TensorBlock

Parameters:

lower - lower index of elements to copy (inclusive)

upper - upper index of elements to copy (exclusive)

src - source TensorBlock

Returns:

the shallow copy of the src TensorBlock

copyExact

public TensorBlock copyExact(int[] lower,
                             int[] upper,
                             TensorBlock src)

Copy a part of another TensorBlock. The difference to copy() is that this allows for exact sub-blocks instead of taking all consecutive data elements from lower to upper.

Parameters:

lower - lower index of elements to copy (inclusive)

upper - upper index of elements to copy (exclusive)

src - source TensorBlock

Returns:

the deep copy of the src TensorBlock

getExactSerializedSize

public long getExactSerializedSize()

Description copied from interface: CacheBlock

Get the exact serialized size in bytes of the cache block.

Specified by:

getExactSerializedSize in interface CacheBlock

Returns:

exact serialized size in bytes of cache block

getExactBlockDataSerializedSize

public long getExactBlockDataSerializedSize(BasicTensorBlock bt)

Get the exact serialized size of a BasicTensorBlock if written by TensorBlock.writeBlockData(DataOutput,BasicTensorBlock).

Parameters:

bt - BasicTensorBlock

Returns:

the size of the block data in serialized form

write

public void write(DataOutput out)
           throws IOException

Specified by:

write in interface org.apache.hadoop.io.Writable

Throws:

IOException

writeBlockData

public void writeBlockData(DataOutput out,
                           BasicTensorBlock bt)
                    throws IOException

Write a BasicTensorBlock.

Parameters:

out - output stream

bt - source BasicTensorBlock

Throws:

IOException - if writing with the output stream fails

readFields

public void readFields(DataInput in)
                throws IOException

Specified by:

readFields in interface org.apache.hadoop.io.Writable

Throws:

IOException

writeExternal

public void writeExternal(ObjectOutput out)
                   throws IOException

Specified by:

writeExternal in interface Externalizable

Throws:

IOException

readExternal

public void readExternal(ObjectInput in)
                  throws IOException

Specified by:

readExternal in interface Externalizable

Throws:

IOException

binaryOperations

public TensorBlock binaryOperations(BinaryOperator op,
                                    TensorBlock thatValue,
                                    TensorBlock result)

resultValueType

public static Types.ValueType resultValueType(Types.ValueType in1,
                                              Types.ValueType in2)