org.apache.sysds.runtime.instructions.gpu.context

Class GPUContext

java.lang.Object

org.apache.sysds.runtime.instructions.gpu.context.GPUContext

public class GPUContext
extends Object

Represents a context per GPU accessible through the same JVM. Each context holds cublas, cusparse, cudnn... handles which are separate for each GPU.

Method Summary

Modifier and Type	Method and Description
jcuda.Pointer	allocate(String instructionName, long size) Invokes memory manager's malloc method
void	clearMemory() Clears all memory used by this GPUContext.
void	clearTemporaryMemory()
GPUObject	createGPUObject(MatrixObject mo) Instantiates a new GPUObject initialized with the given MatrixObject.
void	cudaFreeHelper(String instructionName, jcuda.Pointer toFree, boolean eager) Does cudaFree calls, lazily.
static int	cudaGetDevice() Returns which device is currently being used.
void	destroy() Destroys this GPUContext object.
void	ensureComputeCapability() Makes sure that GPU that SystemDS is trying to use has the minimum compute capability needed.
long	getAvailableMemory() Gets the available memory on GPU that SystemDS can use.
jcuda.jcublas.cublasHandle	getCublasHandle() Returns cublasHandle for BLAS operations on the GPU.
jcuda.jcudnn.cudnnHandle	getCudnnHandle() Returns the cudnnHandle for Deep Neural Network operations on the GPU.
jcuda.jcusolver.cusolverDnHandle	getCusolverDnHandle() Returns cusolverDnHandle for invoking solve() function on dense matrices on the GPU.
jcuda.jcusparse.cusparseHandle	getCusparseHandle() Returns cusparseHandle for certain sparse BLAS operations on the GPU.
int	getDeviceNum() Returns which device is assigned to this GPUContext instance.
jcuda.runtime.cudaDeviceProp	getGPUProperties() Gets the device properties for the active GPU (set with cudaSetDevice()).
JCudaKernels	getKernels() Returns utility class used to launch custom CUDA kernel, specific to the active GPU for this GPUContext.
int	getMaxBlocks() Gets the maximum number of blocks supported by the active cuda device.
long	getMaxSharedMemory() Gets the shared memory per block supported by the active cuda device.
int	getMaxThreadsPerBlock() Gets the maximum number of threads per block for "active" GPU.
GPUMemoryManager	getMemoryManager()
int	getWarpSize() Gets the warp size supported by the active cuda device.
void	initializeThread() Sets the device for the calling thread.
void	printMemoryInfo(String opcode) Print information of memory usage.
GPUObject	shallowCopyGPUObject(GPUObject source, MatrixObject mo) Shallow copy the given source GPUObject to a new GPUObject and assign that to the given MatrixObject.
String	toString()

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

getMemoryManager

public GPUMemoryManager getMemoryManager()

cudaGetDevice

public static int cudaGetDevice()

Returns which device is currently being used.

Returns:

the current device for the calling host thread

printMemoryInfo

public void printMemoryInfo(String opcode)

Print information of memory usage.

Parameters:

opcode - opcode of caller

getDeviceNum

public int getDeviceNum()

Returns which device is assigned to this GPUContext instance.

Returns:

active device assigned to this GPUContext instance

initializeThread

public void initializeThread()

Sets the device for the calling thread. This method must be called after ExecutionContext.getGPUContext(int) If in a multi-threaded environment like parfor, this method must be called when in the appropriate thread.

allocate

public jcuda.Pointer allocate(String instructionName,
                              long size)

Invokes memory manager's malloc method

Parameters:

instructionName - name of instruction for which to record per instruction performance statistics, null if don't want to record

size - size of data (in bytes) to allocate

Returns:

jcuda pointer

cudaFreeHelper

public void cudaFreeHelper(String instructionName,
                           jcuda.Pointer toFree,
                           boolean eager)

Does cudaFree calls, lazily.

Parameters:

instructionName - name of the instruction for which to record per instruction free time, null if do not want to record

toFree - Pointer instance to be freed

eager - true if to be done eagerly

getAvailableMemory

public long getAvailableMemory()

Gets the available memory on GPU that SystemDS can use.

Returns:

the available memory in bytes

ensureComputeCapability

public void ensureComputeCapability()

Makes sure that GPU that SystemDS is trying to use has the minimum compute capability needed.

createGPUObject

public GPUObject createGPUObject(MatrixObject mo)

Instantiates a new GPUObject initialized with the given MatrixObject.

Parameters:

mo - a MatrixObject that represents a matrix

Returns:

a new GPUObject instance

shallowCopyGPUObject

public GPUObject shallowCopyGPUObject(GPUObject source,
                                      MatrixObject mo)

Shallow copy the given source GPUObject to a new GPUObject and assign that to the given MatrixObject. This copy doesn't memcopy the device memory.

Parameters:

source - a GPUObject which is the source of the copy

mo - a MatrixObject to associate with the new GPUObject

Returns:

a new GPUObject instance

getGPUProperties

public jcuda.runtime.cudaDeviceProp getGPUProperties()

Gets the device properties for the active GPU (set with cudaSetDevice()).

Returns:

the device properties

getMaxThreadsPerBlock

public int getMaxThreadsPerBlock()

Gets the maximum number of threads per block for "active" GPU.

Returns:

the maximum number of threads per block

getMaxBlocks

public int getMaxBlocks()

Gets the maximum number of blocks supported by the active cuda device.

Returns:

the maximum number of blocks supported

getMaxSharedMemory

public long getMaxSharedMemory()

Gets the shared memory per block supported by the active cuda device.

Returns:

the shared memory per block

getWarpSize

public int getWarpSize()

Gets the warp size supported by the active cuda device.

Returns:

the warp size

getCudnnHandle

public jcuda.jcudnn.cudnnHandle getCudnnHandle()

Returns the cudnnHandle for Deep Neural Network operations on the GPU.

Returns:

cudnnHandle for current thread

getCublasHandle

public jcuda.jcublas.cublasHandle getCublasHandle()

Returns cublasHandle for BLAS operations on the GPU.

Returns:

cublasHandle for current thread

getCusparseHandle

public jcuda.jcusparse.cusparseHandle getCusparseHandle()

Returns cusparseHandle for certain sparse BLAS operations on the GPU.

Returns:

cusparseHandle for current thread

getCusolverDnHandle

public jcuda.jcusolver.cusolverDnHandle getCusolverDnHandle()

Returns cusolverDnHandle for invoking solve() function on dense matrices on the GPU.

Returns:

cusolverDnHandle for current thread

getKernels

public JCudaKernels getKernels()

Returns utility class used to launch custom CUDA kernel, specific to the active GPU for this GPUContext.

Returns:

JCudaKernels for current thread

destroy

public void destroy()

Destroys this GPUContext object.

clearMemory

public void clearMemory()

Clears all memory used by this GPUContext. Be careful to ensure that no memory is currently being used in the temporary memory before invoking this. If memory is being used between MLContext invocations, they are pointed to by a GPUObject instance which would be part of the MatrixObject. The cleanup of that MatrixObject instance will cause the memory associated with that block on the GPU to be freed up.

clearTemporaryMemory

public void clearTemporaryMemory()

toString

public String toString()

Overrides:

toString in class Object