# tensorplay.cuda

Source: https://www.tensorplay.cn/docs/cuda.html

| tensorplay.cuda.StreamContext |Context-manager that selects a given stream. |
| --- | --- |
| tensorplay.cuda.can_device_access_peer | Check if peer access between two devices is possible. |
| tensorplay.cuda.check_error | Raise an error if the result of a CUDA runtime API call is not success. |
| tensorplay.cuda.current_blas_handle | Return cublasHandle_t pointer to current cuBLAS handle |
| tensorplay.cuda.current_solver_handle | Return cusolverDnHandle_t pointer to current cuSOLVER handle |
| tensorplay.cuda.current_device | Return the index of a currently selected device. |
| tensorplay.cuda.current_stream | Return the currently selected Stream for a given device. |
| tensorplay.cuda.cudart | Retrieves the CUDA runtime API module. |
| tensorplay.cuda.default_stream | Return the default Stream for a given device. |
| tensorplay.cuda.device | Context-manager that changes the selected device. |
| tensorplay.cuda.device_count | Return the number of GPUs available. |
| tensorplay.cuda.device_memory_used | Return used global (device) memory in bytes as given by nvidia-smi. |
| tensorplay.cuda.device_of | Context-manager that changes the current device to that of given object. |
| tensorplay.cuda.get_arch_list | Return list CUDA architectures this library was compiled for. |
| tensorplay.cuda.get_device_capability | Get the cuda capability of a device. |
| tensorplay.cuda.get_device_name | Get the name of a device. |
| tensorplay.cuda.get_device_properties | Get the properties of a device. |
| tensorplay.cuda.get_gencode_flags | Return NVCC gencode flags this library was compiled with. |
| tensorplay.cuda.get_stream_from_external | Return a Stream from an externally allocated CUDA stream. |
| tensorplay.cuda.get_sync_debug_mode | Return current value of debug mode for cuda synchronizing operations. |
| tensorplay.cuda.init | Initialize TensorPlay's CUDA state. |
| tensorplay.cuda.ipc_collect | Force collects GPU memory after it has been released by CUDA IPC. |
| tensorplay.cuda.is_available | Return a bool indicating if CUDA is currently available. |
| tensorplay.cuda.is_bf16_supported | Return a bool indicating if the current CUDA device supports dtype bfloat16. |
| tensorplay.cuda.is_initialized | Return whether TensorPlay's CUDA state has been initialized. |
| tensorplay.cuda.is_tf32_supported | Return a bool indicating if the current CUDA device supports dtype tf32. |
| tensorplay.cuda.memory_usage | Return the percent of time over the past sample period during which global (device) memory was being read or written as given by nvidia-smi. |
| tensorplay.cuda.set_device | Set the current device. |
| tensorplay.cuda.set_stream | Set the current stream. This is a wrapper API to set the stream. |
| tensorplay.cuda.set_sync_debug_mode | Set the debug mode for cuda synchronizing operations. |
| tensorplay.cuda.stream | Wrap around the Context-manager StreamContext that selects a given stream. |
| tensorplay.cuda.synchronize | Wait for all kernels in all streams on a CUDA device to complete. |
| tensorplay.cuda.utilization | Return the percent of time over the past sample period during which one or more kernels was executing on the GPU as given by nvidia-smi. |
| tensorplay.cuda.temperature | Return the average temperature of the GPU sensor in Degrees C (Centigrades). |
| tensorplay.cuda.power_draw | Return the average power draw of the GPU sensor in mW (MilliWatts) over the past sample period as given by nvidia-smi for Fermi or newer fully supported devices. |
| tensorplay.cuda.clock_rate | Return the clock speed of the GPU SM in MHz (megahertz) over the past sample period as given by nvidia-smi. |
| tensorplay.cuda.AcceleratorError |  |
| tensorplay.cuda.OutOfMemoryError |  |

## Random Number Generator

| tensorplay.cuda.random.get_rng_state |Return the random number generator state of the specified GPU as a ByteTensor. |
| --- | --- |
| tensorplay.cuda.random.get_rng_state_all | Return a list of ByteTensor representing the random number states of all devices. |
| tensorplay.cuda.random.set_rng_state | Set the random number generator state of the specified GPU. |
| tensorplay.cuda.random.set_rng_state_all | Set the random number generator state of all devices. |
| tensorplay.cuda.random.manual_seed | Set the seed for generating random numbers for the current GPU. |
| tensorplay.cuda.random.manual_seed_all | Set the seed for generating random numbers on all GPUs. |
| tensorplay.cuda.random.seed | Set the seed for generating random numbers to a random number for the current GPU. |
| tensorplay.cuda.random.seed_all | Set the seed for generating random numbers to a random number on all GPUs. |
| tensorplay.cuda.random.initial_seed | Return the current random seed of the current GPU. |

## Streams and events

| tensorplay.cuda.streams.Stream |An in-order queue of executing the respective tasks asynchronously in first in first out (FIFO) order. |
| --- | --- |
| tensorplay.cuda.streams.ExternalStream | Wrapper around an externally allocated CUDA stream. |
| tensorplay.cuda.streams.Event | Query and record Stream status to identify or control dependencies across Stream and measure timing. |

## Graphs (beta)

| tensorplay.cuda.graphs.is_current_stream_capturing |Return True if CUDA graph capture is underway on the current thread. |
| --- | --- |
| tensorplay.cuda.graphs.graph_pool_handle | Return an opaque token representing the id of a graph memory pool. |
| tensorplay.cuda.graphs.CUDAGraph |  |
| tensorplay.cuda.graphs.graph | Context-manager that captures CUDA work into a tensorplay.cuda.CUDAGraph. |
| tensorplay.cuda.graphs.make_graphed_callables | Callables that run per-iteration with CUDA graph capture. |
| tensorplay.cuda.graphs.export_dot | Export the most recently completed capture to a DOT file. |

## Graph Kernel Annotations (prototype)

| tensorplay.cuda.graph_annotations.is_available |Whether graph-kernel annotation recording is supported (always False here). |
| --- | --- |
| tensorplay.cuda.graph_annotations.mark_kernels | Tag the GPU work captured within its scope with user metadata. |
| tensorplay.cuda.graph_annotations.get_kernel_annotations | Return the recorded kernel annotations mapping. |
| tensorplay.cuda.graph_annotations.clear_kernel_annotations | Clear all recorded kernel annotations. |

## Memory management

| tensorplay.cuda.memory.caching_allocator_disabled |Context manager that temporarily disables the CUDA caching allocator. |
| --- | --- |
| tensorplay.cuda.memory.caching_allocator_enable | Enable or disable the CUDA memory allocator. |
| tensorplay.cuda.memory.use_mem_pool | Route allocations to a MemPool (not supported by this build). |
| tensorplay.cuda.nccl.version | Returns the version of the NCCL. |
| tensorplay.cuda.profiler.profile | Enable profiling. |
| tensorplay.cuda.profiler.start | Starts cuda profiler data collection. |
| tensorplay.cuda.profiler.stop | Stops cuda profiler data collection. |

## NVIDIA Tools Extension (NVTX)

| tensorplay.cuda.nvtx.mark |Describe an instantaneous event that occurred at some point. |
| --- | --- |
| tensorplay.cuda.nvtx.range_push | Push a range onto a stack of nested range span. |
| tensorplay.cuda.nvtx.range_pop | Pop a range off of a stack of nested range spans. |
| tensorplay.cuda.nvtx.range | Context manager / decorator that pushes an NVTX range at the beginning of its scope, and pops it at the end. |
| tensorplay.cuda.nvtx.range_end | Mark the end of a range for a given range_id. |
| tensorplay.cuda.nvtx.range_start | Mark the start of a range with string message. |

## GPUDirect Storage (prototype)

| tensorplay.cuda.gds.GdsFile |A GDS file handle (not supported by this TensorPlay build). |
| --- | --- |

## Green Contexts (experimental)

| tensorplay.cuda.green_contexts.GreenContext | |
| --- | --- |
| tensorplay.cuda.nccl.is_available |  |

## TensorPlay-specific additions

| CUDAPluggableAllocator |CUDA memory allocator plugin (not supported by this TensorPlay build). |
| --- | --- |
| CudaError |  |
| DeferredCudaCallError |  |
| MemPool | MemPool context (not supported by this TensorPlay build). |
| caching_allocator_alloc | Perform a memory allocation using the CUDA memory allocator. |
| caching_allocator_delete | Delete memory allocated using the CUDA memory allocator. |
| change_current_allocator | Change the currently used memory allocator (not supported by this build). |
| classproperty |  |
| empty_cache | Release all unoccupied cached memory currently held by the caching allocator so that those can be used in other GPU application and visible in nvidia-smi. |
| get_allocator_backend | Returns the active allocator backend as a string. |
| get_per_process_memory_fraction | Get memory fraction for a process. |
| host_memory_stats | Return a dictionary of pinned (host) allocator statistics. |
| host_memory_stats_as_nested_dict | Return the result of host_memory_stats() as a nested dictionary. |
| is_gds_available | Return whether GDS is available. |
| list_gpu_processes | Return a human-readable printout of the running processes and their GPU memory use for a given device. |
| max_memory_allocated | Return the maximum GPU memory occupied by tensors in bytes for a given device. |
| max_memory_reserved | Return the maximum GPU memory managed by the caching allocator in bytes for a given device. |
| mem_get_info | Return the global free and total GPU memory for a given device using cudaMemGetInfo. |
| memory_allocated | Return the current GPU memory occupied by tensors in bytes for a given device. |
| memory_reserved | Return the current GPU memory managed by the caching allocator in bytes for a given device. |
| memory_snapshot | Return a snapshot of the CUDA memory allocator state across all devices. |
| memory_stats | Return a dictionary of CUDA memory allocator statistics for a given device. |
| memory_stats_as_nested_dict | Return the result of memory_stats() as a nested dictionary. |
| memory_summary | Return a human-readable printout of the current memory allocator statistics for a given device. |
| reset_accumulated_host_memory_stats | Reset the "accumulated" (historical) stats tracked by the host memory allocator. |
| reset_accumulated_memory_stats | Reset the "accumulated" (historical) stats tracked by the CUDA memory allocator. |
| reset_peak_host_memory_stats | Reset the "peak" stats tracked by the host memory allocator. |
| reset_peak_memory_stats | Reset the "peak" stats tracked by the CUDA memory allocator. |
| set_per_process_memory_fraction | Set memory fraction for a process. |