tensorplay.cuda.streams API

Query and record Stream status to identify or control dependencies across Stream and measure timing.

Parameters:

• device (tensorplay.device, optional) – the desired device for the Event. If not given, the current the relevant documentation type will be used.

• enable_timing (bool, optional) – indicates if the event should measure time (default: False)

• blocking (bool, optional) – if True, wait() will be blocking (default: False)

• interprocess (bool) – if True, the event can be shared between processes (default: False)

Returns:

An tensorplay.Event object.

Return type:

Event

Example:

>>> event = tensorplay.Event()
>>> e_cuda = tensorplay.Event(device='cuda')

elapsed_time(end_event) → float[source]

Returns the elapsed time in milliseconds between when this event and the end_event are each recorded via tensorplay.Stream.record_event().

Parameters:

end_event (tensorplay.Event) – The ending event has been recorded.

Returns:

Time between starting and ending event in milliseconds.

Return type:

float

Example:

>>> s_cuda = tensorplay.Stream(device='cuda')
>>> e1_cuda = s_cuda.record_event()
>>> e2_cuda = s_cuda.record_event()
>>> ms = e1_cuda.elapsed_time(e2_cuda)

classmethod from_ipc_handle(device, handle)[source]

Reconstruct an event from an IPC handle on the given device.

ipc_handle()[source]

Return an IPC handle of this event.

query() → bool[source]

Check if the stream where this event was recorded already moved past the point where the event was recorded. Always returns True if the Event was not recorded.

Returns:

A boolean indicating if all work currently captured by event has completed.

Return type:

bool

Example:

>>> s_cuda = tensorplay.Stream(device='cuda')
>>> e_cuda = s_cuda.record_event()
>>> e_cuda.query()
True

record(stream=None) → None[source]

Record the event in a given stream. The stream’s device must match the event’s device. This function is equivalent to stream.record_event(self).

Parameters:

stream (tensorplay.Stream, optional) – A stream to be recorded. If not given, the current stream will be used.

Example:

>>> e_cuda = tensorplay.Event(device='cuda')
>>> e_cuda.record()

synchronize() → None[source]

Wait for the event to complete. This prevents the CPU thread from proceeding until the event completes.

Example:

>>> s_cuda = tensorplay.Stream(device='cuda')
>>> e_cuda = s_cuda.record_event()
>>> e_cuda.synchronize()

wait(stream=None) → None[source]

Make all future work submitted to the given stream wait for this event.

Parameters:

stream (tensorplay.Stream, optional) – A stream to synchronize. If not given, the current stream will be used.

Example:

>>> s1_cuda = tensorplay.Stream(device='cuda')
>>> s2_cuda = tensorplay.Stream(device='cuda')
>>> e_cuda = s1_cuda.record()
>>> e_cuda.wait(s2)

Wrapper around an externally allocated CUDA stream.

This class is used to wrap streams allocated in other libraries in order to facilitate data exchange and multi-library interactions.

Parameters:

• stream_ptr (int) – Integer representation of the cudaStream_t value allocated externally.

• device (tensorplay.Device or int, optional) – the device where the stream was originally allocated. If device is specified incorrectly, subsequent launches using this stream may fail.

query() → bool

Check if all the work submitted has been completed.

Returns:

A boolean indicating if all kernels in this stream are completed.

Return type:

bool

Example:

>>> s_cuda = tensorplay.Stream(device='cuda')
>>> s_cuda.query()
True

record_event(event) → Event

Record an event. En-queuing it into the Stream to allow further synchronization from the current point in the FIFO queue.

Parameters:

event (tensorplay.Event, optional) – event to record. If not given, a new one will be allocated.

Returns:

Recorded event.

Return type:

Event

Example:

>>> s_cuda = tensorplay.Stream(device='cuda')
>>> e_cuda = s_cuda.record_event()

synchronize() → None

Wait for all the kernels in this stream to complete.

Example:

>>> s_cuda = tensorplay.Stream(device='cuda')
>>> s_cuda.synchronize()

wait_event(event) → None

Make all future work submitted to the stream wait for an event.

Parameters:

event (tensorplay.Event) – an event to wait for.

Example:

>>> s1_cuda = tensorplay.Stream(device='cuda')
>>> s2_cuda = tensorplay.Stream(device='cuda')
>>> e_cuda = s1_cuda.record_event()
>>> s2_cuda.wait_event(e_cuda)

wait_stream(stream) → None

Synchronize with another stream. All future work submitted to this stream will wait until all kernels already submitted to the given stream are completed.

Parameters:

stream (tensorplay.Stream) – a stream to synchronize.

Example:

>>> s1_cuda = tensorplay.Stream(device='cuda')
>>> s2_cuda = tensorplay.Stream(device='cuda')
>>> s2_cuda.wait_stream(s1_cuda)

An in-order queue of executing the respective tasks asynchronously in first in first out (FIFO) order. It can control or synchronize the execution of other Stream or block the current host thread to ensure the correct task sequencing. It supports with statement as a context manager to ensure the operators within the with block are running on the corresponding stream.

See in-depth description of the CUDA behavior at CUDA semantics for details on the exact semantic that applies to all devices.

Parameters:

• device (tensorplay.device, optional) – the desired device for the Stream. If not given, the current the relevant documentation type will be used.

• priority (int, optional) – priority of the stream, should be 0 or negative, where negative numbers indicate higher priority. By default, streams have priority 0.

Returns:

An tensorplay.Stream object.

Return type:

Stream

Example:

>>> with tensorplay.Stream(device='cuda') as s_cuda:
>>>     a = tensorplay.randn(10, 5, device='cuda')
>>>     b = tensorplay.randn(5, 10, device='cuda')
>>>     c = tensorplay.mm(a, b)

query() → bool[source]

Check if all the work submitted has been completed.

Returns:

A boolean indicating if all kernels in this stream are completed.

Return type:

bool

Example:

>>> s_cuda = tensorplay.Stream(device='cuda')
>>> s_cuda.query()
True

record_event(event) → Event[source]

Record an event. En-queuing it into the Stream to allow further synchronization from the current point in the FIFO queue.

Parameters:

event (tensorplay.Event, optional) – event to record. If not given, a new one will be allocated.

Returns:

Recorded event.

Return type:

Event

Example:

>>> s_cuda = tensorplay.Stream(device='cuda')
>>> e_cuda = s_cuda.record_event()

synchronize() → None[source]

Wait for all the kernels in this stream to complete.

Example:

>>> s_cuda = tensorplay.Stream(device='cuda')
>>> s_cuda.synchronize()

wait_event(event) → None[source]

Make all future work submitted to the stream wait for an event.

Parameters:

event (tensorplay.Event) – an event to wait for.

Example:

>>> s1_cuda = tensorplay.Stream(device='cuda')
>>> s2_cuda = tensorplay.Stream(device='cuda')
>>> e_cuda = s1_cuda.record_event()
>>> s2_cuda.wait_event(e_cuda)

wait_stream(stream) → None[source]

Synchronize with another stream. All future work submitted to this stream will wait until all kernels already submitted to the given stream are completed.

Parameters:

stream (tensorplay.Stream) – a stream to synchronize.

Example:

>>> s1_cuda = tensorplay.Stream(device='cuda')
>>> s2_cuda = tensorplay.Stream(device='cuda')
>>> s2_cuda.wait_stream(s1_cuda)