# tensorplay.nn.utils.rnn API

Source: https://www.tensorplay.cn/docs/api/tensorplay.nn.utils.rnn.html

## Functions 7

[#](#api-tensorplay.nn.utils.rnn.invert_permutation)

### invert_permutation

function[Full reference ↗](/docs/generated/tensorplay.nn.utils.rnn.invert_permutation.html)

```python
tensorplay.nn.utils.rnn.invert_permutation(permutation: Tensor | None) → Tensor | None
```

Returns the inverse of permutation.

This is useful for converting between sorted and unsorted indices in
a PackedSequence.

Parameters:

permutation ([Tensor](/docs/generated/tensorplay.Tensor.html#tensorplay.Tensor), optional) – a 1-D tensor of indices to invert

[#](#api-tensorplay.nn.utils.rnn.pack_padded_sequence)

### pack_padded_sequence

function[Full reference ↗](/docs/generated/tensorplay.nn.utils.rnn.pack_padded_sequence.html)

```python
tensorplay.nn.utils.rnn.pack_padded_sequence(input: Tensor, lengths, batch_first: bool = False, enforce_sorted: bool = True) → PackedSequence
```

Packs a Tensor containing padded sequences of variable length.

input can be of size T x B x * (if batch_first is False)
or B x T x * (if batch_first is True) where T is the length
of the longest sequence, B is the batch size, and * is any number of dimensions
(including 0).

For unsorted sequences, use enforce_sorted = False. If enforce_sorted is
True, the sequences should be sorted by length in a decreasing order, i.e.
input[:,0] should be the longest sequence, and input[:,B-1] the shortest
one. enforce_sorted = True is only necessary for ONNX export.

It is an inverse operation to [pad_packed_sequence()](/docs/generated/tensorplay.nn.utils.rnn.pad_packed_sequence.html#tensorplay.nn.utils.rnn.pad_packed_sequence), and hence [pad_packed_sequence()](/docs/generated/tensorplay.nn.utils.rnn.pad_packed_sequence.html#tensorplay.nn.utils.rnn.pad_packed_sequence)
can be used to recover the underlying tensor packed in [PackedSequence](/docs/generated/tensorplay.nn.utils.rnn.PackedSequence.html#tensorplay.nn.utils.rnn.PackedSequence).

> **Note**
>
> This function accepts any input that has at least two dimensions. You can apply it to pack the labels, and use the output of the RNN with them to compute the loss directly. A Tensor can be retrieved from a PackedSequence object by accessing its .data attribute.

Parameters:

- input ([Tensor](/docs/generated/tensorplay.Tensor.html#tensorplay.Tensor)) – padded batch of variable length sequences.

- lengths ([Tensor](/docs/generated/tensorplay.Tensor.html#tensorplay.Tensor) or [list](https://docs.python.org/3/builtins/stdtypes.html#list)([int](https://docs.python.org/3/builtins/functions.html#int))) – list of sequence lengths of each batch element (must be on the CPU if provided as a tensor).

- batch_first ([bool](https://docs.python.org/3/builtins/functions.html#bool), optional) – if True, the input is expected in B x T x * format, T x B x * otherwise. Default: False.

- enforce_sorted ([bool](https://docs.python.org/3/builtins/functions.html#bool), optional) – if True, the input is expected to contain sequences sorted by length in a decreasing order. If False, the input will get sorted unconditionally. Default: True.

> **Warning**
>
> The dim of input tensor will be truncated if its length larger than correspond value in length.

Returns:

a [PackedSequence](/docs/generated/tensorplay.nn.utils.rnn.PackedSequence.html#tensorplay.nn.utils.rnn.PackedSequence) object

[#](#api-tensorplay.nn.utils.rnn.pack_sequence)

### pack_sequence

function[Full reference ↗](/docs/generated/tensorplay.nn.utils.rnn.pack_sequence.html)

```python
tensorplay.nn.utils.rnn.pack_sequence(sequences, enforce_sorted: bool = True) → PackedSequence
```

Packs a list of variable length Tensors.

Consecutive call of the next functions: pad_sequence, pack_padded_sequence.

sequences should be a list of Tensors of size L x *, where L is
the length of a sequence and * is any number of trailing dimensions,
including 0.

For unsorted sequences, use enforce_sorted = False. If enforce_sorted is
True, the sequences should be sorted in the order of decreasing length.
enforce_sorted = True is only necessary for ONNX export.

Example

```
>>> from tensorplay.nn.utils.rnn import pack_sequence
>>> a = tp.tensor([1, 2, 3])
>>> b = tp.tensor([4, 5])
>>> c = tp.tensor([6])
>>> pack_sequence([a, b, c])
PackedSequence(data=tensor([1, 4, 6, 2, 5, 3]), batch_sizes=tensor([3, 2, 1]), sorted_indices=None, unsorted_indices=None)
```

Parameters:

- sequences ([list](https://docs.python.org/3/builtins/stdtypes.html#list)[[Tensor](/docs/generated/tensorplay.Tensor.html#tensorplay.Tensor)]) – A list of sequences of decreasing length.

- enforce_sorted ([bool](https://docs.python.org/3/builtins/functions.html#bool), optional) – if True, checks that the input contains sequences sorted by length in a decreasing order. If False, this condition is not checked. Default: True.

Returns:

a [PackedSequence](/docs/generated/tensorplay.nn.utils.rnn.PackedSequence.html#tensorplay.nn.utils.rnn.PackedSequence) object

[#](#api-tensorplay.nn.utils.rnn.pad_packed_sequence)

### pad_packed_sequence

function[Full reference ↗](/docs/generated/tensorplay.nn.utils.rnn.pad_packed_sequence.html)

```python
tensorplay.nn.utils.rnn.pad_packed_sequence(sequence: PackedSequence, batch_first: bool = False, padding_value: float = 0.0, total_length: int | None = None)
```

Pad a packed batch of variable length sequences.

It is an inverse operation to [pack_padded_sequence()](/docs/generated/tensorplay.nn.utils.rnn.pack_padded_sequence.html#tensorplay.nn.utils.rnn.pack_padded_sequence).

The returned Tensor’s data will be of size T x B x * (if batch_first is False)
or B x T x * (if batch_first is True) , where T is the length
of the longest sequence and B is the batch size.

Example

```
>>> from tensorplay.nn.utils.rnn import pack_padded_sequence, pad_packed_sequence
>>> seq = tp.tensor([[1, 2, 0], [3, 0, 0], [4, 5, 6]])
>>> lens = [2, 1, 3]
>>> packed = pack_padded_sequence(
...     seq, lens, batch_first=True, enforce_sorted=False
... )
>>> packed
PackedSequence(data=tensor([4, 1, 3, 5, 2, 6]), batch_sizes=tensor([3, 2, 1]),
               sorted_indices=tensor([2, 0, 1]), unsorted_indices=tensor([1, 2, 0]))
>>> seq_unpacked, lens_unpacked = pad_packed_sequence(packed, batch_first=True)
>>> seq_unpacked
tensor([[1, 2, 0],
        [3, 0, 0],
        [4, 5, 6]])
>>> lens_unpacked
tensor([2, 1, 3])
```

> **Note**
>
> total_length is useful to implement the pack sequence -> recurrent network -> unpack sequence pattern in a model wrapped in DataParallel.

Parameters:

- sequence ([PackedSequence](/docs/generated/tensorplay.nn.utils.rnn.PackedSequence.html#tensorplay.nn.utils.rnn.PackedSequence)) – batch to pad

- batch_first ([bool](https://docs.python.org/3/builtins/functions.html#bool), optional) – if True, the output will be in B x T x * format, T x B x * otherwise.

- padding_value ([float](https://docs.python.org/3/builtins/functions.html#float), optional) – values for padded elements.

- total_length ([int](https://docs.python.org/3/builtins/functions.html#int), optional) – if not None, the output will be padded to have length total_length. This method will throw [ValueError](https://docs.python.org/3/builtins/exceptions.html#ValueError) if total_length is less than the max sequence length in sequence.

Returns:

Tuple of Tensor containing the padded sequence, and a Tensor
containing the list of lengths of each sequence in the batch.
Batch elements will be re-ordered as they were ordered originally when
the batch was passed to [pack_padded_sequence()](/docs/generated/tensorplay.nn.utils.rnn.pack_padded_sequence.html#tensorplay.nn.utils.rnn.pack_padded_sequence) or [pack_sequence()](/docs/generated/tensorplay.nn.utils.rnn.pack_sequence.html#tensorplay.nn.utils.rnn.pack_sequence).

[#](#api-tensorplay.nn.utils.rnn.pad_sequence)

### pad_sequence

function[Full reference ↗](/docs/generated/tensorplay.nn.utils.rnn.pad_sequence.html)

```python
tensorplay.nn.utils.rnn.pad_sequence(sequences, batch_first: bool = False, padding_value: float = 0.0, padding_side: str = 'right') → Tensor
```

Pad a list of variable length Tensors with padding_value.

pad_sequence stacks a list of Tensors along a new dimension, and pads them
to equal length.  sequences can be list of sequences with size L x *,
where L is length of the sequence and * is any number of dimensions
(including 0). If batch_first is False, the output is of size
T x B x *, and B x T x * otherwise, where B is the batch size
(the number of elements in sequences`), T is the length of the
longest sequence.

Example

```
>>> from tensorplay.nn.utils.rnn import pad_sequence
>>> a = tp.ones(25, 300)
>>> b = tp.ones(22, 300)
>>> c = tp.ones(15, 300)
>>> pad_sequence([a, b, c]).size()
tensorplay.Size([25, 3, 300])
```

> **Note**
>
> This function returns a Tensor of size T x B x * or B x T x * where T is the length of the longest sequence. This function assumes trailing dimensions and type of all the Tensors in sequences are same.

Parameters:

- sequences ([list](https://docs.python.org/3/builtins/stdtypes.html#list)[[Tensor](/docs/generated/tensorplay.Tensor.html#tensorplay.Tensor)]) – list of variable length sequences.

- batch_first ([bool](https://docs.python.org/3/builtins/functions.html#bool), optional) – if True, the output will be in B x T x * format, T x B x * otherwise. Default: False.

- padding_value ([float](https://docs.python.org/3/builtins/functions.html#float), optional) – value for padded elements. Default: 0.

- padding_side ([str](https://docs.python.org/3/builtins/stdtypes.html#str), optional) – the side to pad the sequences on. Default: 'right'.

Returns:

Tensor of size T x B x * if batch_first is False.
Tensor of size B x T x * otherwise

[#](#api-tensorplay.nn.utils.rnn.unpack_sequence)

### unpack_sequence

function[Full reference ↗](/docs/generated/tensorplay.nn.utils.rnn.unpack_sequence.html)

```python
tensorplay.nn.utils.rnn.unpack_sequence(packed_sequences: PackedSequence)
```

Unpack PackedSequence into a list of variable length Tensors.

packed_sequences should be a PackedSequence object.

Example

```
>>> from tensorplay.nn.utils.rnn import pack_sequence, unpack_sequence
>>> a = tp.tensor([1, 2, 3])
>>> b = tp.tensor([4, 5])
>>> c = tp.tensor([6])
>>> sequences = [a, b, c]
>>> packed_sequences = pack_sequence(sequences)
>>> unpacked_sequences = unpack_sequence(packed_sequences)
```

Parameters:

packed_sequences ([PackedSequence](/docs/generated/tensorplay.nn.utils.rnn.PackedSequence.html#tensorplay.nn.utils.rnn.PackedSequence)) – A PackedSequence object.

Returns:

a list of Tensor objects

[#](#api-tensorplay.nn.utils.rnn.unpad_sequence)

### unpad_sequence

function[Full reference ↗](/docs/generated/tensorplay.nn.utils.rnn.unpad_sequence.html)

```python
tensorplay.nn.utils.rnn.unpad_sequence(padded_sequences: Tensor, lengths: Tensor, batch_first: bool = False)
```

Unpad padded Tensor into a list of variable length Tensors.

unpad_sequence unstacks padded Tensor into a list of variable length Tensors.

Example

```
>>> from tensorplay.nn.utils.rnn import pad_sequence, unpad_sequence
>>> a = tp.ones(25, 300)
>>> b = tp.ones(22, 300)
>>> c = tp.ones(15, 300)
>>> sequences = [a, b, c]
>>> padded_sequences = pad_sequence(sequences)
>>> lengths = tp.as_tensor([v.size(0) for v in sequences])
>>> unpadded_sequences = unpad_sequence(padded_sequences, lengths)
>>> tp.allclose(sequences[0], unpadded_sequences[0])
True
```

Parameters:

- padded_sequences ([Tensor](/docs/generated/tensorplay.Tensor.html#tensorplay.Tensor)) – padded sequences.

- lengths ([Tensor](/docs/generated/tensorplay.Tensor.html#tensorplay.Tensor)) – length of original (unpadded) sequences.

- batch_first ([bool](https://docs.python.org/3/builtins/functions.html#bool), optional) – whether batch dimension first or not. Default: False.

Returns:

a list of Tensor objects

## Classes 1

[#](#api-tensorplay.nn.utils.rnn.PackedSequence)

### PackedSequence

class[Full reference ↗](/docs/generated/tensorplay.nn.utils.rnn.PackedSequence.html)

```python
class tensorplay.nn.utils.rnn.PackedSequence(data, batch_sizes=None, sorted_indices=None, unsorted_indices=None)
```

Holds the data and list of [batch_sizes](#tensorplay.nn.utils.rnn.PackedSequence.batch_sizes) of a packed sequence.

All RNN modules accept packed sequences as inputs.

> **Note**
>
> Instances of this class should never be created manually. They are meant to be instantiated by functions like pack_padded_sequence(). Batch sizes represent the number elements at each sequence step in the batch, not the varying sequence lengths passed to pack_padded_sequence(). For instance, given data abc and x the PackedSequence would contain data axbc with batch_sizes=[2,1,1].

Variables:

- data ([Tensor](/docs/generated/tensorplay.Tensor.html#tensorplay.Tensor)) – Tensor containing packed sequence

- batch_sizes ([Tensor](/docs/generated/tensorplay.Tensor.html#tensorplay.Tensor)) – Tensor of integers holding information about the batch size at each sequence step

- sorted_indices ([Tensor](/docs/generated/tensorplay.Tensor.html#tensorplay.Tensor), optional) – Tensor of integers holding how this [PackedSequence](#tensorplay.nn.utils.rnn.PackedSequence) is constructed from sequences.

- unsorted_indices ([Tensor](/docs/generated/tensorplay.Tensor.html#tensorplay.Tensor), optional) – Tensor of integers holding how this to recover the original sequences with correct order.

> **Note**
>
> data can be on arbitrary device and of arbitrary dtype. sorted_indices and unsorted_indices must be int64 tensors on the same device as data. However, batch_sizes should always be a CPU int64 tensor. This invariant is maintained throughout PackedSequence class, and all functions that construct a PackedSequence in TensorPlay (i.e. they only pass in tensors conforming to this constraint).

```python
batch_sizes: Tensor
```

Alias for field number 1

```python
count(value, /)
```

Return number of occurrences of value.

```python
data: Tensor
```

Alias for field number 0

```python
index(value, start=0, stop=9223372036854775807, /)
```

Return first index of value.

Raises ValueError if the value is not present.

```python
property is_cuda: bool
```

Return true if self.data stored on a gpu.

```python
is_pinned() → bool
```

Return true if self.data stored on in pinned memory.

```python
sorted_indices: Tensor | None
```

Alias for field number 2

```python
to(*args: Any, **kwargs: Any)
```

Perform dtype and/or device conversion on self.data.

It has similar signature as tensorplay.Tensor.to()

> **Note**
>
> If the self.data Tensor already has the correct tensorplay.DType and tensorplay.Device, then self is returned. Otherwise, returns a copy with the desired configuration.

```python
unsorted_indices: Tensor | None
```

Alias for field number 3