# tensorplay.nn.functional

Source: https://www.tensorplay.cn/docs/nn.functional.html

## Convolution functions

| tensorplay.nn.functional.conv1d |Applies a 1D convolution over an input signal composed of several input planes. |
| --- | --- |
| tensorplay.nn.functional.conv2d | Applies a 2D convolution over an input image composed of several input planes. |
| tensorplay.nn.functional.conv3d | Applies a 3D convolution over an input image composed of several input planes. |
| tensorplay.nn.functional.conv_transpose1d |  |
| tensorplay.nn.functional.conv_transpose2d |  |
| tensorplay.nn.functional.conv_transpose3d |  |
| tensorplay.nn.functional.unfold |  |
| tensorplay.nn.functional.fold | Combine an array of sliding local blocks into a tensor containing |

## Pooling functions

| tensorplay.nn.functional.avg_pool1d |avg_pool1d(input, kernel_size, stride=None, padding=0, ceil_mode=False, count_include_pad=True, divisor_override=None) -> Tensor |
| --- | --- |
| tensorplay.nn.functional.avg_pool2d |  |
| tensorplay.nn.functional.avg_pool3d | avg_pool3d(input, kernel_size, stride=None, padding=0, ceil_mode=False, count_include_pad=True, divisor_override=None) -> Tensor |
| tensorplay.nn.functional.max_pool1d | max_pool1d(input, kernel_size, stride=None, padding=0, dilation=1, ceil_mode=False, return_indices=False) -> Tensor |
| tensorplay.nn.functional.max_pool2d |  |
| tensorplay.nn.functional.max_pool3d | max_pool3d(input, kernel_size, stride=None, padding=0, dilation=1, ceil_mode=False, return_indices=False) -> Tensor |
| tensorplay.nn.functional.max_unpool1d | Compute a partial inverse of MaxPool1d. |
| tensorplay.nn.functional.max_unpool2d | Compute a partial inverse of MaxPool2d. |
| tensorplay.nn.functional.max_unpool3d | Compute a partial inverse of MaxPool3d. |
| tensorplay.nn.functional.lp_pool1d | Apply a 1D power-average pooling over an input signal. |
| tensorplay.nn.functional.lp_pool2d | Apply a 2D power-average pooling over an input signal. |
| tensorplay.nn.functional.lp_pool3d | Apply a 3D power-average pooling over an input signal. |
| tensorplay.nn.functional.adaptive_max_pool1d |  |
| tensorplay.nn.functional.adaptive_max_pool2d |  |
| tensorplay.nn.functional.adaptive_max_pool3d | Applies a 3D adaptive max pooling over an input signal composed of several input planes. |
| tensorplay.nn.functional.adaptive_avg_pool1d |  |
| tensorplay.nn.functional.adaptive_avg_pool2d |  |
| tensorplay.nn.functional.adaptive_avg_pool3d | Apply a 3D adaptive average pooling over an input signal. |
| tensorplay.nn.functional.fractional_max_pool2d | Applies 2D fractional max pooling over an input signal. |
| tensorplay.nn.functional.fractional_max_pool3d | Applies 3D fractional max pooling over an input signal. |

## Attention Mechanisms

The [tensorplay.nn.attention.bias](/docs/nn.attention.bias.html#module-tensorplay.nn.attention.bias) module contains attention_biases that are designed to be used with
scaled_dot_product_attention.

| tensorplay.nn.functional.scaled_dot_product_attention |scaled_dot_product_attention(query, key, value, attn_mask=None, dropout_p=0.0, is_causal=False, scale=None, enable_gqa=False, backend=None) -> Tensor |
| --- | --- |

## Non-linear activation functions

| tensorplay.nn.functional.threshold |Apply a threshold to each element of the input Tensor. |
| --- | --- |
| tensorplay.nn.functional.threshold_ | In-place version of threshold(). |
| tensorplay.nn.functional.relu |  |
| tensorplay.nn.functional.relu_ | In-place version of relu(). |
| tensorplay.nn.functional.hardtanh |  |
| tensorplay.nn.functional.hardtanh_ | In-place version of hardtanh(). |
| tensorplay.nn.functional.hardswish |  |
| tensorplay.nn.functional.relu6 |  |
| tensorplay.nn.functional.elu |  |
| tensorplay.nn.functional.elu_ | In-place version of elu(). |
| tensorplay.nn.functional.selu |  |
| tensorplay.nn.functional.celu |  |
| tensorplay.nn.functional.leaky_relu |  |
| tensorplay.nn.functional.leaky_relu_ | In-place version of leaky_relu(). |
| tensorplay.nn.functional.prelu |  |
| tensorplay.nn.functional.rrelu | Randomized leaky ReLU. |
| tensorplay.nn.functional.rrelu_ | In-place version of rrelu(). |
| tensorplay.nn.functional.glu | Gated Linear Unit: $a * \sigma(b)$ where the input is split in half along dim. |
| tensorplay.nn.functional.gelu | When approximate is 'none', applies $\text{GELU}(x) = x * \Phi(x)$; 'tanh' uses the tanh estimation. |
| tensorplay.nn.functional.logsigmoid | Applies element-wise $\text{LogSigmoid}(x_i) = \log \left(\frac{1}{1 + \exp(-x_i)}\right)$ |
| tensorplay.nn.functional.hardshrink | Applies the hard shrinkage function element-wise. |
| tensorplay.nn.functional.tanhshrink | Applies element-wise, $\text{Tanhshrink}(x) = x - \text{Tanh}(x)$ |
| tensorplay.nn.functional.softsign | Applies element-wise, the function $\text{SoftSign}(x) = \frac{x}{1 + \|x\|}$ |
| tensorplay.nn.functional.softplus |  |
| tensorplay.nn.functional.softmin | Apply a softmin function. |
| tensorplay.nn.functional.softmax |  |
| tensorplay.nn.functional.softshrink | Applies the soft shrinkage function element-wise. |
| tensorplay.nn.functional.gumbel_softmax | Sample from the Gumbel-Softmax distribution and optionally discretize. |
| tensorplay.nn.functional.log_softmax |  |
| tensorplay.nn.functional.tanh | Applies element-wise $\text{Tanh}(x) = \frac{\exp(x) - \exp(-x)}{\exp(x) + \exp(-x)}$ |
| tensorplay.nn.functional.sigmoid | Applies the element-wise function $\text{Sigmoid}(x) = \frac{1}{1 + \exp(-x)}$ |
| tensorplay.nn.functional.hardsigmoid |  |
| tensorplay.nn.functional.silu | Apply the Sigmoid Linear Unit (SiLU) function, element-wise. |
| tensorplay.nn.functional.mish |  |
| tensorplay.nn.functional.batch_norm |  |
| tensorplay.nn.functional.group_norm |  |
| tensorplay.nn.functional.instance_norm |  |
| tensorplay.nn.functional.layer_norm |  |
| tensorplay.nn.functional.local_response_norm | Apply local response normalization over an input signal. |
| tensorplay.nn.functional.rms_norm | Apply Root Mean Square Layer Normalization. |
| tensorplay.nn.functional.normalize | Performs $L_p$ normalization over the specified dimension — |

## Linear functions

| tensorplay.nn.functional.linear |Applies a linear transformation to the incoming data: $y = xA^T + b$. |
| --- | --- |
| tensorplay.nn.functional.bilinear |  |

## Dropout functions

| tensorplay.nn.functional.dropout | |
| --- | --- |
| tensorplay.nn.functional.alpha_dropout |  |
| tensorplay.nn.functional.feature_alpha_dropout | Randomly masks out entire channels, setting activations to the negative saturation value of the SELU activation function. |
| tensorplay.nn.functional.dropout1d | Randomly zero out entire channels (a channel is a 1D feature map). |
| tensorplay.nn.functional.dropout2d |  |
| tensorplay.nn.functional.dropout3d |  |

## Sparse functions

| tensorplay.nn.functional.embedding | |
| --- | --- |
| tensorplay.nn.functional.embedding_bag | Compute sums, means or maxes of bags of embeddings. |
| tensorplay.nn.functional.one_hot | Returns long tensor shaped tensor.shape + (num_classes,) with a 1 at |

## Distance functions

| tensorplay.nn.functional.pairwise_distance |Computes the pairwise distance between input vectors. |
| --- | --- |
| tensorplay.nn.functional.cosine_similarity | Returns cosine similarity between x1 and x2, computed along dim. |
| tensorplay.nn.functional.pdist | Computes the pairwise distance between rows of input. |

## Loss functions

| tensorplay.nn.functional.binary_cross_entropy |Compute Binary Cross Entropy between the target and input probabilities. |
| --- | --- |
| tensorplay.nn.functional.binary_cross_entropy_with_logits | Compute Binary Cross Entropy between target and input logits. |
| tensorplay.nn.functional.poisson_nll_loss | Compute the Poisson negative log likelihood loss. |
| tensorplay.nn.functional.cosine_embedding_loss | Compute the cosine embedding loss. |
| tensorplay.nn.functional.cross_entropy | Compute the cross entropy loss between input logits and target. |
| tensorplay.nn.functional.ctc_loss | Compute the Connectionist Temporal Classification loss. |
| tensorplay.nn.functional.gaussian_nll_loss | Compute the Gaussian negative log likelihood loss. |
| tensorplay.nn.functional.hinge_embedding_loss | Compute the hinge embedding loss. |
| tensorplay.nn.functional.kl_div | Compute the KL Divergence loss. |
| tensorplay.nn.functional.l1_loss | Compute the L1 loss, with optional weighting. |
| tensorplay.nn.functional.linear_cross_entropy | Compute cross entropy between input, transformed linearly, and target. |
| tensorplay.nn.functional.mse_loss |  |
| tensorplay.nn.functional.margin_ranking_loss | Compute the margin ranking loss. |
| tensorplay.nn.functional.multilabel_margin_loss | Compute the multilabel margin loss. |
| tensorplay.nn.functional.multilabel_soft_margin_loss | Compute the multilabel soft margin loss. |
| tensorplay.nn.functional.multi_margin_loss | Compute the multi margin loss, with optional weighting. |
| tensorplay.nn.functional.nll_loss | The negative log likelihood loss. |
| tensorplay.nn.functional.huber_loss | Compute the Huber loss, with optional weighting. |
| tensorplay.nn.functional.smooth_l1_loss | Compute the Smooth L1 loss. |
| tensorplay.nn.functional.soft_margin_loss | Compute the soft margin loss. |
| tensorplay.nn.functional.triplet_margin_loss | Compute the triplet loss between given input tensors and a margin greater than 0. |
| tensorplay.nn.functional.triplet_margin_with_distance_loss | Compute the triplet margin loss using a custom distance function. |

## Vision functions

| tensorplay.nn.functional.pixel_shuffle |Rearranges elements in a tensor of shape (*, C x r^2, H, W) to a tensor of shape (*, C, H x r, W x r). |
| --- | --- |
| tensorplay.nn.functional.pixel_unshuffle | Reverses the pixel_shuffle() transformation: (*, C, H x r, W x r) -> (*, C x r^2, H, W). |
| tensorplay.nn.functional.pad | Pads tensor. |
| tensorplay.nn.functional.interpolate | interpolate(input, size=None, scale_factor=None, mode='nearest', align_corners=None) -> Tensor |
| tensorplay.nn.functional.grid_sample | Compute grid sample. |
| tensorplay.nn.functional.affine_grid | Generate 2D or 3D flow field (sampling grid), given a batch of affine matrices theta. |

## Low-Precision functions

| tensorplay.nn.functional.grouped_mm | |
| --- | --- |
| tensorplay.nn.functional.scaled_mm |  |
| tensorplay.nn.functional.scaled_grouped_mm |  |

## TensorPlay-specific additions

| adaptive_max_pool1d_with_indices |Applies a 1D adaptive max pooling over an input signal, returning (output, indices). |
| --- | --- |
| adaptive_max_pool2d_with_indices | Applies a 2D adaptive max pooling over an input signal composed of several input planes, returning (output, indices). |
| adaptive_max_pool3d_with_indices | Applies a 3D adaptive max pooling over an input signal, returning (output, indices). |
| celu_ | In-place version of celu(). |
| channel_shuffle | Divide the channels in a tensor into g groups and rearrange them as in ShuffleNet: (*, C, H, W) -> (*, C, H, W) with channels interleaved across groups. |
| conv_tbc | Applies a 1D convolution over an input of shape (T, B, C) along the (kernel_width, in_channels, out_channels); the math is a standard cross-channel conv1d after permuting to (B, C, T). |
| dropout_ | In-place version of dropout(). |
| feature_dropout | Randomly zeroes entire channels (dim 1). |
| feature_dropout_ | In-place version of feature_dropout(). |
| flatten |  |
| fractional_max_pool2d_with_indices | Applies 2D fractional max pooling over an input signal composed of several input planes, returning (output, indices). |
| fractional_max_pool3d_with_indices | Applies 3D fractional max pooling over an input signal composed of several input planes, returning (output, indices). |
| gru_cell | One time step of a gated recurrent unit cell. |
| lstm_cell | One time step of a long short-term memory cell. |
| max_pool1d_with_indices | Applies a 1D max pooling over an input signal, returning (output, indices). |
| max_pool2d_with_indices | Applies a 2D max pooling over an input composed of several input planes, returning (output, indices). |
| max_pool3d_with_indices | Applies a 3D max pooling over an input signal, returning (output, indices). |
| multi_head_attention_forward | multi_head_attention_forward(query, key, value, embed_dim_to_check, num_heads, in_proj_weight, in_proj_bias=None, bias_k=None, bias_v=None, add_zero_attn=False, dropout_p=0.0, out_proj_weight=None, out_proj_bias=None, training=True, key_padding_mask=None, need_weights=True, attn_mask=None, use_separate_proj_weight=False, q_proj_weight=None, k_proj_weight=None, v_proj_weight=None, static_k=None, static_v=None, average_attn_weights=True, is_causal=False) -> (Tensor, Optional[Tensor]) |
| native_channel_shuffle |  |
| rnn_relu_cell | One time step of an Elman RNN cell with ReLU nonlinearity. |
| rnn_tanh_cell | One time step of an Elman RNN cell with tanh nonlinearity. |
| selu_ | In-place version of selu(). |