# tensorplay.autograd.functional API

Source: https://www.tensorplay.cn/docs/api/tensorplay.autograd.functional.html

## Functions 6

[#](#api-tensorplay.autograd.functional.hessian)

### hessian

function[Full reference ↗](/docs/generated/tensorplay.autograd.functional.hessian.html)

```python
tensorplay.autograd.functional.hessian(func, inputs, create_graph=False, strict=False, vectorize=False, outer_jacobian_strategy='reverse-mode')
```

Compute the Hessian of a given scalar function.

Parameters:

- func (function) – a Python function that takes Tensor inputs and returns a Tensor with a single element.

- inputs ([tuple](https://docs.python.org/3/builtins/stdtypes.html#tuple) of Tensors or [Tensor](/docs/generated/tensorplay.Tensor.html#tensorplay.Tensor)) – inputs to the function func.

- create_graph ([bool](https://docs.python.org/3/builtins/functions.html#bool), optional) – If True, the Hessian will be computed in a differentiable manner. Note that when strict is False, the result can not require gradients or be disconnected from the inputs. Defaults to False.

- strict ([bool](https://docs.python.org/3/builtins/functions.html#bool), optional) – If True, an error will be raised when we detect that there exists an input such that all the outputs are independent of it. If False, we return a Tensor of zeros as the hessian for said inputs, which is the expected mathematical value. Defaults to False.

- vectorize ([bool](https://docs.python.org/3/builtins/functions.html#bool), optional) – Not supported by this engine yet; passing True raises [NotImplementedError](https://docs.python.org/3/builtins/exceptions.html#NotImplementedError).

- outer_jacobian_strategy ([str](https://docs.python.org/3/builtins/stdtypes.html#str), optional) – Only "reverse-mode" is supported; forward-mode AD raises [NotImplementedError](https://docs.python.org/3/builtins/exceptions.html#NotImplementedError).

Returns:

if there is a single input,
this will be a single Tensor containing the Hessian for the input.
If it is a tuple, then the Hessian will be a tuple of tuples where
Hessian[i][j] will contain the Hessian of the ith input
and jth input with size the sum of the size of the ith input plus
the size of the jth input. Hessian[i][j] will have the same
dtype and device as the corresponding ith input.

Return type:

Hessian ([Tensor](/docs/generated/tensorplay.Tensor.html#tensorplay.Tensor) or a tuple of [tuple](https://docs.python.org/3/builtins/stdtypes.html#tuple) of Tensors)

Example

```
>>> def pow_reducer(x):
...     return x.pow(3).sum()
>>> inputs = tensorplay.rand(2, 2)
>>> hessian(pow_reducer, inputs)
tensor([[[[5.2265, 0.0000],
          [0.0000, 0.0000]],
         [[0.0000, 4.8221],
          [0.0000, 0.0000]]],
        [[[0.0000, 0.0000],
          [1.9456, 0.0000]],
         [[0.0000, 0.0000],
          [0.0000, 3.2550]]]])
```

[#](#api-tensorplay.autograd.functional.hvp)

### hvp

function[Full reference ↗](/docs/generated/tensorplay.autograd.functional.hvp.html)

```python
tensorplay.autograd.functional.hvp(func, inputs, v=None, create_graph=False, strict=False)
```

Compute the dot product between the scalar function’s Hessian and a vector v at a specified point.

Parameters:

- func (function) – a Python function that takes Tensor inputs and returns a Tensor with a single element.

- inputs ([tuple](https://docs.python.org/3/builtins/stdtypes.html#tuple) of Tensors or [Tensor](/docs/generated/tensorplay.Tensor.html#tensorplay.Tensor)) – inputs to the function func.

- v ([tuple](https://docs.python.org/3/builtins/stdtypes.html#tuple) of Tensors or [Tensor](/docs/generated/tensorplay.Tensor.html#tensorplay.Tensor)) – The vector for which the Hessian vector product is computed. Must be the same size as the input of func. This argument is optional when func’s input contains a single element and (if it is not provided) will be set as a Tensor containing a single 1.

- create_graph ([bool](https://docs.python.org/3/builtins/functions.html#bool), optional) – If True, both the output and result will be computed in a differentiable way. Note that when strict is False, the result can not require gradients or be disconnected from the inputs. Defaults to False.

- strict ([bool](https://docs.python.org/3/builtins/functions.html#bool), optional) – If True, an error will be raised when we detect that there exists an input such that all the outputs are independent of it. If False, we return a Tensor of zeros as the hvp for said inputs, which is the expected mathematical value. Defaults to False.

Returns:

tuple with:

func_output (tuple of Tensors or Tensor): output of func(inputs)

hvp (tuple of Tensors or Tensor): result of the dot product with
the same shape as the inputs.

Return type:

output ([tuple](https://docs.python.org/3/builtins/stdtypes.html#tuple))

Example

```
>>> def pow_reducer(x):
...     return x.pow(3).sum()
>>> inputs = tensorplay.rand(2, 2)
>>> v = tensorplay.ones(2, 2)
>>> output = hvp(pow_reducer, inputs, v)
>>> output[0]
tensor(0.1448)
>>> output[1]
tensor([[2.0239, 1.6456],
        [2.4988, 1.4310]])
```

> **Note**
>
> This function is significantly slower than vhp due to backward mode AD constraints. If your function is twice continuously differentiable, then hvp = vhp.t(). So if you know that your function satisfies this condition, you should use vhp instead that is much faster with the current implementation.

[#](#api-tensorplay.autograd.functional.jacobian)

### jacobian

function[Full reference ↗](/docs/generated/tensorplay.autograd.functional.jacobian.html)

```python
tensorplay.autograd.functional.jacobian(func, inputs, create_graph=False, strict=False, vectorize=False, strategy='reverse-mode')
```

Compute the Jacobian of a given function.

Parameters:

- func (function) – a Python function that takes Tensor inputs and returns a tuple of Tensors or a Tensor.

- inputs ([tuple](https://docs.python.org/3/builtins/stdtypes.html#tuple) of Tensors or [Tensor](/docs/generated/tensorplay.Tensor.html#tensorplay.Tensor)) – inputs to the function func.

- create_graph ([bool](https://docs.python.org/3/builtins/functions.html#bool), optional) – If True, the Jacobian will be computed in a differentiable manner. Note that when strict is False, the result can not require gradients or be disconnected from the inputs. Defaults to False.

- strict ([bool](https://docs.python.org/3/builtins/functions.html#bool), optional) – If True, an error will be raised when we detect that there exists an input such that all the outputs are independent of it. If False, we return a Tensor of zeros as the jacobian for said inputs, which is the expected mathematical value. Defaults to False.

- vectorize ([bool](https://docs.python.org/3/builtins/functions.html#bool), optional) – Not supported by this engine yet; passing True raises [NotImplementedError](https://docs.python.org/3/builtins/exceptions.html#NotImplementedError).

- strategy ([str](https://docs.python.org/3/builtins/stdtypes.html#str), optional) – Set to "reverse-mode" (default) or "forward-mode". Forward-mode AD is not supported by this engine yet; passing it raises [NotImplementedError](https://docs.python.org/3/builtins/exceptions.html#NotImplementedError).

Returns:

if there is a single
input and output, this will be a single Tensor containing the
Jacobian for the linearized inputs and output. If one of the two is
a tuple, then the Jacobian will be a tuple of Tensors. If both of
them are tuples, then the Jacobian will be a tuple of tuple of
Tensors where Jacobian[i][j] will contain the Jacobian of the
ith output and jth input and will have as size the
concatenation of the sizes of the corresponding output and the
corresponding input and will have same dtype and device as the
corresponding input.

Return type:

Jacobian ([Tensor](/docs/generated/tensorplay.Tensor.html#tensorplay.Tensor) or nested tuple of Tensors)

Example

```
>>> def exp_reducer(x):
...     return x.exp().sum(dim=1)
>>> inputs = tensorplay.rand(2, 2)
>>> jacobian(exp_reducer, inputs)
tensor([[[1.4917, 2.4352],
         [0.0000, 0.0000]],
        [[0.0000, 0.0000],
         [2.4369, 2.3799]]])
```

[#](#api-tensorplay.autograd.functional.jvp)

### jvp

function[Full reference ↗](/docs/generated/tensorplay.autograd.functional.jvp.html)

```python
tensorplay.autograd.functional.jvp(func, inputs, v=None, create_graph=False, strict=False, mode='forward')
```

Compute the dot product between the Jacobian of the given function at the point given by the inputs and a vector v.

Parameters:

- func (function) – a Python function that takes Tensor inputs and returns a tuple of Tensors or a Tensor.

- inputs ([tuple](https://docs.python.org/3/builtins/stdtypes.html#tuple) of Tensors or [Tensor](/docs/generated/tensorplay.Tensor.html#tensorplay.Tensor)) – inputs to the function func.

- v ([tuple](https://docs.python.org/3/builtins/stdtypes.html#tuple) of Tensors or [Tensor](/docs/generated/tensorplay.Tensor.html#tensorplay.Tensor)) – The vector for which the Jacobian vector product is computed. Must be the same size as the input of func. This argument is optional when the input to func contains a single element and (if it is not provided) will be set as a Tensor containing a single 1.

- create_graph ([bool](https://docs.python.org/3/builtins/functions.html#bool), optional) – If True, both the output and result will be computed in a differentiable way. Note that when strict is False, the result can not require gradients or be disconnected from the inputs. Defaults to False.

- strict ([bool](https://docs.python.org/3/builtins/functions.html#bool), optional) – If True, an error will be raised when we detect that there exists an input such that all the outputs are independent of it. If False, we return a Tensor of zeros as the jvp for said inputs, which is the expected mathematical value. Defaults to False.

Returns:

tuple with:

func_output (tuple of Tensors or Tensor): output of func(inputs)

jvp (tuple of Tensors or Tensor): result of the dot product with
the same shape as the output.

Return type:

output ([tuple](https://docs.python.org/3/builtins/stdtypes.html#tuple))

Example

```
>>> def exp_reducer(x):
...     return x.exp().sum(dim=1)
>>> inputs = tensorplay.rand(4, 4)
>>> v = tensorplay.ones(4, 4)
>>> jvp(exp_reducer, inputs, v)
(tensor([6.3090, 4.6742, 7.9114, 8.2106]),
 tensor([6.3090, 4.6742, 7.9114, 8.2106]))
```

```
>>> def adder(x, y):
...     return 2 * x + 3 * y
>>> inputs = (tensorplay.rand(2), tensorplay.rand(2))
>>> v = (tensorplay.ones(2), tensorplay.ones(2))
>>> jvp(adder, inputs, v)
(tensor([2.2399, 2.5005]),
 tensor([5., 5.]))
```

mode (str, optional): “forward” runs func once with the dual-tensor

engine and propagates tangents op by op (the default; func must
be written with operators supported by forward-mode).  “reversed”
computes the jvp via the double backwards trick and works for any
twice-differentiable func.

[#](#api-tensorplay.autograd.functional.vhp)

### vhp

function[Full reference ↗](/docs/generated/tensorplay.autograd.functional.vhp.html)

```python
tensorplay.autograd.functional.vhp(func, inputs, v=None, create_graph=False, strict=False)
```

Compute the dot product between vector v and Hessian of a  given scalar function at a specified point.

Parameters:

- func (function) – a Python function that takes Tensor inputs and returns a Tensor with a single element.

- inputs ([tuple](https://docs.python.org/3/builtins/stdtypes.html#tuple) of Tensors or [Tensor](/docs/generated/tensorplay.Tensor.html#tensorplay.Tensor)) – inputs to the function func.

- v ([tuple](https://docs.python.org/3/builtins/stdtypes.html#tuple) of Tensors or [Tensor](/docs/generated/tensorplay.Tensor.html#tensorplay.Tensor)) – The vector for which the vector Hessian product is computed. Must be the same size as the input of func. This argument is optional when func’s input contains a single element and (if it is not provided) will be set as a Tensor containing a single 1.

- create_graph ([bool](https://docs.python.org/3/builtins/functions.html#bool), optional) – If True, both the output and result will be computed in a differentiable way. Note that when strict is False, the result can not require gradients or be disconnected from the inputs. Defaults to False.

- strict ([bool](https://docs.python.org/3/builtins/functions.html#bool), optional) – If True, an error will be raised when we detect that there exists an input such that all the outputs are independent of it. If False, we return a Tensor of zeros as the vhp for said inputs, which is the expected mathematical value. Defaults to False.

Returns:

tuple with:

func_output (tuple of Tensors or Tensor): output of func(inputs)

vhp (tuple of Tensors or Tensor): result of the dot product with the
same shape as the inputs.

Return type:

output ([tuple](https://docs.python.org/3/builtins/stdtypes.html#tuple))

Example

```
>>> def pow_reducer(x):
...     return x.pow(3).sum()
>>> inputs = tensorplay.rand(2, 2)
>>> v = tensorplay.ones(2, 2)
>>> output = vhp(pow_reducer, inputs, v)
>>> output[0]
tensor(0.5591)
>>> output[1]
tensor([[1.0689, 1.2431],
        [3.0989, 4.4456]])
```

[#](#api-tensorplay.autograd.functional.vjp)

### vjp

function[Full reference ↗](/docs/generated/tensorplay.autograd.functional.vjp.html)

```python
tensorplay.autograd.functional.vjp(func, inputs, v=None, create_graph=False, strict=False)
```

Compute the dot product between a vector v and the Jacobian of the given function at the point given by the inputs.

Parameters:

- func (function) – a Python function that takes Tensor inputs and returns a tuple of Tensors or a Tensor.

- inputs ([tuple](https://docs.python.org/3/builtins/stdtypes.html#tuple) of Tensors or [Tensor](/docs/generated/tensorplay.Tensor.html#tensorplay.Tensor)) – inputs to the function func.

- v ([tuple](https://docs.python.org/3/builtins/stdtypes.html#tuple) of Tensors or [Tensor](/docs/generated/tensorplay.Tensor.html#tensorplay.Tensor)) – The vector for which the vector Jacobian product is computed. Must be the same size as the output of func. This argument is optional when the output of func contains a single element and (if it is not provided) will be set as a Tensor containing a single 1.

- create_graph ([bool](https://docs.python.org/3/builtins/functions.html#bool), optional) – If True, both the output and result will be computed in a differentiable way. Note that when strict is False, the result can not require gradients or be disconnected from the inputs. Defaults to False.

- strict ([bool](https://docs.python.org/3/builtins/functions.html#bool), optional) – If True, an error will be raised when we detect that there exists an input such that all the outputs are independent of it. If False, we return a Tensor of zeros as the vjp for said inputs, which is the expected mathematical value. Defaults to False.

Returns:

tuple with:

func_output (tuple of Tensors or Tensor): output of func(inputs)

vjpval (tuple of Tensors or Tensor): result of the dot product with
the same shape as the inputs.

Return type:

output ([tuple](https://docs.python.org/3/builtins/stdtypes.html#tuple))

Example

```
>>> def exp_reducer(x):
...     return x.exp().sum(dim=1)
>>> inputs = tensorplay.rand(4, 4)
>>> v = tensorplay.ones(4)
>>> vjp(exp_reducer, inputs, v)
(tensor([5.7817, 7.2458, 5.7830, 6.7782]),
 tensor([[1.4458, 1.3962, 1.3042, 1.6354],
        [2.1288, 1.0652, 1.5483, 2.5035],
        [2.2046, 1.1292, 1.1432, 1.3059],
        [1.3225, 1.6652, 1.7753, 2.0152]]))
```