# Automatic differentiation package - tensorplay.autograd

Source: https://www.tensorplay.cn/docs/autograd.html

| tensorplay.autograd.backward |Compute the sum of gradients of given tensors with respect to graph leaves. |
| --- | --- |
| tensorplay.autograd.grad | Compute and return the sum of gradients of outputs with respect to the inputs. |

## Forward-mode Automatic Differentiation

| tensorplay.autograd.forward_ad.make_dual |Associates a tensor value with its tangent to create a "dual tensor". |
| --- | --- |
| tensorplay.autograd.forward_ad.unpack_dual | Unpacks a dual tensor into its primal value and forward gradient. |
| tensorplay.autograd.forward_ad.enter_dual_level | Enters a new forward grad level and returns its index. |
| tensorplay.autograd.forward_ad.exit_dual_level | Exits a forward grad level. |

## Functional higher level API

| tensorplay.autograd.functional.jacobian |Compute the Jacobian of a given function. |
| --- | --- |
| tensorplay.autograd.functional.hessian | Compute the Hessian of a given scalar function. |
| tensorplay.autograd.functional.vjp | Compute the dot product between a vector v and the Jacobian of the given function at the point given by the inputs. |
| tensorplay.autograd.functional.jvp | Compute the dot product between the Jacobian of the given function at the point given by the inputs and a vector v. |
| tensorplay.autograd.functional.vhp | Compute the dot product between vector v and Hessian of a given scalar function at a specified point. |
| tensorplay.autograd.functional.hvp | Compute the dot product between the scalar function's Hessian and a vector v at a specified point. |

## Function

| tensorplay.autograd.function.Function |Records operation history and defines formulas for differentiating ops. |
| --- | --- |
| tensorplay.autograd.jvp | Compute the dot product between the Jacobian of the given function at the point given by the inputs and a vector v. |

## Context method mixins

| tensorplay.autograd.function.FunctionMeta |the name classproperty ("<Cls>Backward", used for node naming) and a friendlier repr for subclasses. |
| --- | --- |

## Custom Function utilities

| tensorplay.autograd.function.once_differentiable |Decorator to make a custom autograd Function's backward run once, with gradients detached and grad-mode disabled inside. |
| --- | --- |
| tensorplay.autograd.function.InplaceFunction | In-place operations must call ctx.mark_dirty on the mutated inputs inside forward; this subclass exists only so historical code that subclasses it keeps working. |
| tensorplay.autograd.function.NestedIOFunction | Kept only for import compatibility; the modern contract is to define forward + backward on Function directly. |

## Numerical gradient checking

| tensorplay.autograd.gradcheck.gradcheck |Check gradients computed via small finite differences against analytical gradients wrt tensors in inputs that are of floating point or complex type and with requires_grad=True. |
| --- | --- |
| tensorplay.autograd.gradcheck.gradgradcheck | Check gradients of gradients computed via small finite differences against analytical gradients wrt tensors in inputs and grad_outputs that are of floating point or complex type and with requires_grad=True. |
| tensorplay.autograd.gradcheck.GradcheckError | Error raised by gradcheck() and gradgradcheck(). |
| tensorplay.autograd.gradcheck.get_numerical_jacobian_wrt_specific_input |  |

## Profiler

| tensorplay.profiler.profiler.profile |Context manager that records dispatched operations and annotations. |
| --- | --- |
| tensorplay.profiler.profiler.record_function | Mark a user-defined span in the active profiling session. |
| tensorplay.autograd.profiler_util.EventList | A list of profiling events with helper methods for analysis and visualization. |
| tensorplay.autograd.profiler_util.FormattedTimesMixin | Helpers for FunctionEvent and FunctionEventAvg. |
| tensorplay.autograd.profiler_util.FunctionEvent | Profiling information about a single function. |
| tensorplay.autograd.profiler_util.FunctionEventAvg | Averaged profiling statistics over multiple FunctionEvent objects. |
| tensorplay.autograd.profiler_util.Interval |  |
| tensorplay.autograd.profiler_util.Kernel |  |
| tensorplay.autograd.profiler_util.MemRecordsAcc | Acceleration structure for accessing mem_records in interval. |
| tensorplay.autograd.profiler_util.StringTable |  |
| tensorplay.profiler.profiler.emit_nvtx | Emit named ranges for active operations. |
| tensorplay.profiler.profiler.emit_itt | Emit instrumentation ranges for active operations. |

## Debugging and anomaly detection

| tensorplay.autograd.anomaly_mode.detect_anomaly |Context-manager that enables anomaly detection for the autograd engine. |
| --- | --- |
| tensorplay.autograd.anomaly_mode.set_detect_anomaly | Context-manager that sets the anomaly detection for the autograd engine on or off. |

## Autograd graph

| tensorplay.autograd.graph.saved_tensors_hooks |Context manager that installs a (pack, unpack) pair for every tensor saved via Function()'s save_for_backward while active. |
| --- | --- |
| tensorplay.autograd.graph.save_on_cpu | Store saved values in host memory and restore their original device. |
| tensorplay.autograd.variable.Variable |  |
| tensorplay.autograd.variable.VariableMeta |  |