tensorplay.autograd.grad

tensorplay.autograd.grad(outputs: Tensor | Sequence[Tensor], inputs: Tensor | Sequence[Tensor], grad_outputs: Tensor | Sequence[Tensor] | None = None, retain_graph: bool | None = None, create_graph: bool = False, only_inputs: bool = True, allow_unused: bool | None = None, is_grads_batched: bool = False, materialize_grads: bool = False)[source]

Compute and return the sum of gradients of outputs with respect to the inputs.

grad_outputs should be a sequence of length matching output containing the “vector” in vector-Jacobian product, usually the pre-computed gradients w.r.t. each of the outputs. If an output doesn’t require_grad, then the gradient can be None).

Note

If you run any forward ops, create grad_outputs, and/or call grad in a user-specified CUDA stream context, see Stream semantics of backward passes.

Parameters:

• outputs (sequence of Tensor or GradientEdge) – outputs of the differentiated function.

• inputs (sequence of Tensor or GradientEdge) – Inputs w.r.t. which the gradient will be returned (and not accumulated into .grad).

• grad_outputs (sequence of Tensor) – The “vector” in the vector-Jacobian product. Usually gradients w.r.t. each output. None values can be specified for scalar Tensors or ones that don’t require grad. If a None value would be acceptable for all grad_tensors, then this argument is optional. Default: None.

• retain_graph (bool, optional) – If False, the graph used to compute the grad will be freed. Note that in nearly all cases setting this option to True is not needed and often can be worked around in a much more efficient way. Defaults to the value of create_graph.

• create_graph (bool, optional) – If True, graph of the derivative will be constructed, allowing to compute higher order derivative products. Default: False.

• only_inputs (bool, optional) – Deprecated and ignored; gradients are only ever returned for inputs.

• allow_unused (Optional[bool], optional) – If False, specifying inputs that were not used when computing outputs (and therefore their grad is always zero) is an error. Defaults to the value of materialize_grads.

• is_grads_batched (bool, optional) – If True, the first dimension of each tensor in grad_outputs is a batch dimension: one vector-Jacobian product is computed per entry, in a single tensorplay.vmap() over the backward pass. Default: False.

• materialize_grads (bool, optional) – If True, the gradient of an unused input is returned as zeros instead of None. Default: False.

Returns a tuple with one gradient per input, or a dict keyed like inputs when inputs is a dict.