Implementations

DifferentiationInterface.jl provides a handful of operators like gradient or jacobian, each with several variants: out-of-place or in-place behavior, with or without primal output value.

While it is possible to define every operator using just pushforward and pullback, some backends have more efficient implementations of high-level operators. When they are available, we nearly always call these backend-specific overloads. We also adapt the preparation phase accordingly. This page gives details on each backend's bindings.

The tables below summarize all implemented overloads for each backend. The cells can have three values:

❌: the operator is not overloaded because the backend does not support it
✅: the operator is overloaded
NA: the operator does not exist

Tip

Check marks (✅) are clickable and link to the source code.

ChainRulesCore

For pullback, same-point preparation runs the forward sweep and returns the pullback closure.

Diffractor

Out-of-place functions `f(x) = y`

Operator	`op`	`op!`	`value_and_op`	`value_and_op!`
`derivative`	❌	❌	❌	❌
`gradient`	❌	❌	❌	❌
`jacobian`	❌	❌	❌	❌
`hessian`	❌	❌	NA	NA
`hvp`	❌	❌	NA	NA
`pullback`	❌	❌	❌	❌
`pushforward`	✅	❌	✅	❌

In-place functions `f!(y, x) = nothing`

Backend doesn't support in-place functions.

Enzyme

Forward mode

In forward mode, for gradient and jacobian, preparation chooses a number of chunks.

Out-of-place functions `f(x) = y`

Operator	`op`	`op!`	`value_and_op`	`value_and_op!`
`derivative`	❌	❌	❌	❌
`gradient`	✅	✅	✅	✅
`jacobian`	✅	✅	✅	✅
`hessian`	❌	❌	NA	NA
`hvp`	❌	❌	NA	NA
`pullback`	❌	❌	❌	❌
`pushforward`	✅	✅	✅	✅

In-place functions `f!(y, x) = nothing`

Operator	`op`	`op!`	`value_and_op`	`value_and_op!`
`derivative`	❌	❌	❌	❌
`jacobian`	❌	❌	❌	❌
`pullback`	❌	❌	❌	❌
`pushforward`	❌	❌	✅	❌

Reverse mode

Out-of-place functions `f(x) = y`

Operator	`op`	`op!`	`value_and_op`	`value_and_op!`
`derivative`	❌	❌	❌	❌
`gradient`	✅	✅	✅	✅
`jacobian`	✅	✅	✅	✅
`hessian`	❌	❌	NA	NA
`hvp`	❌	❌	NA	NA
`pullback`	✅	✅	✅	✅
`pushforward`	❌	❌	❌	❌

In-place functions `f!(y, x) = nothing`

Operator	`op`	`op!`	`value_and_op`	`value_and_op!`
`derivative`	❌	❌	❌	❌
`jacobian`	❌	❌	❌	❌
`pullback`	❌	❌	✅	❌
`pushforward`	❌	❌	❌	❌

FastDifferentiation

Preparation generates an executable function from the symbolic expression of the differentiated function.

Warning

Preparation can be very slow for symbolic AD.

Out-of-place functions `f(x) = y`

Operator	`op`	`op!`	`value_and_op`	`value_and_op!`
`derivative`	✅	✅	✅	✅
`gradient`	✅	✅	✅	✅
`jacobian`	✅	✅	✅	✅
`hessian`	✅	✅	NA	NA
`hvp`	✅	✅	NA	NA
`pullback`	✅	✅	✅	✅
`pushforward`	✅	✅	✅	✅

In-place functions `f!(y, x) = nothing`

Operator	`op`	`op!`	`value_and_op`	`value_and_op!`
`derivative`	✅	✅	✅	✅
`jacobian`	✅	✅	✅	✅
`pullback`	✅	✅	✅	✅
`pushforward`	✅	✅	✅	✅

FiniteDiff

Whenever possible, preparation creates a cache object.

Out-of-place functions `f(x) = y`

Operator	`op`	`op!`	`value_and_op`	`value_and_op!`
`derivative`	✅	✅	✅	✅
`gradient`	✅	✅	✅	✅
`jacobian`	✅	✅	✅	✅
`hessian`	✅	✅	NA	NA
`hvp`	❌	❌	NA	NA
`pullback`	❌	❌	❌	❌
`pushforward`	✅	❌	✅	❌

In-place functions `f!(y, x) = nothing`

Operator	`op`	`op!`	`value_and_op`	`value_and_op!`
`derivative`	✅	✅	✅	✅
`jacobian`	✅	✅	✅	✅
`pullback`	❌	❌	❌	❌
`pushforward`	❌	❌	✅	❌

FiniteDifferences

Out-of-place functions `f(x) = y`

Operator	`op`	`op!`	`value_and_op`	`value_and_op!`
`derivative`	❌	❌	❌	❌
`gradient`	✅	✅	✅	✅
`jacobian`	✅	✅	✅	✅
`hessian`	❌	❌	NA	NA
`hvp`	❌	❌	NA	NA
`pullback`	✅	❌	✅	❌
`pushforward`	✅	❌	✅	❌

In-place functions `f!(y, x) = nothing`

Backend doesn't support in-place functions.

ForwardDiff

Wherever possible, preparation creates a config. For pushforward, preparation allocates the necessary space for Dual number computations.

Out-of-place functions `f(x) = y`

Operator	`op`	`op!`	`value_and_op`	`value_and_op!`
`derivative`	✅	✅	✅	✅
`gradient`	✅	✅	✅	✅
`jacobian`	✅	✅	✅	✅
`hessian`	✅	✅	NA	NA
`hvp`	❌	❌	NA	NA
`pullback`	❌	❌	❌	❌
`pushforward`	✅	✅	✅	✅

In-place functions `f!(y, x) = nothing`

Operator	`op`	`op!`	`value_and_op`	`value_and_op!`
`derivative`	✅	✅	✅	✅
`jacobian`	✅	✅	✅	✅
`pullback`	❌	❌	❌	❌
`pushforward`	✅	✅	✅	✅

PolyesterForwardDiff

Out-of-place functions `f(x) = y`

Operator	`op`	`op!`	`value_and_op`	`value_and_op!`
`derivative`	✅	✅	✅	✅
`gradient`	✅	✅	✅	✅
`jacobian`	✅	✅	✅	✅
`hessian`	✅	✅	NA	NA
`hvp`	❌	❌	NA	NA
`pullback`	❌	❌	❌	❌
`pushforward`	✅	✅	✅	✅

In-place functions `f!(y, x) = nothing`

Operator	`op`	`op!`	`value_and_op`	`value_and_op!`
`derivative`	✅	✅	✅	✅
`jacobian`	✅	✅	✅	✅
`pullback`	❌	❌	❌	❌
`pushforward`	✅	✅	✅	✅

ReverseDiff

Wherever possible, preparation records a tape of the function's execution.

Warning

This tape is specific to the control flow inside the function, and cannot be reused if the control flow is value-dependent (like if x[1] > 0).

Out-of-place functions `f(x) = y`

Operator	`op`	`op!`	`value_and_op`	`value_and_op!`
`derivative`	❌	❌	❌	❌
`gradient`	✅	✅	✅	✅
`jacobian`	✅	✅	✅	✅
`hessian`	✅	✅	NA	NA
`hvp`	❌	❌	NA	NA
`pullback`	❌	❌	✅	✅
`pushforward`	❌	❌	❌	❌

In-place functions `f!(y, x) = nothing`

Operator	`op`	`op!`	`value_and_op`	`value_and_op!`
`derivative`	❌	❌	❌	❌
`jacobian`	✅	✅	✅	✅
`pullback`	✅	✅	✅	✅
`pushforward`	❌	❌	❌	❌

Symbolics

Preparation generates an executable function from the symbolic expression of the differentiated function.

Warning

Preparation can be very slow for symbolic AD.

Out-of-place functions `f(x) = y`

Operator	`op`	`op!`	`value_and_op`	`value_and_op!`
`derivative`	✅	✅	✅	✅
`gradient`	✅	✅	✅	✅
`jacobian`	✅	✅	✅	✅
`hessian`	✅	✅	NA	NA
`hvp`	❌	❌	NA	NA
`pullback`	❌	❌	❌	❌
`pushforward`	✅	✅	✅	✅

In-place functions `f!(y, x) = nothing`

Operator	`op`	`op!`	`value_and_op`	`value_and_op!`
`derivative`	✅	✅	✅	✅
`jacobian`	✅	✅	✅	✅
`pullback`	❌	❌	❌	❌
`pushforward`	✅	✅	✅	✅

Tapir

For pullback, preparation builds the reverse rule of the function.

Out-of-place functions `f(x) = y`

Operator	`op`	`op!`	`value_and_op`	`value_and_op!`
`derivative`	❌	❌	❌	❌
`gradient`	❌	❌	❌	❌
`jacobian`	❌	❌	❌	❌
`hessian`	❌	❌	NA	NA
`hvp`	❌	❌	NA	NA
`pullback`	✅	✅	✅	✅
`pushforward`	❌	❌	❌	❌

In-place functions `f!(y, x) = nothing`

Operator	`op`	`op!`	`value_and_op`	`value_and_op!`
`derivative`	❌	❌	❌	❌
`jacobian`	❌	❌	❌	❌
`pullback`	❌	❌	✅	❌
`pushforward`	❌	❌	❌	❌

Tracker

For pullback, same-point preparation runs the forward sweep and returns the pullback closure at x.

Out-of-place functions `f(x) = y`

Operator	`op`	`op!`	`value_and_op`	`value_and_op!`
`derivative`	❌	❌	❌	❌
`gradient`	✅	✅	✅	✅
`jacobian`	❌	❌	❌	❌
`hessian`	❌	❌	NA	NA
`hvp`	❌	❌	NA	NA
`pullback`	✅	❌	✅	❌
`pushforward`	❌	❌	❌	❌

In-place functions `f!(y, x) = nothing`

Backend doesn't support in-place functions.

Zygote

For pullback, same-point preparation runs the forward sweep and returns the pullback closure at x.

Out-of-place functions `f(x) = y`

Operator	`op`	`op!`	`value_and_op`	`value_and_op!`
`derivative`	❌	❌	❌	❌
`gradient`	✅	✅	✅	✅
`jacobian`	✅	✅	✅	✅
`hessian`	✅	✅	NA	NA
`hvp`	✅	✅	NA	NA
`pullback`	✅	❌	✅	❌
`pushforward`	❌	❌	❌	❌

In-place functions `f!(y, x) = nothing`

Backend doesn't support in-place functions.

Out-of-place functions f(x) = y

In-place functions f!(y, x) = nothing

Out-of-place functions f(x) = y

In-place functions f!(y, x) = nothing

Out-of-place functions f(x) = y

In-place functions f!(y, x) = nothing

Out-of-place functions f(x) = y

In-place functions f!(y, x) = nothing

Out-of-place functions f(x) = y

In-place functions f!(y, x) = nothing

Out-of-place functions f(x) = y

In-place functions f!(y, x) = nothing

Out-of-place functions f(x) = y

In-place functions f!(y, x) = nothing

Out-of-place functions f(x) = y

In-place functions f!(y, x) = nothing

Out-of-place functions f(x) = y

In-place functions f!(y, x) = nothing

Out-of-place functions f(x) = y

In-place functions f!(y, x) = nothing

Out-of-place functions f(x) = y

In-place functions f!(y, x) = nothing

Out-of-place functions f(x) = y

In-place functions f!(y, x) = nothing

Out-of-place functions f(x) = y

In-place functions f!(y, x) = nothing

Out-of-place functions `f(x) = y`

In-place functions `f!(y, x) = nothing`

Out-of-place functions `f(x) = y`

In-place functions `f!(y, x) = nothing`

Out-of-place functions `f(x) = y`

In-place functions `f!(y, x) = nothing`

Out-of-place functions `f(x) = y`

In-place functions `f!(y, x) = nothing`

Out-of-place functions `f(x) = y`

In-place functions `f!(y, x) = nothing`

Out-of-place functions `f(x) = y`

In-place functions `f!(y, x) = nothing`

Out-of-place functions `f(x) = y`

In-place functions `f!(y, x) = nothing`

Out-of-place functions `f(x) = y`

In-place functions `f!(y, x) = nothing`

Out-of-place functions `f(x) = y`

In-place functions `f!(y, x) = nothing`

Out-of-place functions `f(x) = y`

In-place functions `f!(y, x) = nothing`

Out-of-place functions `f(x) = y`

In-place functions `f!(y, x) = nothing`

Out-of-place functions `f(x) = y`

In-place functions `f!(y, x) = nothing`

Out-of-place functions `f(x) = y`

In-place functions `f!(y, x) = nothing`