DifferentiationInterface.jl provides a handful of operators like gradient
or jacobian
, each with several variants: out-of-place or in-place behavior, with or without primal output value.
While it is possible to define every operator using just pushforward
and pullback
, some backends have more efficient implementations of high-level operators. When they are available, we nearly always call these backend-specific overloads. We also adapt the preparation phase accordingly. This page gives details on each backend's bindings.
The tables below summarize all implemented overloads for each backend. The cells can have three values:
- ❌: the operator is not overloaded because the backend does not support it
- ✅: the operator is overloaded
- NA: the operator does not exist
Check marks (✅) are clickable and link to the source code.
For pullback
, same-point preparation runs the forward sweep and returns the pullback closure.
Out-of-place functions f(x) = y
Operator | op | op! | value_and_op | value_and_op! |
---|
derivative | ❌ | ❌ | ❌ | ❌ |
gradient | ❌ | ❌ | ❌ | ❌ |
jacobian | ❌ | ❌ | ❌ | ❌ |
hessian | ❌ | ❌ | NA | NA |
hvp | ❌ | ❌ | NA | NA |
pullback | ❌ | ❌ | ❌ | ❌ |
pushforward | ✅ | ❌ | ✅ | ❌ |
In-place functions f!(y, x) = nothing
Backend doesn't support in-place functions.
In forward mode, for gradient
and jacobian
, preparation chooses a number of chunks.
Out-of-place functions f(x) = y
Operator | op | op! | value_and_op | value_and_op! |
---|
derivative | ❌ | ❌ | ❌ | ❌ |
gradient | ✅ | ✅ | ✅ | ✅ |
jacobian | ✅ | ✅ | ✅ | ✅ |
hessian | ❌ | ❌ | NA | NA |
hvp | ❌ | ❌ | NA | NA |
pullback | ❌ | ❌ | ❌ | ❌ |
pushforward | ✅ | ✅ | ✅ | ✅ |
In-place functions f!(y, x) = nothing
Operator | op | op! | value_and_op | value_and_op! |
---|
derivative | ❌ | ❌ | ❌ | ❌ |
jacobian | ❌ | ❌ | ❌ | ❌ |
pullback | ❌ | ❌ | ❌ | ❌ |
pushforward | ❌ | ❌ | ✅ | ❌ |
Out-of-place functions f(x) = y
Operator | op | op! | value_and_op | value_and_op! |
---|
derivative | ❌ | ❌ | ❌ | ❌ |
gradient | ✅ | ✅ | ✅ | ✅ |
jacobian | ✅ | ✅ | ✅ | ✅ |
hessian | ❌ | ❌ | NA | NA |
hvp | ❌ | ❌ | NA | NA |
pullback | ✅ | ✅ | ✅ | ✅ |
pushforward | ❌ | ❌ | ❌ | ❌ |
In-place functions f!(y, x) = nothing
Operator | op | op! | value_and_op | value_and_op! |
---|
derivative | ❌ | ❌ | ❌ | ❌ |
jacobian | ❌ | ❌ | ❌ | ❌ |
pullback | ❌ | ❌ | ✅ | ❌ |
pushforward | ❌ | ❌ | ❌ | ❌ |
Preparation generates an executable function from the symbolic expression of the differentiated function.
Preparation can be very slow for symbolic AD.
Out-of-place functions f(x) = y
Operator | op | op! | value_and_op | value_and_op! |
---|
derivative | ✅ | ✅ | ✅ | ✅ |
gradient | ✅ | ✅ | ✅ | ✅ |
jacobian | ✅ | ✅ | ✅ | ✅ |
hessian | ✅ | ✅ | NA | NA |
hvp | ✅ | ✅ | NA | NA |
pullback | ✅ | ✅ | ✅ | ✅ |
pushforward | ✅ | ✅ | ✅ | ✅ |
In-place functions f!(y, x) = nothing
Operator | op | op! | value_and_op | value_and_op! |
---|
derivative | ✅ | ✅ | ✅ | ✅ |
jacobian | ✅ | ✅ | ✅ | ✅ |
pullback | ✅ | ✅ | ✅ | ✅ |
pushforward | ✅ | ✅ | ✅ | ✅ |
Whenever possible, preparation creates a cache object.
Out-of-place functions f(x) = y
Operator | op | op! | value_and_op | value_and_op! |
---|
derivative | ✅ | ✅ | ✅ | ✅ |
gradient | ✅ | ✅ | ✅ | ✅ |
jacobian | ✅ | ✅ | ✅ | ✅ |
hessian | ✅ | ✅ | NA | NA |
hvp | ❌ | ❌ | NA | NA |
pullback | ❌ | ❌ | ❌ | ❌ |
pushforward | ✅ | ❌ | ✅ | ❌ |
In-place functions f!(y, x) = nothing
Operator | op | op! | value_and_op | value_and_op! |
---|
derivative | ✅ | ✅ | ✅ | ✅ |
jacobian | ✅ | ✅ | ✅ | ✅ |
pullback | ❌ | ❌ | ❌ | ❌ |
pushforward | ❌ | ❌ | ✅ | ❌ |
Out-of-place functions f(x) = y
Operator | op | op! | value_and_op | value_and_op! |
---|
derivative | ❌ | ❌ | ❌ | ❌ |
gradient | ✅ | ✅ | ✅ | ✅ |
jacobian | ✅ | ✅ | ✅ | ✅ |
hessian | ❌ | ❌ | NA | NA |
hvp | ❌ | ❌ | NA | NA |
pullback | ✅ | ❌ | ✅ | ❌ |
pushforward | ✅ | ❌ | ✅ | ❌ |
In-place functions f!(y, x) = nothing
Backend doesn't support in-place functions.
Wherever possible, preparation creates a config. For pushforward
, preparation allocates the necessary space for Dual
number computations.
Out-of-place functions f(x) = y
Operator | op | op! | value_and_op | value_and_op! |
---|
derivative | ✅ | ✅ | ✅ | ✅ |
gradient | ✅ | ✅ | ✅ | ✅ |
jacobian | ✅ | ✅ | ✅ | ✅ |
hessian | ✅ | ✅ | NA | NA |
hvp | ❌ | ❌ | NA | NA |
pullback | ❌ | ❌ | ❌ | ❌ |
pushforward | ✅ | ✅ | ✅ | ✅ |
In-place functions f!(y, x) = nothing
Operator | op | op! | value_and_op | value_and_op! |
---|
derivative | ✅ | ✅ | ✅ | ✅ |
jacobian | ✅ | ✅ | ✅ | ✅ |
pullback | ❌ | ❌ | ❌ | ❌ |
pushforward | ✅ | ✅ | ✅ | ✅ |
Out-of-place functions f(x) = y
Operator | op | op! | value_and_op | value_and_op! |
---|
derivative | ✅ | ✅ | ✅ | ✅ |
gradient | ✅ | ✅ | ✅ | ✅ |
jacobian | ✅ | ✅ | ✅ | ✅ |
hessian | ✅ | ✅ | NA | NA |
hvp | ❌ | ❌ | NA | NA |
pullback | ❌ | ❌ | ❌ | ❌ |
pushforward | ✅ | ✅ | ✅ | ✅ |
In-place functions f!(y, x) = nothing
Operator | op | op! | value_and_op | value_and_op! |
---|
derivative | ✅ | ✅ | ✅ | ✅ |
jacobian | ✅ | ✅ | ✅ | ✅ |
pullback | ❌ | ❌ | ❌ | ❌ |
pushforward | ✅ | ✅ | ✅ | ✅ |
Wherever possible, preparation records a tape of the function's execution.
This tape is specific to the control flow inside the function, and cannot be reused if the control flow is value-dependent (like if x[1] > 0
).
Out-of-place functions f(x) = y
Operator | op | op! | value_and_op | value_and_op! |
---|
derivative | ❌ | ❌ | ❌ | ❌ |
gradient | ✅ | ✅ | ✅ | ✅ |
jacobian | ✅ | ✅ | ✅ | ✅ |
hessian | ✅ | ✅ | NA | NA |
hvp | ❌ | ❌ | NA | NA |
pullback | ❌ | ❌ | ✅ | ✅ |
pushforward | ❌ | ❌ | ❌ | ❌ |
In-place functions f!(y, x) = nothing
Operator | op | op! | value_and_op | value_and_op! |
---|
derivative | ❌ | ❌ | ❌ | ❌ |
jacobian | ✅ | ✅ | ✅ | ✅ |
pullback | ✅ | ✅ | ✅ | ✅ |
pushforward | ❌ | ❌ | ❌ | ❌ |
Preparation generates an executable function from the symbolic expression of the differentiated function.
Preparation can be very slow for symbolic AD.
Out-of-place functions f(x) = y
Operator | op | op! | value_and_op | value_and_op! |
---|
derivative | ✅ | ✅ | ✅ | ✅ |
gradient | ✅ | ✅ | ✅ | ✅ |
jacobian | ✅ | ✅ | ✅ | ✅ |
hessian | ✅ | ✅ | NA | NA |
hvp | ❌ | ❌ | NA | NA |
pullback | ❌ | ❌ | ❌ | ❌ |
pushforward | ✅ | ✅ | ✅ | ✅ |
In-place functions f!(y, x) = nothing
Operator | op | op! | value_and_op | value_and_op! |
---|
derivative | ✅ | ✅ | ✅ | ✅ |
jacobian | ✅ | ✅ | ✅ | ✅ |
pullback | ❌ | ❌ | ❌ | ❌ |
pushforward | ✅ | ✅ | ✅ | ✅ |
For pullback
, preparation builds the reverse rule of the function.
Out-of-place functions f(x) = y
Operator | op | op! | value_and_op | value_and_op! |
---|
derivative | ❌ | ❌ | ❌ | ❌ |
gradient | ❌ | ❌ | ❌ | ❌ |
jacobian | ❌ | ❌ | ❌ | ❌ |
hessian | ❌ | ❌ | NA | NA |
hvp | ❌ | ❌ | NA | NA |
pullback | ✅ | ✅ | ✅ | ✅ |
pushforward | ❌ | ❌ | ❌ | ❌ |
In-place functions f!(y, x) = nothing
Operator | op | op! | value_and_op | value_and_op! |
---|
derivative | ❌ | ❌ | ❌ | ❌ |
jacobian | ❌ | ❌ | ❌ | ❌ |
pullback | ❌ | ❌ | ✅ | ❌ |
pushforward | ❌ | ❌ | ❌ | ❌ |
For pullback
, same-point preparation runs the forward sweep and returns the pullback closure at x
.
Out-of-place functions f(x) = y
Operator | op | op! | value_and_op | value_and_op! |
---|
derivative | ❌ | ❌ | ❌ | ❌ |
gradient | ✅ | ✅ | ✅ | ✅ |
jacobian | ❌ | ❌ | ❌ | ❌ |
hessian | ❌ | ❌ | NA | NA |
hvp | ❌ | ❌ | NA | NA |
pullback | ✅ | ❌ | ✅ | ❌ |
pushforward | ❌ | ❌ | ❌ | ❌ |
In-place functions f!(y, x) = nothing
Backend doesn't support in-place functions.
For pullback
, same-point preparation runs the forward sweep and returns the pullback closure at x
.
Out-of-place functions f(x) = y
Operator | op | op! | value_and_op | value_and_op! |
---|
derivative | ❌ | ❌ | ❌ | ❌ |
gradient | ✅ | ✅ | ✅ | ✅ |
jacobian | ✅ | ✅ | ✅ | ✅ |
hessian | ✅ | ✅ | NA | NA |
hvp | ✅ | ✅ | NA | NA |
pullback | ✅ | ❌ | ✅ | ❌ |
pushforward | ❌ | ❌ | ❌ | ❌ |
In-place functions f!(y, x) = nothing
Backend doesn't support in-place functions.