Vectorial objectives

AbstractVectorFunction{E, FT} <: Function

Represent an abstract vectorial function $f:\mathcal{M} → ℝ^n$ with an AbstractEvaluationType E and an AbstractVectorialType to specify the format $f$ is implemented as.

Representations of $f$

There are three different representations of $f$, which might be beneficial in one or the other situation:

• the FunctionVectorialType storing a single function $f$ that returns a vector,
• the ComponentVectorialType storing a vector of functions $f_i$ that return a single value each,
• the CoefficientVectorialType storing functions with respect to a specific basis of the tangent space for gradients and Hessians. Gradients of this type are usually referred to as Jacobians.

For the ComponentVectorialType imagine that $f$ could also be written using its component functions,

\[f(p) = \bigl( f_1(p),f_2(p),…,.f_n(p) \bigr)^{\mathrm{T}}\]

In this representation f is given as a vector [f1(M,p), f2(M,p), ..., fn(M,p)] of its component functions. An advantage is that the single components can be evaluated and from this representation one even can directly read of the number n. A disadvantage might be, that one has to implement a lot of individual (component) functions.

For the FunctionVectorialType $f$ is implemented as a single function f(M, p), that returns an AbstractArray. And advantage here is, that this is a single function. A disadvantage might be, that if this is expensive even to compute a single component, all of f has to be evaluated

AbstractVectorGradientFunction{E, FT, JT} <: AbstractFirstOrderVectorFunction{E, FT, JT}

Represent an abstract vectorial function $f:\mathcal{M} → ℝ^n$ that provides a (component wise) gradient. The AbstractEvaluationType E indicates the evaluation type, and the AbstractVectorialTypes FT and JT the formats in which the function and the gradient are provided, see AbstractVectorFunction for an explanation.

AbstractFirstOrderVectorFunction{E, FT, JT, F, J, I} <: AbstractManifoldObjective{E}

Represent an abstract vectorial function $f:\mathcal{M} → ℝ^n$ that provides some first order differential information.

The AbstractEvaluationType E indicates the evaluation type, and the AbstractVectorialTypes FT and JT the formats in which the function and the first order information, e.g.

• a gradient – see AbstractVectorGradientFunction
• a differential (or Jacobian) – see VectorDifferentialFunction

are provided, respectively.

VectorDifferentialFunction{E, FT, JT, AJT, F, J, A, I} <: AbstractFirstOrderVectorFunction{E, FT, JT}

Represent a function $f:\mathcal{M} → ℝ^n$ including it first derivative information as its differential, and optionally its adjoint differential.

All 3 can be either single functions FunctionVectorialType or vector of functions (ComponentVectorialType)

Fields

• value!!::F: the cost function $f$, which can take different formats
• cost_type::AbstractVectorialType: indicating the format how the vector function is stored, e.g. as a single function (FunctionVectorialType, default) or as a vector of functions (ComponentVectorialType)
• jacobian!!::J: the Jacobian $J_f$of $f$
• jacobian_type::AbstractVectorialType: indicating / storing data for the type of $J_f$, e.g. as a single function (FunctionVectorialType, default) -, as a vector of functions (ComponentVectorialType), or as a function returning a matrix in coordinates at every point (CoefficientVectorialType)
• range_dimension: the number n from, the size of the vector $f$ returns.

Constructor

VectorGradientFunction(f, Jf, range_dimension;
    evaluation::AbstractEvaluationType=AllocatingEvaluation(),
    function_type::AbstractVectorialType=FunctionVectorialType(),
    jacobian_type::AbstractVectorialType=FunctionVectorialType(),
    range_dimension::Integer.
)

VectorGradientFunction(f, Jf, Jsf, range_dimension;
    evaluation::AbstractEvaluationType=AllocatingEvaluation(),
    function_type::AbstractVectorialType=FunctionVectorialType(),
    jacobian_type::AbstractVectorialType=FunctionVectorialType(),
    adjoint_jacobian_type::AbstractVectorialType=FunctionVectorialType(),
    range_dimension::Integer.
)

Create a VectorGradientFunction of f and its Jacobian Jf, and optionally its adjoint Jacobian. If the adjoint is not provided, its type is also set to Nothing

The Jacobian snf its adjoint can further be given as an allocating variant or an in-place variant, specified by the evaluation= keyword.

VectorGradientFunction{E, FT, JT, F, J, I} <: AbstractVectorGradientFunction{E, FT, JT}

Represent a function $f:\mathcal{M} → ℝ^n$ including it first derivative, either as a vector of gradients of a Jacobian

And hence has a gradient $\operatorname{grad} f_i(p) ∈ T_{p}\mathcal{M}$. Putting these gradients into a vector the same way as the functions, yields a ComponentVectorialType

\[\operatorname{grad} f(p) = \Bigl( \operatorname{grad} f_1(p), \operatorname{grad} f_2(p), …, \operatorname{grad} f_n(p) \Bigr)^\mathrm{T} ∈ (T_{p}\mathcal{M})^n\]

And advantage here is, that again the single components can be evaluated individually

Fields

• value!!::F: the cost function $f$, which can take different formats
• cost_type::AbstractVectorialType: indicating the format how the vector function is stored, e.g. as a single function (FunctionVectorialType, default) or as a vector of functions (ComponentVectorialType)
• jacobian!!::J: the Jacobian $J_f$of $f$
• jacobian_type::AbstractVectorialType: indicating / storing data for the type of $J_f$, e.g. as a single function (FunctionVectorialType, default) -, as a vector of functions (ComponentVectorialType), or as a function returning a matrix in coordinates at every point (CoefficientVectorialType)
• range_dimension: the number n from, the size of the vector $f$ returns.

Constructor

VectorGradientFunction(f, Jf, range_dimension;
    evaluation::AbstractEvaluationType=AllocatingEvaluation(),
    function_type::AbstractVectorialType=FunctionVectorialType(),
    jacobian_type::AbstractVectorialType=FunctionVectorialType(),
    range_dimension::Integer.
)

Create a VectorGradientFunction of f and its Jacobian (vector of gradients) Jf, where f maps into the Euclidean space of dimension range_dimension. Their types are specified by the function_type, and jacobian_type, respectively. The Jacobian can further be given as an allocating variant or an in-place variant, specified by the evaluation= keyword.

VectorHessianFunction{E, FT, JT, HT, F, J, H, I} <: AbstractVectorGradientFunction{E, FT, JT}

Represent a function $f:\mathcal{M} M → ℝ^n$ including it first derivative, either as a vector of gradients of a Jacobian, and the Hessian, as a vector of Hessians of the component functions.

Both the Jacobian and the Hessian can map into either a sequence of tangent spaces or a single tangent space of the power manifold of length n.

Fields

• value!!::F: the cost function $f$, which can take different formats
• cost_type::AbstractVectorialType: indicating / string data for the type of f
• jacobian!!::G: the Jacobian $J_f$ of $f$
• jacobian_type::AbstractVectorialType: indicating / storing data for the type of $J_f$
• hessians!!::H: the Hessians of $f$ (in a component wise sense)
• hessian_type::AbstractVectorialType: indicating / storing data for the type of $H_f$
• range_dimension: the number n from, the size of the vector $f$ returns.

Constructor

VectorHessianFunction(f, Jf, Hess_f, range_dimension;
    evaluation::AbstractEvaluationType=AllocatingEvaluation(),
    function_type::AbstractVectorialType=FunctionVectorialType(),
    jacobian_type::AbstractVectorialType=FunctionVectorialType(),
    hessian_type::AbstractVectorialType=FunctionVectorialType(),
)

Create a VectorHessianFunction of f and its Jacobian (vector of gradients) Jf and (vector of) Hessians, where f maps into the Euclidean space of dimension range_dimension. Their types are specified by the function_type, and jacobian_type, and hessian_type, respectively. The Jacobian and Hessian can further be given as an allocating variant or an inplace-variant, specified by the evaluation= keyword.

AbstractVectorialType

An abstract type for different representations of a vectorial function $f: \mathcal{M} → ℝ^m$ and its (component-wise) gradient/Jacobian

CoefficientVectorialType{B<:AbstractBasis} <: AbstractVectorialType

A type to indicate that the component, e.g. the Jacobian of a vectorial function $F: \mathcal{M} → ℝ^m$ is implemented in coordinates, i.e. with respect to a certain basis $\mathcal{B}$ of $T_{p}\mathcal{M}$, at $p∈\mathcal{M}$. For example the Jacobian $J_F(p) = (c_1^{\mathrm{T}},…,c_m^{\mathrm{T}})^{\mathrm{T}} ∈ ℝ^{m,d}$ is then an actual metric, where each row $c_i$ is the coordinate representation of the gradient $\operatorname{grad} f_i$ of the component functions of $F$, cf. get_coordinates.

Fields

• basis an AbstractBasis to indicate the basis with respect to which this representation is done.

Constructor

CoefficientVectorialType(basis = DefaultOrthonormalBasis())

ComponentVectorialType <: AbstractVectorialType

A type to indicate that a vectorial function $F: \mathcal{M} → ℝ^m$ or one of its ingredients is implemented in single components $f_i,\quad i=1,…,m$.

This can also be used to indicate that the Jacobian $J_F$ of $F$ is provided as single gradient functions $\operatorname{grad} f_i: \mathcal{M} → T\mathcal{M},\quad p ↦ \operatorname{grad} f_i(p) ∈ T_{p}\mathcal{M}$.

FunctionVectorialType{P<:AbstractPowerRepresentation} <: AbstractVectorialType

A type to indicate that a vectorial function $F: \mathcal{M} → ℝ^m$ is implemented as a single function.

Similarly, its Jacobian is implemented as a single function $J_F(p) ∈ (T_{p}\mathcal{M})^m$, where an AbstractPowerRepresentation is used to indicate how this $m$-fold power of the tangent space is represented.

Fields

• range::P the range this function maps into.

get_adjoint_jacobian(M::AbstractManifold, vgf::AbstractVectorGradientFunction, p, a; kwargs...)
get_adjoint_jacobian!(M::AbstractManifold, X, vgf::AbstractVectorGradientFunction, p, a; kwargs...)

Compute the adjoint Jacobian $J_F^*(p)[a]$ of a vectorial function $F: \mathcal{M} → ℝ^n$ how it acts on a vector a at p, i.e., it is given by the relation

\[⟨J_F^*(p)[a],X⟩_{p} = ⟨a,J_F(p)[X]⟩_{},\]

where the inner product on the right hand side is the standard Euclidean inner product on $ℝ^n$. To be precise, the adjoint Jacobian is defined using the Riemannian gradients of the component functions $F_i$ of $F$ as

\[J_F^*(p): ℝ^m → T_{p}\mathcal{M}, \qquad J_F^*(p)[a] = \sum_{i=1}^{m} a_i \operatorname{grad}F_i(p),\]

This can be computed in-place of X. To directly add a Jacobian to X see add_adjoint_jacobian!

get_adjoint_jacobian(M::AbstractManifold, vgf::AbstractVectorGradientFunction, p, a::AbstractVector, B::AbstractBasis; kwargs...)
get_adjoint_jacobian!(M::AbstractManifold, c, vgf::AbstractVectorGradientFunction, p, a::AbstractVector, B::AbstractBasis; kwargs...)

Compute the adjoint Jacobian $J_F^*(p)[a]$ of a vectorial function $F: \mathcal{M} → ℝ^n$ how it acts on a vector a at p, i.e., it is given by the relation

\[⟨J_F^*(p)[a],X⟩_{p} = ⟨a,J_F(p)[X]⟩_{},\]

where the basis indicates that the result should be given in coordinates c with respect to that basis. This can be done in-place of c.

Note that if vgf works internally in a basis different from the one provided, and additional change of basis is performed.

get_adjoint_jacobian(M::AbstractManifold, vgf::AbstractVectorGradientFunction, p, a; kwargs...)
get_adjoint_jacobian!(M::AbstractManifold, X, vgf::AbstractVectorGradientFunction, p, a; kwargs...)

Compute the adjoint Jacobian $J_F^*(p)[a]$ of a vectorial function $F: \mathcal{M} → ℝ^n$ how it acts on a vector a at p, i.e., it is given by the relation

\[⟨J_F^*(p)[a],X⟩_{p} = ⟨a,J_F(p)[X]⟩_{},\]

where the inner product on the right hand side is the standard Euclidean inner product on $ℝ^n$. To be precise, the adjoint Jacobian is defined using the Riemannian gradients of the component functions $F_i$ of $F$ as

\[J_F^*(p): ℝ^m → T_{p}\mathcal{M}, \qquad J_F^*(p)[a] = \sum_{i=1}^{m} a_i \operatorname{grad}F_i(p),\]

This can be computed in-place of X. To directly add a Jacobian to X see add_adjoint_jacobian!

get_adjoint_jacobian(M::AbstractManifold, vgf::AbstractVectorGradientFunction, p, a::AbstractVector, B::AbstractBasis; kwargs...)
get_adjoint_jacobian!(M::AbstractManifold, c, vgf::AbstractVectorGradientFunction, p, a::AbstractVector, B::AbstractBasis; kwargs...)

Compute the adjoint Jacobian $J_F^*(p)[a]$ of a vectorial function $F: \mathcal{M} → ℝ^n$ how it acts on a vector a at p, i.e., it is given by the relation

\[⟨J_F^*(p)[a],X⟩_{p} = ⟨a,J_F(p)[X]⟩_{},\]

where the basis indicates that the result should be given in coordinates c with respect to that basis. This can be done in-place of c.

Note that if vgf works internally in a basis different from the one provided, and additional change of basis is performed.

get_jacobian(M::AbstractManifold, vgf::AbstractVectorGradientFunction, p; kwargs...)
get_jacobian!(M::AbstractManifold, J, vgf::AbstractVectorGradientFunction, p; kwargs...)

Return the Jacobian $J_F(p): T_{p}\mathcal{M} → ℝ^m$ of a AbstractVectorGradientFunction vgf, i.e. a function $F: \mathcal{M} → ℝ^m$, where p ∈ \mathcal{M}, in matrix form with respect to a basis $\mathcal{B} = \{Y_1,…,Y_n\}$of the tangent space.

Then decomposing a tangent vector $X = \displaystyle\sum_{i=1}^{d} c_iX_i$ the evaluation of the Jacobian can be written as

\[J_F(p)[X] = J c.\]

In other words, the jth column of $J$ is given by $DF(p)[Y_j]$ and this function returns the matrix $J$. The computation can be computed in-place of J.

Keyword arguments

• basis::AbstractBasis =get_basis(vgf) basis with respect to which the matrix is built. For the CoefficientVectorialType of the vectorial functions gradient, this might lead to a change of basis, if this basis and the one the coordinates are given in do not agree.
• range::AbstractPowerRepresentation =get_range(vgf.jacobian_type) specify the range of the gradients in the case of a FunctionVectorialType, that is, on which type of power manifold the gradient(s) of the function is/are given on.

get_jacobian(M::AbstractManifold, vgf::AbstractVectorGradientFunction, p, X; kwargs...)
get_jacobian!(M::AbstractManifold, a, vgf::AbstractVectorGradientFunction, p, X; kwargs...)

Compute the Jacobian $J_F(p)$ of a vectorial function $F: \mathcal{M} → ℝ^m$, to be precise how it acts on a tangent vector X at p on the manifold M, i.e., compute

\[J_F(p)[X] = DF(p)[X] ∈ ℝ^m\]

If the gradient functions of the single component functions are provided, this is given by

\[J_F(p)[X] = \begin{pmatrix} ⟨\operatorname{grad}F_1(p),X⟩_{}\\ ⟨\operatorname{grad}F_2(p),X⟩_{}\\ \vdots\\ ⟨\operatorname{grad}F_m(p),X⟩_{}\end{pmatrix} ∈ ℝ^m\]

Given a basis $\{Y_1,…,Y_n\}$ this can also be computed in coordinates of this basis. Then it simplifies to a matrix multiplication.

This can be computed in-place of a.

get_jacobian(M::AbstractManifold, vgf::AbstractVectorGradientFunction, p, c, B::AbstractBasis; kwargs...)
get_jacobian!(M::AbstractManifold, a, vgf::AbstractVectorGradientFunction, p, c, B::AbstractBasis; kwargs...)

Compute the Jacobian $J_F(p)$ of a vectorial function $F: \mathcal{M} → ℝ^m$, to be precise how it acts on a tangent vector X at p on the manifold M, i.e., compute

\[J_F(p)[X] = DF(p)[X] ∈ ℝ^m\]

where a basis $\{Y_1,…,Y_n\}$ allows to decompose / provide the tangent vector in coordinates $c$ given by $X = \displaystyle\sum_{i=1}^{d} c_iY_i$ and the computation simplifies to a matrix multiplication.

This can be computed in-place of a.

Keyword arguments

• X::T =zero_vector(M, p): a tangent vector at the point $p$ on the manifold $\mathcal{M}$ used as memory to compute the interims tangent vector where necessary, non-allocating and/or ignored where not necessary

!!! Technical Note This variant only differs in the last argument from get_jacobian(M, vgf, p, X), which works for tangent vectors X provided directly. Hence the basis is necessary to indicate that this method shall work in coordinates. For performance reasons, try to pass get_basis(vgf.jacobian_type) as B where possible.

get_jacobian(M::AbstractManifold, vgf::AbstractVectorGradientFunction, p; kwargs...)
get_jacobian!(M::AbstractManifold, J, vgf::AbstractVectorGradientFunction, p; kwargs...)

Return the Jacobian $J_F(p): T_{p}\mathcal{M} → ℝ^m$ of a AbstractVectorGradientFunction vgf, i.e. a function $F: \mathcal{M} → ℝ^m$, where p ∈ \mathcal{M}, in matrix form with respect to a basis $\mathcal{B} = \{Y_1,…,Y_n\}$of the tangent space.

Then decomposing a tangent vector $X = \displaystyle\sum_{i=1}^{d} c_iX_i$ the evaluation of the Jacobian can be written as

\[J_F(p)[X] = J c.\]

In other words, the jth column of $J$ is given by $DF(p)[Y_j]$ and this function returns the matrix $J$. The computation can be computed in-place of J.

Keyword arguments

• basis::AbstractBasis =get_basis(vgf) basis with respect to which the matrix is built. For the CoefficientVectorialType of the vectorial functions gradient, this might lead to a change of basis, if this basis and the one the coordinates are given in do not agree.
• range::AbstractPowerRepresentation =get_range(vgf.jacobian_type) specify the range of the gradients in the case of a FunctionVectorialType, that is, on which type of power manifold the gradient(s) of the function is/are given on.

get_jacobian(M::AbstractManifold, vgf::AbstractVectorGradientFunction, p, X; kwargs...)
get_jacobian!(M::AbstractManifold, a, vgf::AbstractVectorGradientFunction, p, X; kwargs...)

Compute the Jacobian $J_F(p)$ of a vectorial function $F: \mathcal{M} → ℝ^m$, to be precise how it acts on a tangent vector X at p on the manifold M, i.e., compute

\[J_F(p)[X] = DF(p)[X] ∈ ℝ^m\]

If the gradient functions of the single component functions are provided, this is given by

\[J_F(p)[X] = \begin{pmatrix} ⟨\operatorname{grad}F_1(p),X⟩_{}\\ ⟨\operatorname{grad}F_2(p),X⟩_{}\\ \vdots\\ ⟨\operatorname{grad}F_m(p),X⟩_{}\end{pmatrix} ∈ ℝ^m\]

Given a basis $\{Y_1,…,Y_n\}$ this can also be computed in coordinates of this basis. Then it simplifies to a matrix multiplication.

This can be computed in-place of a.

get_jacobian(M::AbstractManifold, vgf::AbstractVectorGradientFunction, p, c, B::AbstractBasis; kwargs...)
get_jacobian!(M::AbstractManifold, a, vgf::AbstractVectorGradientFunction, p, c, B::AbstractBasis; kwargs...)

Compute the Jacobian $J_F(p)$ of a vectorial function $F: \mathcal{M} → ℝ^m$, to be precise how it acts on a tangent vector X at p on the manifold M, i.e., compute

\[J_F(p)[X] = DF(p)[X] ∈ ℝ^m\]

where a basis $\{Y_1,…,Y_n\}$ allows to decompose / provide the tangent vector in coordinates $c$ given by $X = \displaystyle\sum_{i=1}^{d} c_iY_i$ and the computation simplifies to a matrix multiplication.

This can be computed in-place of a.

Keyword arguments

• X::T =zero_vector(M, p): a tangent vector at the point $p$ on the manifold $\mathcal{M}$ used as memory to compute the interims tangent vector where necessary, non-allocating and/or ignored where not necessary

!!! Technical Note This variant only differs in the last argument from get_jacobian(M, vgf, p, X), which works for tangent vectors X provided directly. Hence the basis is necessary to indicate that this method shall work in coordinates. For performance reasons, try to pass get_basis(vgf.jacobian_type) as B where possible.

get_value(M::AbstractManifold, vgf::AbstractVectorFunction, p[, i=:])
get_value!(M::AbstractManifold, V, vgf::AbstractVectorFunction, p[, i=:])

Evaluate the vector function VectorGradientFunction vgf at p. The range can be used to specify a potential range, but is currently only present for consistency.

The i can be a linear index, you can provide

• a single integer
• a UnitRange to specify a range to be returned like 1:3
• a BitVector specifying a selection
• a AbstractVector{<:Integer} to specify indices
• : to return the vector of all gradients, which is also the default

This function can perform the evaluation inplace of V.

get_value_function(vgf::VectorGradientFunction, recursive=false)

return the internally stored function computing get_value.

length(vgf::AbstractVectorFunction)

Return the length of the vector the function $f: \mathcal{M} → ℝ^n$ maps into, that is the number n.

_to_iterable_indices(A::AbstractVector, i)

Convert index i (integer, colon, vector of indices, etc.) for array A into an iterable structure of indices.

_change_basis!(M::AbstractManifold, JF, p, from_basis::B1, to_basis::B; X=zero_vector(M,p))

Given a jacobian matrix JF on a manifold M at p with respect to the from_basis in the tangent space of p on M. Change the basis of the Jacobian to to_basis in place of JF.

Keyword Arguments

• X a temporary vector to store a generated vector, before decomposing it again with respect to the new basis

get_basis(::AbstractVectorialType)

Return a basis that fits a vector function representation.

For the case, where some vectorial data is stored with respect to a basis, this function returns the corresponding basis, most prominently for the CoefficientVectorialType.

If a type is not with respect to a certain basis, the DefaultOrthonormalBasis is returned.

get_range(::AbstractVectorialType)

Return an abstract power manifold representation that fits a vector function's range. Most prominently a FunctionVectorialType returns its internal range.

Otherwise the default NestedPowerRepresentation() is used to work on a vector of data.

add_adjoint_jacobian!(M::AbstractManifold, X, vgf::AbstractVectorGradientFunction, p, a; kwargs...)

Compute the adjoint Jacobian $J_F^*(p)[a]$ of a vectorial function $F: \mathcal{M} → ℝ^n$ and add it to X. For more details see get_adjoint_jacobian.

add_adjoint_jacobian!(M::AbstractManifold, c, vgf::AbstractVectorGradientFunction, p, a, B::AbstractBasis; kwargs...)

Compute the adjoint Jacobian $J_F^*(p)[a]$ of a vectorial function $F: \mathcal{M} → ℝ^n$ as a matrix in a tangent space. For more details see get_adjoint_jacobian.