Modelling functions in Manopt.jl

AbstractEvaluationType

An abstract type to specify the kind of evaluation an AbstractManifoldObjective supports.

AllocatingEvaluation <: AbstractEvaluationType

A parameter indicating that functions work out of place.

They allocate memory for their result. This can refer to an AbstractManoptProblem, meaning the functions it contains, as well as to a single Function.

InplaceEvaluation <: AbstractEvaluationType

A parameter indicating that functions work in place.

They do not allocate memory but work on their input. This can refer to an AbstractManoptProblem, meaning the functions it contains, as well as to a single Function.

maybe_wrap_function(f, p, evaluation = InplaceEvaluation(); result = :Number)
maybe_wrap_function(f, evaluation = InplaceEvaluation(); result = :Number)

Wrap a function f defined on a manifold to work in-place on mutable variables, i.e. first if the input variable p is a number, the function f is wrapped in a MutableManifoldFunction. If the function then has an AllocatingEvaluation as its evaluation type, it is wrapped in an InplaceManifoldFunction to work in-place of the result.

The first step is skipped if the input variable p is not a number, missing or not provided.

maybe_wrap_variable(v)

For a number variable v wrap it in a 1-element vector to make it mutable. Otherwise return the variable as is.

maybe_unwrap_variable(p::P, q::P)
maybe_unwrap_variable(p::P, q::Vector{P})

Undo the wrapping performed by maybe_wrap_variable, i.e. given the original input variable p and the possibly wrapped variable q, return the unwrapped variable, i.e. if q is a 1-element vector of same element-type P as the type of p, return this one element.

AbstractApproximateHessianFunction <: AbstractManifoldFunction

An abstract supertype for approximate Hessian functions, declares them also to be functions.

AbstractDecoratedManifoldFunction{F} <: AbstractManifoldFunction

An abstract function type to indicate that another function is wrapped. Often, the wrapped function of type F is itself an AbstractManifoldFunction.

AbstractManifoldFunction <: Function

An abstract function type to wrap functions defined on Riemannian manifolds. Within Manopt.jl, most functions are assumed to work in-place, like gradient and Hessian functions. Exceptions are those functions that return single values, e.g. a cost function, the differential of a function mapping into the real or complex numbers or a boolean function like a StoppingCriterion.

set_parameter!(amf::AbstractManifoldFunction, element::Symbol, args...)

Set a certain element from the AbstractManifoldFunction amf to a value specified by args.... This function by default dispatches on the same function replacing the second argument by Val(element).

AbstractConstrainedFunction{T} <: AbstractManifoldFunction

A common supertype for functors that model constraint functions.

This supertype provides access to the fields λ and μ, the dual variables of constraints of type T.

AbstractConstrainedSlackFunction{T,R} <: AbstractManifoldFunction

A common supertype for functors that model constraint functions with slack.

This supertype additionally provides access to the fields

• μ::T: the dual for the inequality constraints
• s::T: the slack parameter
• β::R: the barrier parameter

AbstractRobustifierFunction <: Function

An abstract type to represent robustifiers, i.e., functions $ρ: ℝ → ℝ$, currently mainly used within Levenberg-Marquardt.

Usually these should be twice continuously differentiable functions with

• $ρ(0) = 0$ and $ρ'(0) = 1$, to mimic the classical least squares behaviour around zero residuals
• $ρ'(x) < 1$ in outlier regions
• $ρ''(x) < 0$ in outlier regions

Note that the robustifier is applied to the squared residuals within the nonlinear least squares framework, i.e., $ρ(f_i(p)^2)$.

(a, b, c) = get_robustifier_values(ρ::AbstractRobustifierFunction, x::Real)

Evaluate the robustifier $ρ$ and its first two derivatives at x.

Returns

A tuple (a, b, c) with

• a = ρ(x)
• b = ρ'(x)
• c = ρ''(x)

AbstractVectorFunction{FT} <: AbstractManifoldFunction

Represent an abstract vectorial function $f:\mathcal{M} → ℝ^n$ with an AbstractVectorialType to specify the format $f$ is implemented as.

Representations of $f$

There are three different representations of $f$, which might be beneficial in one or the other situation:

• the FunctionVectorialType storing a single function $f$ that returns a vector,
• the ComponentVectorialType storing a vector of functions $f_i$ that return a single value each,
• the CoefficientVectorialType storing functions with respect to a specific basis of the tangent space for gradients and Hessians. Gradients of this type are usually referred to as Jacobians.

For the ComponentVectorialType imagine that $f$ could also be written using its component functions,

\[f(p) = \bigl( f_1(p),f_2(p),…,f_n(p) \bigr)^{\mathrm{T}}\]

In this representation f is given as a vector [f1(M,p), f2(M,p), ..., fn(M,p)] of its component functions. An advantage is that the single components can be evaluated and from this representation one even can directly read off the number n. A disadvantage might be, that one has to implement a lot of individual (component) functions.

For the FunctionVectorialType $f$ is implemented as a single function f(M, p), that returns an AbstractArray. An advantage here is, that this is a single function. A disadvantage might be, that if it is expensive to compute even a single component, all of f has to be evaluated.

AbstractVectorGradientFunction{FT, JT} <: AbstractFirstOrderVectorFunction{FT, JT}

Represent an abstract vectorial function $f:\mathcal{M} → ℝ^n$ that provides a (component-wise) gradient. The AbstractVectorialTypes FT and JT indicate the formats in which the function and the gradient are provided, see AbstractVectorFunction for an explanation.

AbstractVectorialType

An abstract type for different representations of a vectorial function $f: \mathcal{M} → ℝ^m$ and its (component-wise) gradient/Jacobian.

CoefficientVectorialType{B<:AbstractBasis} <: AbstractVectorialType

A type to indicate that the component, e.g. the Jacobian of a vectorial function $F: \mathcal{M} → ℝ^m$ is implemented in coordinates, i.e. with respect to a certain basis $\mathcal{B}$ of $T_{p}\mathcal{M}$, at $p∈\mathcal{M}$. For example the Jacobian $J_F(p) = (c_1^{\mathrm{T}},…,c_m^{\mathrm{T}})^{\mathrm{T}} ∈ ℝ^{m,d}$ is then an actual matrix, where each row $c_i$ is the coordinate representation of the gradient $\operatorname{grad} f_i$ of the component functions of $F$, cf. get_coordinates.

Fields

• basis: an AbstractBasis to indicate the basis with respect to which this representation is done.

Constructor

CoefficientVectorialType(basis = DefaultOrthonormalBasis())

ComponentVectorialType <: AbstractVectorialType

A type to indicate that a vectorial function $F: \mathcal{M} → ℝ^m$ or one of its ingredients is implemented in single components $f_i,\quad i=1,…,m$.

This can also be used to indicate that the Jacobian $J_F$ of $F$ is provided as single gradient functions $\operatorname{grad} f_i: \mathcal{M} → T\mathcal{M},\quad p ↦ \operatorname{grad} f_i(p) ∈ T_{p}\mathcal{M}$.

FunctionVectorialType{P<:AbstractPowerRepresentation} <: AbstractVectorialType

A type to indicate that a vectorial function $F: \mathcal{M} → ℝ^m$ is implemented as a single function.

Similarly, its Jacobian is implemented as a single function $J_F(p) ∈ (T_{p}\mathcal{M})^m$, where an AbstractPowerRepresentation is used to indicate how this $m$-fold power of the tangent space is represented.

Fields

• range::P: the range this function maps into.

get_adjoint_jacobian(M::AbstractManifold, vgf::AbstractFirstOrderVectorFunction, p, a::AbstractVector, B::AbstractBasis; kwargs...)
get_adjoint_jacobian!(M::AbstractManifold, c, vgf::AbstractFirstOrderVectorFunction, p, a::AbstractVector, B::AbstractBasis; kwargs...)

Compute the adjoint Jacobian $J_F^*(p)[a]$ of a vectorial function $F: \mathcal{M} → ℝ^n$, which acts on a vector a at p and is given by the relation

\[⟨J_F^*(p)[a],X⟩_{p} = ⟨a,J_F(p)[X]⟩_{},\]

where the basis indicates that the result should be given in coordinates c with respect to that basis. This can be done in-place of c.

Note that if vgf works internally in a basis different from the one provided, an additional change of basis is performed.

get_adjoint_jacobian(M::AbstractManifold, vgf::AbstractFirstOrderVectorFunction, p, a; kwargs...)
get_adjoint_jacobian!(M::AbstractManifold, X, vgf::AbstractFirstOrderVectorFunction, p, a; kwargs...)

Compute the adjoint Jacobian $J_F^*(p)[a]$ of a vectorial function $F: \mathcal{M} → ℝ^n$, which acts on a vector a at p and is given by the relation

\[⟨J_F^*(p)[a],X⟩_{p} = ⟨a,J_F(p)[X]⟩_{},\]

where the inner product on the right hand side is the standard Euclidean inner product on $ℝ^n$. To be precise, the adjoint Jacobian is defined using the Riemannian gradients of the component functions $F_i$ of $F$ as

\[J_F^*(p): ℝ^n → T_{p}\mathcal{M}, \qquad J_F^*(p)[a] = \sum_{i=1}^{n} a_i \operatorname{grad}F_i(p).\]

This can be computed in-place of X. To directly add a Jacobian to X see add_adjoint_jacobian!.

get_adjoint_jacobian(M::AbstractManifold, vgf::AbstractFirstOrderVectorFunction, p, a::AbstractVector, B::AbstractBasis; kwargs...)
get_adjoint_jacobian!(M::AbstractManifold, c, vgf::AbstractFirstOrderVectorFunction, p, a::AbstractVector, B::AbstractBasis; kwargs...)

Compute the adjoint Jacobian $J_F^*(p)[a]$ of a vectorial function $F: \mathcal{M} → ℝ^n$, which acts on a vector a at p and is given by the relation

\[⟨J_F^*(p)[a],X⟩_{p} = ⟨a,J_F(p)[X]⟩_{},\]

where the basis indicates that the result should be given in coordinates c with respect to that basis. This can be done in-place of c.

Note that if vgf works internally in a basis different from the one provided, an additional change of basis is performed.

get_adjoint_jacobian(M::AbstractManifold, vgf::AbstractFirstOrderVectorFunction, p, a; kwargs...)
get_adjoint_jacobian!(M::AbstractManifold, X, vgf::AbstractFirstOrderVectorFunction, p, a; kwargs...)

Compute the adjoint Jacobian $J_F^*(p)[a]$ of a vectorial function $F: \mathcal{M} → ℝ^n$, which acts on a vector a at p and is given by the relation

\[⟨J_F^*(p)[a],X⟩_{p} = ⟨a,J_F(p)[X]⟩_{},\]

where the inner product on the right hand side is the standard Euclidean inner product on $ℝ^n$. To be precise, the adjoint Jacobian is defined using the Riemannian gradients of the component functions $F_i$ of $F$ as

\[J_F^*(p): ℝ^n → T_{p}\mathcal{M}, \qquad J_F^*(p)[a] = \sum_{i=1}^{n} a_i \operatorname{grad}F_i(p).\]

This can be computed in-place of X. To directly add a Jacobian to X see add_adjoint_jacobian!.

get_gradient(M::AbstractManifold, vgf::AbstractFirstOrderVectorFunction, p, i)
get_gradient(M::AbstractManifold, vgf::AbstractFirstOrderVectorFunction, p, i, range)
get_gradient!(M::AbstractManifold, X, vgf::AbstractFirstOrderVectorFunction, p, i)
get_gradient!(M::AbstractManifold, X, vgf::AbstractFirstOrderVectorFunction, p, i, range)

Evaluate the gradient(s) of the vector function vgf on the manifold M at p, where range specifies the representation the gradients are returned in.

Since i is assumed to be a linear index, you can provide

• a single integer
• a UnitRange to specify a range to be returned like 1:3
• a BitVector specifying a selection
• an AbstractVector{<:Integer} to specify indices
• : to return the vector of all gradients

get_jacobian(M::AbstractManifold, vgf::AbstractFirstOrderVectorFunction, p, c, B::AbstractBasis; kwargs...)
get_jacobian!(M::AbstractManifold, a, vgf::AbstractFirstOrderVectorFunction, p, c, B::AbstractBasis; kwargs...)

Compute the Jacobian $J_F(p)$ of a vectorial function $F: \mathcal{M} → ℝ^m$, to be precise the way it acts on the tangent vector at p that is given in the coordinates c with respect to the basis B, that is, compute

\[J_F(p)[X] = DF(p)[X] ∈ ℝ^m\]

where the basis $B = \{Y_1,…,Y_n\}$ allows to decompose / provide the tangent vector in coordinates $c$ given by $X = \displaystyle\sum_{i=1}^{n} c_iY_i$ and the computation simplifies to a matrix multiplication.

This can be computed in-place of a.

Keyword arguments

• X::T =zero_vector(M, p): a tangent vector at the point $p$ on the manifold $\mathcal{M}$ It is used as memory to compute the interim tangent vector where necessary, and is non-allocating and/or ignored where it is not.

get_jacobian(M::AbstractManifold, vgf::AbstractFirstOrderVectorFunction, p, X; kwargs...)
get_jacobian!(M::AbstractManifold, a, vgf::AbstractFirstOrderVectorFunction, p, X; kwargs...)

Compute the Jacobian $J_F(p)$ of a vectorial function $F: \mathcal{M} → ℝ^m$, to be precise the way it acts on a tangent vector X at p on the manifold M, that is, compute

\[J_F(p)[X] = DF(p)[X] ∈ ℝ^m\]

If the gradient functions of the single component functions are provided, this is given by

\[J_F(p)[X] = \begin{pmatrix} ⟨\operatorname{grad}F_1(p),X⟩_{}\\ ⟨\operatorname{grad}F_2(p),X⟩_{}\\ \vdots\\ ⟨\operatorname{grad}F_m(p),X⟩_{}\end{pmatrix} ∈ ℝ^m\]

Given a basis $\{Y_1,…,Y_n\}$ this can also be computed in coordinates of this basis. Then it simplifies to a matrix multiplication.

This can be computed in-place of a.

get_jacobian(M::AbstractManifold, vgf::AbstractFirstOrderVectorFunction, p; kwargs...)
get_jacobian!(M::AbstractManifold, J, vgf::AbstractFirstOrderVectorFunction, p; kwargs...)

Return the Jacobian $J_F(p): T_{p}\mathcal{M} → ℝ^m$ of an AbstractFirstOrderVectorFunction vgf, that is of a function $F: \mathcal{M} → ℝ^m$, where p ∈ \mathcal{M}, in matrix form with respect to a basis $\mathcal{B} = \{Y_1,…,Y_n\}$ of the tangent space.

Then decomposing a tangent vector $X = \displaystyle\sum_{i=1}^{n} c_iY_i$ the evaluation of the Jacobian can be written as

\[J_F(p)[X] = J c.\]

In other words, the $j$-th column of $J$ is given by $DF(p)[Y_j]$ and this function returns the matrix $J$. This can be computed in-place of J.

Keyword arguments

• basis::AbstractBasis =get_basis(vgf.jacobian_type): basis with respect to which the matrix is built. For the CoefficientVectorialType of the vectorial function's gradient, this might lead to a change of basis, if this basis and the one the coordinates are given in do not agree.
• range::AbstractPowerRepresentation =get_range(vgf.jacobian_type): specify the range of the gradients in the case of a FunctionVectorialType, that is, on which type of power manifold the gradient(s) of the function are given.

get_jacobian(M::AbstractManifold, vgf::AbstractFirstOrderVectorFunction, p, c, B::AbstractBasis; kwargs...)
get_jacobian!(M::AbstractManifold, a, vgf::AbstractFirstOrderVectorFunction, p, c, B::AbstractBasis; kwargs...)

Compute the Jacobian $J_F(p)$ of a vectorial function $F: \mathcal{M} → ℝ^m$, to be precise the way it acts on the tangent vector at p that is given in the coordinates c with respect to the basis B, that is, compute

\[J_F(p)[X] = DF(p)[X] ∈ ℝ^m\]

where the basis $B = \{Y_1,…,Y_n\}$ allows to decompose / provide the tangent vector in coordinates $c$ given by $X = \displaystyle\sum_{i=1}^{n} c_iY_i$ and the computation simplifies to a matrix multiplication.

This can be computed in-place of a.

Keyword arguments

• X::T =zero_vector(M, p): a tangent vector at the point $p$ on the manifold $\mathcal{M}$ It is used as memory to compute the interim tangent vector where necessary, and is non-allocating and/or ignored where it is not.

get_jacobian(M::AbstractManifold, vgf::AbstractFirstOrderVectorFunction, p, X; kwargs...)
get_jacobian!(M::AbstractManifold, a, vgf::AbstractFirstOrderVectorFunction, p, X; kwargs...)

Compute the Jacobian $J_F(p)$ of a vectorial function $F: \mathcal{M} → ℝ^m$, to be precise the way it acts on a tangent vector X at p on the manifold M, that is, compute

\[J_F(p)[X] = DF(p)[X] ∈ ℝ^m\]

If the gradient functions of the single component functions are provided, this is given by

\[J_F(p)[X] = \begin{pmatrix} ⟨\operatorname{grad}F_1(p),X⟩_{}\\ ⟨\operatorname{grad}F_2(p),X⟩_{}\\ \vdots\\ ⟨\operatorname{grad}F_m(p),X⟩_{}\end{pmatrix} ∈ ℝ^m\]

Given a basis $\{Y_1,…,Y_n\}$ this can also be computed in coordinates of this basis. Then it simplifies to a matrix multiplication.

This can be computed in-place of a.

get_jacobian(M::AbstractManifold, vgf::AbstractFirstOrderVectorFunction, p; kwargs...)
get_jacobian!(M::AbstractManifold, J, vgf::AbstractFirstOrderVectorFunction, p; kwargs...)

Return the Jacobian $J_F(p): T_{p}\mathcal{M} → ℝ^m$ of an AbstractFirstOrderVectorFunction vgf, that is of a function $F: \mathcal{M} → ℝ^m$, where p ∈ \mathcal{M}, in matrix form with respect to a basis $\mathcal{B} = \{Y_1,…,Y_n\}$ of the tangent space.

Then decomposing a tangent vector $X = \displaystyle\sum_{i=1}^{n} c_iY_i$ the evaluation of the Jacobian can be written as

\[J_F(p)[X] = J c.\]

In other words, the $j$-th column of $J$ is given by $DF(p)[Y_j]$ and this function returns the matrix $J$. This can be computed in-place of J.

Keyword arguments

• basis::AbstractBasis =get_basis(vgf.jacobian_type): basis with respect to which the matrix is built. For the CoefficientVectorialType of the vectorial function's gradient, this might lead to a change of basis, if this basis and the one the coordinates are given in do not agree.
• range::AbstractPowerRepresentation =get_range(vgf.jacobian_type): specify the range of the gradients in the case of a FunctionVectorialType, that is, on which type of power manifold the gradient(s) of the function are given.

AbstractFirstOrderVectorFunction{FT, JT} <: AbstractVectorFunction{FT}

Represent an abstract vectorial function $f:\mathcal{M} → ℝ^n$ that provides some first order differential information.

The AbstractVectorialTypes FT and JT indicate the formats in which the function and the first order information are provided, respectively. The first order information is either

• a gradient – see AbstractVectorGradientFunction
• a differential (or Jacobian) – see VectorDifferentialFunction

length(vgf::AbstractVectorFunction)

Return the length of the vector the function $f: \mathcal{M} → ℝ^n$ maps into, that is the number n.

get_basis(::AbstractVectorialType)

Return a basis that fits a vector function representation.

For the case, where some vectorial data is stored with respect to a basis, this function returns the corresponding basis, most prominently for the CoefficientVectorialType.

If a type is not with respect to a certain basis, the DefaultOrthonormalBasis is returned.

_change_basis!(M::AbstractManifold, JF, p, from_basis::B1, to_basis::B; X=zero_vector(M,p))

Given a Jacobian matrix JF on a manifold M at p with respect to the from_basis in the tangent space of p on M, change the basis of the Jacobian to to_basis in place of JF.

Keyword arguments

• X: a temporary vector to store a generated vector, before decomposing it again with respect to the new basis

_to_iterable_indices(A::AbstractVector, i)

Convert index i (integer, colon, vector of indices, etc.) for array A into an iterable structure of indices.

add_adjoint_jacobian!(M::AbstractManifold, c, vgf::AbstractFirstOrderVectorFunction, p, a, B::AbstractBasis; kwargs...)

Compute the adjoint Jacobian $J_F^*(p)[a]$ of a vectorial function $F: \mathcal{M} → ℝ^n$ as a matrix in a tangent space. For more details see get_adjoint_jacobian.

add_adjoint_jacobian!(M::AbstractManifold, X, vgf::AbstractFirstOrderVectorFunction, p, a; kwargs...)

Compute the adjoint Jacobian $J_F^*(p)[a]$ of a vectorial function $F: \mathcal{M} → ℝ^n$ and add it to X. For more details see get_adjoint_jacobian.

get_range(::AbstractVectorialType)

Return an abstract power manifold representation that fits a vector function's range. Most prominently a FunctionVectorialType returns its internal range.

Otherwise the default NestedPowerRepresentation() is used to work on a vector of data.

get_value(M::AbstractManifold, vgf::AbstractVectorFunction, p[, i=:])
get_value!(M::AbstractManifold, V, vgf::AbstractVectorFunction, p[, i=:])

Evaluate the AbstractVectorFunction vgf at p.

Since i is assumed to be a linear index, you can provide

• a single integer
• a UnitRange to specify a range to be returned like 1:3
• a BitVector specifying a selection
• an AbstractVector{<:Integer} to specify indices
• : to return the vector of all values, which is also the default

This function can perform the evaluation in-place of V.