The Objective

AbstractManifoldObjective

Describe the objective function $f: \mathcal{M} → ℝ$ and all its necessary ingredients, for example when it consists of several summands.

Subtypes might depend on the kind of objective in order to distinguish different available access functionality, e.g. to a gradient, or a proximal map.

Such a default component of the objective like the cost itself or the gradient should be implemented in the form

(M, v, args...) -> [...]; v

where M is a AbstractManifold, v is memory the result is computed in, as well as further arguments, most prominently usually the current iterate p.

For an allocating variant, internally the wrapper InplaceManifoldFunction should be used.

AbstractDecoratedManifoldObjective{O<:AbstractManifoldObjective}

A common supertype for all decorators of AbstractManifoldObjectives to simplify dispatch. The parameter O should refer to the undecorated objective, that is, even for multiple decorators it provides insight into the innermost one.

ReturnManifoldObjective{O2,O1<:AbstractManifoldObjective} <: AbstractDecoratedManifoldObjective{O2}

A wrapper to indicate that get_solver_result should return the inner objective.

The types are such that one can still dispatch on the undecorated type O2 of the original objective as well.

get_objective(o::AbstractManifoldObjective, recursive=true)

Return the undecorated AbstractManifoldObjective of the (possibly) decorated o. As long as your decorated objective stores the objective within o.objective and dispatch_objective_decorator is set to Val{true}, the internal objective is extracted automatically.

By default the objective that is stored within a decorated objective is assumed to be at o.objective. Overwrite _get_objective(o, ::Val{true}, recursive) to change this behaviour for your objective o for both the recursive and the direct case.

If recursive is set to false, only the most outer decorator is taken away instead of all.

dispatch_objective_decorator(o::AbstractManifoldObjective)

Indicate internally whether an AbstractManifoldObjective o is of decorating type, that is, whether it stores (encapsulates) another objective in itself, by default in the field o.objective.

Decorators indicate this by returning Val{true} for further dispatch.

The default is Val{false}, so by default an objective is not decorated.

is_objective_decorator(s::AbstractManifoldObjective)

Indicate whether the AbstractManifoldObjective s is of decorator type.

set_parameter!(amo::AbstractManifoldObjective, element::Symbol, args...)

Set a certain element from the AbstractManifoldObjective amo to a value specified by args... This function should dispatch on Val(element).

AbstractManifoldCostObjective{F} <: AbstractManifoldObjective

Representing objectives on manifolds with a cost function implemented. The parameter F represents the type of the cost function.

get_cost(M::AbstractManifold, mco::AbstractManifoldCostObjective, p)

Evaluate the cost function from within the AbstractManifoldCostObjective on M at p.

See also

get_cost_function is used here internally to access the cost function.

get_cost_function(mco::AbstractManifoldCostObjective, recursive=false)

Return the function to evaluate (just) the cost $f(p)=c$ as a function (M,p) -> c. If mco has more than one decorator, recursive determines whether just one (false) or all wrappers (true) should be “unwrapped” at once.

In the non-recursive case, this implementation assumes that the cost function is stored in mco.cost. This way you only have to implement this function if your cost function is stored in a different field.

AbstractManifoldFirstOrderObjective{F,G} <: AbstractManifoldCostObjective{F}

An abstract type for all objectives that provide

• a cost – reflected by the type F
• first order information, so either a (full) gradient or a differential, or a subgradient – reflected by the parameter G.

DirectionUpdateRule

A general functor, that handles direction update rules. Its fields are usually only a StoreStateAction by default initialized to the fields required for the specific coefficient, but can also be replaced by a (common, global) individual one that provides these values.

 get_differential(amp::AbstractManoptProblem, p, X; kwargs...)
 get_differential(M::AbstractManifold, amfo::AbstractManifoldFirstOrderObjective, p, X; kwargs...)
 get_differential(M::AbstractManifold, admo::AbstractDecoratedManifoldObjective, p, X; kwargs...)

Evaluate the differential $Df(p)[X]$ of the function $f$ represented by the AbstractManifoldFirstOrderObjective. For an AbstractManoptProblem amp the inner manifold and objectives are used. Similarly, any objective decorator would “pass through” to its inner objective. By default this falls back to $Df(p)[X] = ⟨\operatorname{grad}f(p), X⟩$.

Keyword arguments

• gradient=nothing: pass a tangent vector to be used internally as interim memory, for example in the default variant to evaluate the gradient in-place.
• evaluated=false: indicate whether gradient is just memory (false, default) or already contains the evaluated gradient (true).

d = get_differential(problem::AbstractManoptProblem, p, X; kwargs...)

Evaluate the differential of an objective and the manifold inside the AbstractManoptProblem problem.

The keyword arguments are passed down to the objective evaluation.

get_gradient(problem::AbstractManoptProblem, p)
get_gradient(M, objective::AbstractManifoldFirstOrderObjective, p)
get_gradient!(problem::AbstractManoptProblem, X, p)
get_gradient!(M, X, objective::AbstractManifoldFirstOrderObjective, p)

Evaluate the gradient of an AbstractManifoldFirstOrderObjective objective on an AbstractManifold M at a point p. This can be evaluated in-place of X and also when passing an AbstractManoptProblem problem.

X = get_subgradient(M::AbstractManifold, agmo::AbstractManifoldFirstOrderObjective, p)
get_subgradient!(M::AbstractManifold, X, agmo::AbstractManifoldFirstOrderObjective, p)

Evaluate the subgradient, which for the case of an objective having a gradient, means evaluating the gradient itself.

While in general, the result might not be deterministic, for this case it is.

AbstractGradientGroupDirectionRule <: DirectionUpdateRule

A generic DirectionUpdateRule type for all rules working with certain splittings of the overall gradient in the direction processing.

(c, d) = get_cost_and_differential(problem::AbstractManoptProblem, p, X; kwargs...)
(c, d) = get_cost_and_differential(M, objective::AbstractManifoldFirstOrderObjective, p, X; kwargs...)

Evaluate the cost and the differential of an AbstractManifoldFirstOrderObjective objective at a point p in direction X. For an AbstractManoptProblem problem the inner manifold and objectives are used. Similarly, any objective decorator would “pass through” to its inner objective.

Keyword arguments are passed down.

(c, X) = get_cost_and_gradient(problem::AbstractManoptProblem, p)
(c, X) = get_cost_and_gradient(M, objective::AbstractManifoldFirstOrderObjective, p)
(c, X) = get_cost_and_gradient!(M, X, objective::AbstractManifoldFirstOrderObjective, p)

Evaluate the cost and the gradient of an AbstractManifoldFirstOrderObjective objective at a point p simultaneously. For an AbstractManoptProblem problem the inner manifold and objectives are used. Similarly, any objective decorator would “pass through” to its inner objective.

The gradient part can be evaluated in-place of X.

 get_differential_function(objective::AbstractManifoldFirstOrderObjective, recursive::Bool=false)

Return the function to evaluate (just) the differential $Df(p)[X]$. For a decorated objective, the recursive positional parameter determines whether to directly call this function on the next decorator or whether to get the “most inner” objective.

get_gradient_function(amgo::AbstractManifoldFirstOrderObjective, recursive=false; evaluation=AllocatingEvaluation())

Return the function to evaluate (just) the gradient $\operatorname{grad} f(p)$, where either the gradient function using the decorator or without the decorator is used.

By default recursive is set to false, since usually to just pass the gradient function somewhere, one still wants for example the cached one or the one that still counts calls.

Use evaluation=InplaceEvaluation() and recursive=true to get access to the internally stored actual function. Note that this actual function might still be wrapped in an InplaceManifoldFunction.

AbstractPrimalDualManifoldObjective{C,P} <: AbstractManifoldCostObjective{C}

A common abstract super type for objectives that consider primal-dual problems.

X = adjoint_linearized_operator(M::AbstractManifold, N::AbstractManifold, apdmo::AbstractPrimalDualManifoldObjective, m, n, Y)
adjoint_linearized_operator!(M::AbstractManifold, N::AbstractManifold, X, apdmo::AbstractPrimalDualManifoldObjective, m, n, Y)

Evaluate the adjoint of the linearized forward operator, $(DΛ(m))^*[Y]$, stored within the AbstractPrimalDualManifoldObjective (in place of X). Since $Y∈T_{n}\mathcal{N}$, both $m$ and $n=Λ(m)$ are necessary arguments, mainly because the forward operator $Λ$ might be missing.

q = forward_operator(M::AbstractManifold, N::AbstractManifold, apdmo::AbstractPrimalDualManifoldObjective, p)
forward_operator!(M::AbstractManifold, N::AbstractManifold, q, apdmo::AbstractPrimalDualManifoldObjective, p)

Evaluate the forward operator $Λ(p)$ stored within the AbstractPrimalDualManifoldObjective (in place of q).

Y = get_dual_prox(N::AbstractManifold, apdmo::AbstractPrimalDualManifoldObjective, n, τ, X)
get_dual_prox!(N::AbstractManifold, Y, apdmo::AbstractPrimalDualManifoldObjective, n, τ, X)

Evaluate the proximal map of $g_n^*$ stored within the AbstractPrimalDualManifoldObjective

\[ Y = \operatorname{prox}_{τg_n^*}(X)\]

which can also be computed in place of Y.

q = get_primal_prox(M::AbstractManifold, apdmo::AbstractPrimalDualManifoldObjective, σ, p)
get_primal_prox!(M::AbstractManifold, q, apdmo::AbstractPrimalDualManifoldObjective, σ, p)

Evaluate the proximal map of $F$ stored within the AbstractPrimalDualManifoldObjective

\[\operatorname{prox}_{σF}(p)\]

which can also be computed in place of q.

get_subgradient(amp::AbstractManoptProblem, p)
get_subgradient!(amp::AbstractManoptProblem, X, p)

Evaluate the subgradient of an AbstractManoptProblem amp at point p.

The evaluation is done in place of X for the !-variant. The result might not be deterministic; one element of the subdifferential is returned.

Y = linearized_forward_operator(M::AbstractManifold, N::AbstractManifold, apdmo::AbstractPrimalDualManifoldObjective, m, X, n)
linearized_forward_operator!(M::AbstractManifold, N::AbstractManifold, Y, apdmo::AbstractPrimalDualManifoldObjective, m, X, n)

Evaluate the linearized operator (differential) $DΛ(m)[X]$ stored within the AbstractPrimalDualManifoldObjective (in place of Y), where n = Λ(m).

AbstractManifoldHessianObjective{F, G, H} <: AbstractManifoldFirstOrderObjective{F, G}

An abstract type for all objectives that provide a (full) Hessian.

get_hessian_function(mho::AbstractManifoldHessianObjective, recursive::Bool=false; evaluation=AllocatingEvaluation())

Return the function to evaluate (just) the Hessian $\operatorname{Hess} f(p)$.

For the default evaluation=AllocatingEvaluation() this function has the form (M, p, X) -> Y; for evaluation=InplaceEvaluation() it has the form (M, Y, p, X) -> Y working in-place of Y.

get_cost(TpM::TangentSpace, aslso::SymmetricLinearSystemObjective, X)

Evaluate the cost

\[f(X) = \frac{1}{2} \lVert \mathcal{A}[X] + b \rVert_{p}^2,\qquad X ∈ T_{p}\mathcal{M},\]

at X.

get_gradient(TpM::TangentSpace, aslso::AbstractSymmetricLinearSystemObjective, X)
get_gradient!(TpM::TangentSpace, Y, aslso::AbstractSymmetricLinearSystemObjective, X)

Evaluate the gradient of

\[f(X) = \frac{1}{2} \lVert \mathcal{A}[X] + b \rVert_{p}^2,\qquad X ∈ T_{p}\mathcal{M},\]

This gradient is given by $\operatorname{grad} f(X) = \mathcal{A}[X]+b$. It can be computed in-place of Y.

get_hessian(TpM::TangentSpace, aslso::AbstractSymmetricLinearSystemObjective, X, V)
get_hessian!(TpM::TangentSpace, W, aslso::AbstractSymmetricLinearSystemObjective, X, V)

Evaluate the Hessian of

\[f(X) = \frac{1}{2} \lVert \mathcal{A}[X] + b \rVert_{p}^2,\qquad X ∈ T_{p}\mathcal{M},\]

This Hessian is given by $\operatorname{Hess} f(X)[V] = \mathcal{A}[V]$. It can be computed in-place of W. Internally this (just) calls the get_linear_operator function.

AbstractSymmetricLinearSystemObjective <: AbstractManifoldObjective

Model the objective

\[f(X) = \frac{1}{2} \lVert \mathcal{A}[X] + b \rVert_{p}^2,\qquad X ∈ T_{p}\mathcal{M},\]

defined on the tangent space $T_{p}\mathcal{M}$ at $p$ on the manifold $\mathcal{M}$.

In other words this is an objective to solve $\mathcal{A}[X] = -b(p)$ for some linear symmetric operator $\mathcal{A}$ and a vector function $b$.

Concrete subtypes of this type should/could implement

• get_linear_operator to evaluate $\mathcal{A}[X]$
• get_vector_field to evaluate $b$ at $p$.

Then the following functions are available directly

• get_cost(TpM, aslso, X) to compute/evaluate the objective
• get_gradient(TpM, aslso, X) to compute/evaluate the objective's gradient at X
• get_linear_operator(TpM, aslso, X) to compute/evaluate the linear operator $\mathcal{A}$ at X

AbstractManifoldSubObjective{O<:AbstractManifoldObjective} <: AbstractManifoldObjective

An abstract type for objectives of sub problems within a solver, which still store the original objective internally to generate generic objectives for sub solvers.

get_objective(amso::AbstractManifoldSubObjective)

Return the (original) objective the sub objective is built on.

get_objective(also::AbstractLinearSurrogateObjective)

Return the objective O associated with the linear surrogate model also. By default, this returns also.objective.

AbstractLinearSurrogateObjective{O <: AbstractManifoldObjective} <: AbstractManifoldSubObjective{O}

Provide a linear surrogate model for a given AbstractManifoldObjective of type O of the form

\[μ_p(X) = \frac{1}{2}\lVert \mathcal{L}(X) + y \rVert_{2}^2 + \frac{λ}{2}\lVert X \rVert_{p}^2, \qquad\text{ for }X ∈ T_{p}\mathcal{M}, λ ≥ 0,\]

where $\mathcal{L}$ is a linear operator on the tangent space at a point $p ∈ M$ that maps into some vector space $V$ and $y ∈ V$ is a fixed vector in that space and $\lVert ⋅ \rVert$ is a norm on $V$.

Both $\mathcal{L}$ and $y$ are derived from the objective O and usually depend on the base point $p ∈ M$.

Besides the usual methods defined for AbstractManifoldObjective that may be implemented like get_cost and get_gradient, the following methods should be implemented for a concrete subtype of AbstractLinearSurrogateObjective

• get_linear_operator to compute/evaluate the linear operator $\mathcal{L}$
• get_vector_field to compute/evaluate the vector $y$
• get_objective to provide access to the underlying objective O

See also the NormalEquationsObjective for the corresponding normal equations.

get_linear_operator(M::AbstractManifold, lsmo::AbstractLinearSurrogateObjective, p)
get_linear_operator(M::AbstractManifold, lsmo::AbstractLinearSurrogateObjective, p, B::AbstractBasis)
get_linear_operator(M::AbstractManifold, lsmo::AbstractLinearSurrogateObjective, p, X)
get_linear_operator!(M::AbstractManifold, L, lsmo::AbstractLinearSurrogateObjective, p, B::AbstractBasis)
get_linear_operator!(M::AbstractManifold, Y, lsmo::AbstractLinearSurrogateObjective, p, X)

Return/Evaluate the linear operator $\mathcal{L}$ of the linear surrogate model lsmo at the point $p ∈ M$.

If a tangent vector X is provided, evaluate $\mathcal{L}(X)$. If a basis B is provided, return the matrix representation of $\mathcal{L}$ with respect to that basis. Otherwise return the operator as a function (TpM, X) -> Y.

get_normal_linear_operator(M::AbstractManifold, lsmo::AbstractLinearSurrogateObjective, p)
get_normal_linear_operator(M::AbstractManifold, lsmo::AbstractLinearSurrogateObjective, p, B::AbstractBasis)
get_normal_linear_operator(M::AbstractManifold, lsmo::AbstractLinearSurrogateObjective, p, X)
get_normal_linear_operator!(M::AbstractManifold, N, lsmo::AbstractLinearSurrogateObjective, p, B::AbstractBasis)
get_normal_linear_operator!(M::AbstractManifold, Y, lsmo::AbstractLinearSurrogateObjective, p, X)

Return/Evaluate the normal operator $\mathcal{L}^* \mathcal{L}$ of the linear surrogate model lsmo at the point $p ∈ M$.

If a tangent vector X is provided, evaluate $\mathcal{L}^* \mathcal{L}(X)$. If a basis B is provided, return the matrix representation of $\mathcal{L}^* \mathcal{L}$ with respect to that basis. Otherwise return the operator as a function (TpM, X) -> Y.

get_normal_vector_field(M::AbstractManifold, lsmo::AbstractLinearSurrogateObjective, p)
get_normal_vector_field!(M::AbstractManifold, y, lsmo::AbstractLinearSurrogateObjective, p)

Return the normal vector $\mathcal{L}^*(y)$ of the linear surrogate model lsmo at the point $p ∈ M$.

get_objective_cost(M, amso::AbstractManifoldSubObjective, p)

Evaluate the cost of the (original) objective stored within the sub objective.

X = get_objective_gradient(M, amso::AbstractManifoldSubObjective, p)
get_objective_gradient!(M, X, amso::AbstractManifoldSubObjective, p)

Evaluate the gradient of the (original) objective stored within the sub objective amso.

Y = get_objective_hessian(M, amso::AbstractManifoldSubObjective, p, X)
get_objective_hessian!(M, Y, amso::AbstractManifoldSubObjective, p, X)

Evaluate the Hessian of the (original) objective stored within the sub objective amso.

Y = get_objective_preconditioner(M, amso::AbstractManifoldSubObjective, p, X)
get_objective_preconditioner!(M, Y, amso::AbstractManifoldSubObjective, p, X)

Evaluate the preconditioner of the (original) objective stored within the sub objective amso.

get_vector_field(M::AbstractManifold, lsmo::AbstractLinearSurrogateObjective, p)
get_vector_field!(M::AbstractManifold, y, lsmo::AbstractLinearSurrogateObjective, p)

Return the vector y of the linear surrogate model lsmo at the point $p ∈ M$.