Objectives

ConstrainedManifoldObjective{C<:ConstraintType} <: AbstractManifoldObjective

Describes a constrained objective

\[\begin{aligned} \operatorname*{arg\,min}_{p ∈ \mathcal{M}} & f(p)\\ \text{subject to}\quad&g_i(p) ≤ 0 \quad \text{ for } i= 1, …, m,\\ \quad & h_j(p)=0 \quad \text{ for } j=1,…,n, \end{aligned}\]

Fields

• objective: an AbstractManifoldObjective representing the unconstrained objective, that is containing cost $f$, the gradient of the cost $f$ and maybe the Hessian.
• equality_constraints: an AbstractManifoldObjective representing the equality constraints $h: \mathcal{M} → ℝ^n$ also possibly containing its gradient and/or Hessian
• inequality_constraints: an AbstractManifoldObjective representing the inequality constraints $g: \mathcal{M} → ℝ^m$ also possibly containing its gradient and/or Hessian

Constructors

ConstrainedManifoldObjective(f, grad_f;    equality_constraints=nothing,    inequality_constraints=nothing,    g=missing, grad_g=missing,    h=missing, grad_h=missing,    hess_f=missing, hess_g=missing, hess_h=missing,    evaluation=AllocatingEvaluation(),    M = missing,    p = ismissing(M) ? missing : rand(M),    atol = 0,)

Generate the constrained objective based on all involved single functions f, grad_f, g, grad_g, h, grad_h, and optionally a Hessian for each of these. With equality_constraints and inequality_constraints you have to provide the dimension of the ranges of h and g, respectively. You can also provide a manifold M and a point p to use one evaluation of the constraints to automatically try to determine these sizes.

ConstrainedManifoldObjective(mho::AbstractManifoldObjective;    equality_constraints = nothing,    inequality_constraints = nothing)

Generate the constrained objective either with explicit constraints $g$ and $h$, and their gradients, or in the form where these are already encapsulated in VectorGradientFunctions.

Both variants require that at least one of the constraints (and its gradient) is provided. If any of the three parts provides a Hessian, the corresponding object, that is a ManifoldHessianObjective for f or a VectorHessianFunction for g or h, respectively, is created.

Feasibility of points with respect to the constraints is determined up to the tolerance atol.

EmbeddedManifoldObjective{P, T, O2, O1<:AbstractManifoldObjective} <:
   AbstractDecoratedManifoldObjective{O2}

Declare an objective to be defined in the embedding. This also declares the gradient to be defined in the embedding, and especially being the Riesz representer with respect to the metric in the embedding. The types can be used to still dispatch on also the undecorated objective type O2, in case the objective being stored here is decorated, e.g. with a cache.

Fields

• objective: the objective that is defined in the embedding
• p=missing: a point in the embedding.
• X=missing: a tangent vector in the embedding

When a point in the embedding p is provided, embed! is used in place of this point to reduce memory allocations. Similarly X is used when embedding tangent vectors.

ManifoldAlternatingGradientObjective{F,G} <: AbstractManifoldFirstOrderObjective{F, G}

An alternating gradient objective consists of

• a cost function $F(p)$
• a gradient $\operatorname{grad}F$ that is either
  • given as one function $\operatorname{grad}F$ returning a tangent vector X on M or
  • an array of gradient functions $\operatorname{grad}F_i$, i=1,…,n, each returning a component of the gradient
  which might be allocating or mutating variants, but not a mix of both.

Constructors

ManifoldAlternatingGradientObjective(F, gradF::Function; evaluation=AllocatingEvaluation(), p = missing)ManifoldAlternatingGradientObjective(F, gradF::AbstractVector{<:Function}; evaluation=AllocatingEvaluation(), p = missing)

Create an alternating gradient problem with an optional cost and the gradient either as one function (returning an array) or a vector of functions.

Keyword Arguments

• evaluation::AbstractEvaluationType=AllocatingEvaluation(): specify whether the functions that return an array, for example a point or a tangent vector, work by allocating its result (AllocatingEvaluation) or whether they modify their input argument to return the result therein (InplaceEvaluation). Since usually the first argument is the manifold, the modified argument is the second.
• p = missing provide a point to automatically ensure the functions of the objective “act” on mutating variables.

ManifoldCachedObjective{P,O<:AbstractManifoldObjective,C<:NamedTuple{}} <: AbstractDecoratedManifoldObjective{P}

Create a cache for an objective, based on a NamedTuple that stores some kind of cache.

Constructor

ManifoldCachedObjective(M, o::AbstractManifoldObjective, caches::Vector{Symbol}; kwargs...)

Create a cache for the AbstractManifoldObjective where the Symbols in caches indicate, which function evaluations to cache.

Supported symbols

Keyword arguments

• p=rand(M): the type of the keys to be used in the caches. Defaults to the default representation on M.
• value=get_cost(M, objective, p): the type of values for numeric values in the cache
• X=zero_vector(M,p): the type of values to be cached for gradient and Hessian calls.
• cache_size=10: number of (least recently used) calls to cache
• cache_sizes=Dict{Symbol,Int}(): a named tuple or dictionary specifying the sizes individually for each cache.

ManifoldConstrainedSetObjective{MO, PF, IF} <: AbstractManifoldObjective

Model a constrained objective restricted to a set

\[\operatorname*{arg\,min}_{p ∈ \mathcal{C}} f(p)\]

where $\mathcal{C} ⊂ \mathcal{M}$ is a convex closed subset.

Fields

• objective::AbstractManifoldObjective the (unconstrained) objective, which contains $f$ and for example its gradient $\operatorname{grad} f$.
• project!::PF a projection function $\operatorname{proj}_{\mathcal{C}}: \mathcal{M} → \mathcal{C}$ that projects onto the set $\mathcal{C}$.
• indicator::IF the indicator function $ι_{\mathcal{C}}(p) = \begin{cases} 0 &\text{ for }p∈\mathcal{C}\\\\ ∞ &\text{ else.}\end{cases}$

Constructor

ManifoldConstrainedSetObjective(f, grad_f, project!; kwargs...)

Generate the constrained objective for a given function f its gradient grad_f and a projection project! $\operatorname{proj}_{\mathcal{C}}$.

Keyword arguments

• indicator=missing: the indicator function $ι_{\mathcal{C}}(p)$. If not provided, a test whether the projection yields the same point is performed. For the InplaceEvaluation this requires one allocation.

ManifoldCostGradientObjective(costgrad; kwargs...)

Create an objective containing one function to perform a combined computation of cost and its gradient

Internally this is stored in a ManifoldFirstOrderObjective. The kwargs... are also passed to this representation, which allows to add a special function to evaluate the differential.

Used with

gradient_descent, conjugate_gradient_descent, quasi_Newton

ManifoldCostObjective{F} <: AbstractManifoldCostObjective{F}

Specify an AbstractManifoldObjective that does only have information about the cost function $f: \mathcal{M} → ℝ$ implemented as a function (M, p) -> c to compute the cost value c at p on the manifold M.

Fields

• cost: a function $f: \mathcal{M} → ℝ$ to minimize

Constructors

ManifoldCostObjective(f; p::P = missing) where {P}ManifoldCostObjective(f, ::Type{P}) where {P}

Generate a ManifoldCostObjective with cost function f.

Keyword Arguments

• p = missing provide a point to automatically ensure the functions of the objective “act” on mutating variables.

See also

NelderMead, particle_swarm

ManifoldCountObjective{P,O<:AbstractManifoldObjective,I<:Union{<:Integer,AbstractVector{<:Integer}}} <: AbstractDecoratedManifoldObjective{P}

A wrapper for any AbstractManifoldObjective of type O to count different calls to parts of the objective.

Fields

• counts a dictionary of symbols mapping to integers keeping the counted values
• objective the wrapped objective

Supported symbols

Constructors

ManifoldCountObjective(objective::AbstractManifoldObjective, counts::Dict{Symbol, <:Integer})

Initialize the ManifoldCountObjective to wrap objective, initializing the set of counts.

ManifoldCountObjective(M::AbstractManifold, objective::AbstractManifoldObjective, count::AbstractVector{Symbol}, init=0)

Count function calls on objective using the symbols in count initializing all entries to init.

ManifoldFirstOrderObjective{F<:NamedTuple} <: AbstractManifoldFirstOrderObjective{F, F}

Specify an objective containing a cost and its gradient or differential.

Fields

• functions::F: a function or a tuple of functions containing the cost and first order information.

Currently the following cases are covered, sorted by their popularity

1. a single function fg representing a combined function (M, X, p) -> (c, X) that computes the cost c=cost(M,p) and gradient X=grad_f(M, X, p);
2. a single function fdf representing a combined function (M, d, p) -> (c, d) that computes the cost c=cost(M,p) and differential d=diff_f(M, d, p);
3. pairs of single functions (f, g), (f, df) of a cost function f and either its gradient g or its differential d, respectively
4. The function (fg, d) and (fdf, g) from 1 and 2, respectively joined by the other missing third information, the differential for the first or the gradient for the second
5. a tuple (f, g, d) of three functions, computing cost f, gradient g, and differential d separately
6. a tuple (f, gd) of a cost function and a combined function (X, d) = gd(M, (X,d), p) to compute gradient and differential together

In all cases a gradient and/or a differential that is present is assumed to work in-place, see the InplaceManifoldFunction wrapper for alternatives.

The cases of a common fg function for cost and gradient and the tuple (f,g) are the most common ones. They can also be addressed by their alternate constructors ManifoldCostGradientObjective(fg) and ManifoldGradientObjective(f,g), respectively.

Constructors

ManifoldFirstOrderObjective(; kwargs...)

Keyword arguments

• cost = missing the cost function c = f(M,p)
• costdifferential = missing the combined cost and differential function fdf(M, p, X)
• costgradient = missing the combined cost and gradient function fg(M,p) or in-place fg!(M, X, p)
• differential = missing the differential d = df(M, p, X)
• evaluation::AbstractEvaluationType=AllocatingEvaluation(): specify whether the functions that return an array, for example a point or a tangent vector, work by allocating its result (AllocatingEvaluation) or whether they modify their input argument to return the result therein (InplaceEvaluation). Since usually the first argument is the manifold, the modified argument is the second.
• gradient=missing the gradient function g(M, p) or in-place g!(M, X, p)
• p = missing provide a point to automatically ensure the functions of the objective “act” on mutating variables.

Where:

• At least one of cost, costgradient or costdifferential must be provided.
• Either gradient, costgradient, differential or costdifferential must be provided.
• If more than one function provides the same thing (e.g. cost), it is assumed that all such functions return the same value. Optimization algorithms will attempt to make the most efficient use of provided functions fitting for the access required.

Used with

gradient_descent, conjugate_gradient_descent, quasi_Newton

ManifoldGradientObjective(cost, gradient; kwargs...)

Generate an objective with a function cost and its gradient. The gradient is assumed to work in-place

• as a function (M, X, p) -> X that works in place of X, an InplaceEvaluation

Internally this is stored in a ManifoldFirstOrderObjective. The kwargs... are also passed to this representation, which allows to add a special function to evaluate the differential.

Used with

gradient_descent, conjugate_gradient_descent, quasi_Newton

ManifoldHessianObjective{C,G,H,Pre} <: AbstractManifoldHessianObjective{C,G,H}

Specify a problem for Hessian based algorithms.

Fields

• cost: a function $f:\mathcal{M}→ℝ$ to minimize
• gradient: the gradient $\operatorname{grad}f:\mathcal{M} → T\mathcal{M}$ of the cost function $f$
• hessian: the Hessian $\operatorname{Hess}f(p)[⋅]: T_{p}\mathcal{M} → T_{p}\mathcal{M}$ of the cost function $f$
• preconditioner: the symmetric, positive definite preconditioner as an approximation of the inverse of the Hessian of $f$, a map with the same input variables as the hessian to numerically stabilize iterations when the Hessian is ill-conditioned

Constructor

ManifoldHessianObjective(    f, grad_f, Hess_f, preconditioner = missing;    p = missing, evaluation=AllocatingEvaluation())

Generate a ManifoldHessianObjective from a cost function f, its gradient grad_f, its Hessian Hess_f, and an optional preconditioner preconditioner. The evaluation keyword argument can be used to specify whether the functions are in-place or allocating, a point p can be used to specify the domain of the functions, which only has a “wrapping” effect if p is an immutable number.

See also

truncated_conjugate_gradient_descent, trust_regions

ManifoldNonlinearLeastSquaresObjective{VFV,RFV,TVC} <: AbstractManifoldFirstOrderObjective{VFV, VFV}

An objective to model the robustified nonlinear least squares problem

\[\operatorname*{arg\,min}_{p ∈ \mathcal{M}} f(p), \qquad f(p) = \frac{1}{2} \sum_{i=1}^{m} ρ_i \bigl( \lVert F_i(p) \rVert^2 \bigr)\]

where $F_i: \mathcal{M} → ℝ^{n_i}$ is the $i$th block component of length $n_i > 0$ and each $ρ_i: ℝ → ℝ$ is a robustifier function, cf. AbstractRobustifierFunction, for each such a block component. The overall residual function is denoted by $F: \mathcal{M} → ℝ^{n}$ with $n = \sum_{i=1}^{m} n_i$ and concatenates all block components.

Fields

• objective: a vector of AbstractFirstOrderVectorFunctions, one for each block component cost function $F_i$, which might internally also be a vector of component costs $(F_i)_j$, as well as their Jacobian $J_{F_i}$ or a vector of gradients $\operatorname{grad} (F_i)_j$ depending on the specified AbstractVectorialTypes.
• robustifier: a vector of AbstractRobustifierFunctions, one for each block component cost function $F_i$.
• value_cache::AbstractVector an internal cache to store the result of evaluating the cost functions

Constructors

ManifoldNonlinearLeastSquaresObjective(f, jacobian, range_dimension::Integer, robustifier=IdentityRobustifier(); kwargs...)

Create a nonlinear least squares objective for a single vectorial function f and its jacobian, where range_dimension is the dimension of the vector space f maps into. These three are internally wrapped into a VectorGradientFunction and this then calls the following constructor.

ManifoldNonlinearLeastSquaresObjective(vf::AbstractFirstOrderVectorFunction, robustifier::AbstractRobustifierFunction=IdentityRobustifier())

Create a nonlinear least squares objective for a given vectorial function. Note that for this constructor the robustifier is applied componentwise to each component of vf, i.e. wrapped in a ComponentwiseRobustifierFunction. Internally this wraps both vf and robustifier in an array and calls the next constructor. Hence to not use the componentwise robustifier but a global one, pass [vf,] and [robustifier,] instead.

ManifoldNonlinearLeastSquaresObjective(fs::Vector{<:AbstractFirstOrderVectorFunction}, robustifiers::Vector{<:AbstractRobustifierFunction}=fill(IdentityRobustifier(), length(fs)))

Given a vector of AbstractFirstOrderVectorFunctions to represent the single blocks and a vector of robustifiers, one for each block, create the corresponding nonlinear least squares objective.

Keyword arguments

The first constructor, that is the variant for a single block, accepts the following keyword arguments, which are passed on to the corresponding VectorGradientFunction constructor.

• function_type::AbstractVectorialType=FunctionVectorialType(): specify the format the residuals are given in. By default a function returning a vector.
• jacobian_tangent_basis::AbstractBasis=DefaultOrthonormalBasis(): shortcut to specify the basis the Jacobian matrix is build with.
• jacobian_type::AbstractVectorialType=CoefficientVectorialType(jacobian_tangent_basis): specify the format the Jacobian is given in. By default a matrix of the differential with respect to a certain basis of the tangent space.

See also

LevenbergMarquardt, LevenbergMarquardtState

ManifoldProximalGradientObjective{TC, TG, TGG, TP} <: AbstractManifoldCostObjective{TC}

Model an objective of the form

\[f(p) = g(p) + h(p), \qquad p ∈ \mathcal{M},\]

where $g: \mathcal{M} → \bar{\mathbb R}$ is a differentiable function and $h: \mathcal{M} → \bar{\mathbb R}$ is a (possibly) lower semicontinuous and proper function.

This objective provides the total cost $f$, its smooth component $g$, as well as $\operatorname{grad} g$ and $\operatorname{prox}_{λ h}$.

Fields

• cost: the overall cost $f = g + h$
• cost_smooth: the smooth cost component $g$
• gradient_g!: the gradient $\operatorname{grad} g$
• proximal_map_h!: the proximal map $\operatorname{prox}_{λ h}$

Constructor

ManifoldProximalGradientObjective(f, g, grad_g, prox_h = missing)

Generate the proximal gradient objective given the total cost $f = g + h$, smooth cost $g$, the gradient of the smooth component $\operatorname{grad} g$, and the proximal map of the nonsmooth component $\operatorname{prox}_{λ h}$.

Keyword Arguments

• evaluation::AbstractEvaluationType=AllocatingEvaluation(): specify whether the functions that return an array, for example a point or a tangent vector, work by allocating its result (AllocatingEvaluation) or whether they modify their input argument to return the result therein (InplaceEvaluation). Since usually the first argument is the manifold, the modified argument is the second.
• p = missing provide a point to automatically ensure the functions of the objective “act” on mutating variables.

ManifoldProximalMapObjective{TC, TP, V} <: AbstractManifoldCostObjective{TC}

Specify a problem for solvers based on the evaluation of proximal maps, which represents proximal maps $\operatorname{prox}_{λf_i}$ for summands $f = f_1 + f_2+ … + f_N$ of the cost function $f$.

Fields

• cost: a function $f:\mathcal{M}→ℝ$ to minimize
• proxes: proximal maps $\operatorname{prox}_{λf_i}:\mathcal{M} → \mathcal{M}$ as functions (M, λ, p) -> q or in-place (M, q, λ, p).
• number_of_proxes: number of proximal maps per function, to specify when one of the maps is a combined one such that the proximal maps functions return more than one entry per function, you have to adapt this value. If not specified, it is set to one prox per function.

Constructor

ManifoldProximalMapObjective( f, proxes_f::Union{Tuple,AbstractVector}, number_of_proxes=ones(length(proxes_f)) )

Generate a proximal problem with a tuple or vector of functions, where by default every function computes a single prox of one component of $f$.

ManifoldProximalMapObjective(f, prox_f)

Generate a proximal objective for $f$ and its proximal map $\operatorname{prox}_{λf}$.

Keyword Arguments

• evaluation::AbstractEvaluationType=AllocatingEvaluation(): specify whether the functions that return an array, for example a point or a tangent vector, work by allocating its result (AllocatingEvaluation) or whether they modify their input argument to return the result therein (InplaceEvaluation). Since usually the first argument is the manifold, the modified argument is the second.
• p = missing provide a point to automatically ensure the functions of the objective “act” on mutating variables.

See also

cyclic_proximal_point, get_cost, get_proximal_map

ManifoldStochasticGradientObjective{C, G} <: AbstractManifoldFirstOrderObjective{C, G}

A stochastic gradient objective consists of

• a(n optional) cost function $f(p) = \displaystyle\sum_{i=1}^{n} f_i(p)$
• an array of gradients, $\operatorname{grad} f_i(p), i=1,…,n$ which can be given in two forms
  • as one single function $(\mathcal{M}, p) ↦ (X_1,…,X_n) ∈ (T_{p}\mathcal{M})^n$
  • as a vector of functions $\bigl( (\mathcal{M}, p) ↦ X_1, …, (\mathcal{M}, p) ↦ X_n\bigr)$.

Where both variants are functions (M, X, p) -> X, where X is the vector of X1,...,Xn and (M, X1, p) -> X1, ..., (M, Xn, p) -> Xn, respectively.

Constructors

ManifoldStochasticGradientObjective(grad_f::Function; cost=missing)ManifoldStochasticGradientObjective(grad_f::AbstractVector{<:Function}; cost=missing)

Create a Stochastic gradient problem with the gradient either as one function (returning an array of tangent vectors) or a vector of functions (each returning one tangent vector).

The optional cost can also be given as either a single function (returning a number) or a vector of functions, each returning a value.

Keyword Arguments

• cost = missing provide the cost e.g. when it is used in a stopping criterion.
• evaluation::AbstractEvaluationType=AllocatingEvaluation(): specify whether the functions that return an array, for example a point or a tangent vector, work by allocating its result (AllocatingEvaluation) or whether they modify their input argument to return the result therein (InplaceEvaluation). Since usually the first argument is the manifold, the modified argument is the second.
• p = missing provide a point to automatically ensure the functions of the objective “act” on mutating variables.

Used with

stochastic_gradient_descent

Note that this can also be used with a gradient_descent, since the (complete) gradient is just the sums of the single gradients.

ManifoldSubgradientObjective{C,S} <: AbstractManifoldCostObjective{C}

A structure to store information about an objective for a subgradient based optimization problem

Fields

• cost: the function $f$ to be minimized
• subgradient: a function returning a subgradient $∂f$ of $f$

Constructor

ManifoldSubgradientObjective(f, ∂f; evaluation = AllocatingEvaluation(), p = missing)

Generate the ManifoldSubgradientObjective for a subgradient objective, consisting of a (cost) function f(M, p) and a function ∂f(M, p) that returns a not necessarily deterministic element from the subdifferential at p on a manifold M.

Keyword Arguments

• evaluation::AbstractEvaluationType=AllocatingEvaluation(): specify whether the functions that return an array, for example a point or a tangent vector, work by allocating its result (AllocatingEvaluation) or whether they modify their input argument to return the result therein (InplaceEvaluation). Since usually the first argument is the manifold, the modified argument is the second.
• p = missing provide a point to automatically ensure the functions of the objective “act” on mutating variables.

ScaledManifoldObjective{O2, O1<:AbstractManifoldObjective,F} <: AbstractDecoratedManifoldObjective{O2}

Declare an objective to be defined as a scaled version of an existing objective.

This rescales all involved functions.

For now the functions rescaled are

• the cost
• the gradient
• the Hessian

Fields

• objective: the objective that is scaled
• scale=1: the scaling applied

Constructors

ScaledManifoldObjective(objective, scale::Real=1)

Generate a scaled manifold objective based on objective, where scale is 1 by default. The unary minus and the multiplication from the left with a scalar are also overloaded.

- objective

The single-parameter minus, that is scale=-1, is overloaded to have a short notation turning a maximization problem into a minimization one, which fits the framework provided within Manopt.jl.

scale * objective

Equivalent to the first constructor, but might be nicer to write in a few places.

SimpleManifoldCachedObjective{O<:AbstractManifoldObjective, P, T, C} <: AbstractDecoratedManifoldObjective{O}

Provide a simple cache for an AbstractManifoldFirstOrderObjective that is, this cache stores a point p and a gradient $\operatorname{grad} f(p)$ in X as well as a cost value $f(p)$ in c. It can also easily evaluate the differential based on the cached gradient.

Both X and c are accompanied by booleans to keep track of their validity.

While this does not provide a cache for the differential, it uses the cached gradient as a help to evaluate the differential, if an up-to-date gradient is available. It otherwise does call the original differential.

This simple cache does not take into account, that some first order objectives have a common function for cost & grad. It only caches the function that is actually called.

Constructors

SimpleManifoldCachedObjective(M::AbstractManifold, obj::AbstractManifoldFirstOrderObjective; kwargs...)

Keyword arguments

• p=rand(M): a point on the manifold to initialize the cache with
• X=get_gradient(M, obj, p) or zero_vector(M,p): a tangent vector to store the gradient in, see also initialized=
• c=get_cost(M, obj, p) or 0.0: a value to store the cost in, see also initialized=
• initialized=true: whether to initialize the cached X and c or not.

where both for p and X copies are generated before they are stored.

SimpleManifoldCachedObjective(obj::AbstractManifoldFirstOrderObjective, p, X, c; initialized = false)

Similar as above but initializing all fields directly and without copies and initialized indicated whether the three values correspond to an evaluation from obj.

equality_constraints_length(co::ConstrainedManifoldObjective)

Return the number of equality constraints of an ConstrainedManifoldObjective. This acts transparently through AbstractDecoratedManifoldObjectives

get_cost(M::AbstractManifold, emo::EmbeddedManifoldObjective, p)

Evaluate the cost function of an objective defined in the embedding by first embedding p before calling the cost function stored in the EmbeddedManifoldObjective.

get_cost(M::AbstractManifold, nlso::ManifoldNonlinearLeastSquaresObjective, p)

Compute the cost of the least squares objective, i.e.

\[\frac{1}{2} \sum_{i=1}^{m} ρ_i \bigl( \lVert F_i(p) \rVert^2 \bigr),\]

where $F_i: \mathcal{M} → ℝ^{n_i}$ is the $i$th block component of length $n_i > 0$ and each $ρ_i: ℝ → ℝ$ is a robustifier function, cf. AbstractRobustifierFunction, one for each such block component.

get_cost(M::AbstractManifold, scaled_objective::ScaledManifoldObjective, p)

Evaluate the scaled objective $s*f(p)$.

get_cost(M::AbstractManifold, sgo::ManifoldStochasticGradientObjective, p, i)

Evaluate the ith summand of the cost.

If you use a single function for the stochastic cost, then only the index i=1 is available to evaluate the whole cost.

get_cost_smooth(M::AbstractManifold, objective, p)

Helper function to extract the smooth part g of a proximal gradient objective at the point p.

get_count(co::ManifoldCountObjective, s::Symbol, mode::Symbol=:None)

Get the number of counts for a certain symbol s.

Depending on the mode different results appear if the symbol does not exist in the dictionary

• :None: (default) silent mode, returns -1 for non-existing entries
• :warn: issues a warning if a field does not exist
• :error: issues an error if a field does not exist

get_equality_constraint(amp::AbstractManoptProblem, p, j=:)
get_equality_constraint(M::AbstractManifold, objective, p, j=:)

Evaluate equality constraints of a ConstrainedManifoldObjective objective at point p and indices j (by default : which corresponds to all indices).

get_equality_constraint(M::AbstractManifold, emo::EmbeddedManifoldObjective, p, j)

Evaluate the jth equality constraint $h_j(p)$ defined in the embedding, that is embed p before calling the constraint functions stored in the EmbeddedManifoldObjective.

get_grad_equality_constraint(amp::AbstractManoptProblem, p, j)
get_grad_equality_constraint(M::AbstractManifold, co::ConstrainedManifoldObjective, p, j, range=NestedPowerRepresentation())
get_grad_equality_constraint!(amp::AbstractManoptProblem, X, p, j)
get_grad_equality_constraint!(M::AbstractManifold, X, co::ConstrainedManifoldObjective, p, j, range=NestedPowerRepresentation())

Evaluate the gradient or gradients of the equality constraint $(\operatorname{grad} h(p))_j$ or $\operatorname{grad} h_j(p)$,

See also the ConstrainedManoptProblem to specify the range of the gradient.

X = get_grad_equality_constraint(M::AbstractManifold, emo::EmbeddedManifoldObjective, p, j)
get_grad_equality_constraint!(M::AbstractManifold, X, emo::EmbeddedManifoldObjective, p, j)

Evaluate the gradient of the jth equality constraint $\operatorname{grad} h_j(p)$ defined in the embedding, that is embed p before calling the gradient function stored in the EmbeddedManifoldObjective.

The returned gradient is then converted to a Riemannian gradient calling riemannian_gradient.

get_grad_inequality_constraint(amp::AbstractManoptProblem, p, j=:)
get_grad_inequality_constraint(M::AbstractManifold, co::ConstrainedManifoldObjective, p, j=:, range=NestedPowerRepresentation())
get_grad_inequality_constraint!(amp::AbstractManoptProblem, X, p, j=:)
get_grad_inequality_constraint!(M::AbstractManifold, X, co::ConstrainedManifoldObjective, p, j=:, range=NestedPowerRepresentation())

Evaluate the gradient or gradients of the inequality constraint $(\operatorname{grad} g(p))_j$ or $\operatorname{grad} g_j(p)$,

See also the ConstrainedManoptProblem to specify the range of the gradient.

X = get_grad_inequality_constraint(M::AbstractManifold, emo::EmbeddedManifoldObjective, p, j)
get_grad_inequality_constraint!(M::AbstractManifold, X, emo::EmbeddedManifoldObjective, p, j)

Evaluate the gradient of the jth inequality constraint $\operatorname{grad} g_j(p)$ defined in the embedding, that is embed p before calling the gradient function stored in the EmbeddedManifoldObjective.

The returned gradient is then converted to a Riemannian gradient calling riemannian_gradient.

get_gradient(M::AbstractManifold, nlso::ManifoldNonlinearLeastSquaresObjective, p; kwargs...)
get_gradient!(M::AbstractManifold, X, nlso::ManifoldNonlinearLeastSquaresObjective, p; kwargs...)

Compute the gradient for the ManifoldNonlinearLeastSquaresObjective nlso at the point $p ∈ M$, i.e.

\[\operatorname{grad} f(p) = \sum_{i=1}^{m} ρ'_i\bigl(\lVert F_i(p) \rVert_{2}^2\bigr) \sum_{j=1}^{n_i} f_{i,j}(p) \operatorname{grad} f_{i,j}(p)\]

where $F_i(p) ∈ ℝ^{n_i}$ is the vector of residuals for the i-th block component cost function and $f_{i,j}(p)$ its j-th component function.

Keyword arguments

• value_cache=nothing: if provided, this vector is used to store the residuals $F(p)$ internally to avoid re-computations.
• jacobian_cache=fill(nothing, length(nlso.objective)): if provided, this is used to store the Jacobians of the component functions.

get_gradient(M::AbstractManifold, mgo::ManifoldProximalGradientObjective, p)
get_gradient!(M::AbstractManifold, X, mgo::ManifoldProximalGradientObjective, p)

Evaluate the gradient of the smooth part of a ManifoldProximalGradientObjective mgo at p.

get_gradient(M::AbstractManifold, sgo::ManifoldStochasticGradientObjective, p, k)
get_gradient!(M::AbstractManifold, X, sgo::ManifoldStochasticGradientObjective, p, k)

Evaluate one of the summands gradients $\operatorname{grad}f_k$, $k ∈ \{1,…,n\}$, at p (in place of X).

If you use a single function for the stochastic gradient, that works in-place, then get_gradient is not available, since the length (or number of elements of the gradient required for allocation) can not be determined.

get_gradient(M::AbstractManifold, scaled_objective::ScaledManifoldObjective, p)
get_gradient!(M::AbstractManifold, X, scaled_objective::ScaledManifoldObjective, p)

Evaluate the scaled gradient $s*\operatorname{grad}f(p)$.

get_gradient(M::AbstractManifold, sgo::ManifoldStochasticGradientObjective, p)
get_gradient!(M::AbstractManifold, X, sgo::ManifoldStochasticGradientObjective, p)

Evaluate the complete gradient $\operatorname{grad} f = \displaystyle\sum_{i=1}^{n} \operatorname{grad} f_i(p)$ at p (in place of X).

If you use a single function for the stochastic gradient, that works in-place, then get_gradient is not available, since the length (or number of elements of the gradient required for allocation) can not be determined.

get_gradient(M::AbstractManifold, emo::EmbeddedManifoldObjective, p)
get_gradient!(M::AbstractManifold, X, emo::EmbeddedManifoldObjective, p)

Evaluate the gradient function of an objective defined in the embedding, that is embed p before calling the gradient function stored in the EmbeddedManifoldObjective.

The returned gradient is then converted to a Riemannian gradient calling riemannian_gradient.

get_gradients(M::AbstractManifold, sgo::ManifoldStochasticGradientObjective, p)
get_gradients!(M::AbstractManifold, X, sgo::ManifoldStochasticGradientObjective, p)

Evaluate all summands gradients $\{\operatorname{grad}f_i\}_{i=1}^{n}$ at p and return them as a vector.

This can be done in-place of a vector of tangent vectors X.

get_hess_equality_constraint(amp::AbstractManoptProblem, p, j=:)
get_hess_equality_constraint(M::AbstractManifold, co::ConstrainedManifoldObjective, p, j, range=NestedPowerRepresentation())
get_hess_equality_constraint!(amp::AbstractManoptProblem, X, p, j=:)
get_hess_equality_constraint!(M::AbstractManifold, X, co::ConstrainedManifoldObjective, p, j, range=NestedPowerRepresentation())

Evaluate the Hessian or Hessians of the equality constraint $(\operatorname{Hess} h(p))_j$ or $\operatorname{Hess} h_j(p)$,

See also the ConstrainedManoptProblem to specify the range of the Hessian.

get_hess_inequality_constraint(amp::AbstractManoptProblem, p, X, j=:)
get_hess_inequality_constraint(M::AbstractManifold, co::ConstrainedManifoldObjective, p, j=:, range=NestedPowerRepresentation())
get_hess_inequality_constraint!(amp::AbstractManoptProblem, Y, p, j=:)
get_hess_inequality_constraint!(M::AbstractManifold, Y, co::ConstrainedManifoldObjective, p, X, j=:, range=NestedPowerRepresentation())

Evaluate the Hessian or Hessians of the inequality constraint $(\operatorname{Hess} g(p)[X])_j$ or $\operatorname{Hess} g_j(p)[X]$,

See also the ConstrainedManoptProblem to specify the range of the Hessian.

get_hessian(M::AbstractManifold, scaled_objective::ScaledManifoldObjective, p, X)
get_hessian!(M::AbstractManifold, Y, scaled_objective::ScaledManifoldObjective, p, X)

Evaluate the scaled Hessian $s*\operatorname{Hess}f(p)$.

get_hessian(M::AbstractManifold, emo::EmbeddedManifoldObjective, p, X)
get_hessian!(M::AbstractManifold, Y, emo::EmbeddedManifoldObjective, p, X)

Evaluate the Hessian of an objective defined in the embedding, that is embed p and X before calling the Hessian function stored in the EmbeddedManifoldObjective.

The returned Hessian is then converted to a Riemannian Hessian calling riemannian_Hessian.

get_inequality_constraint(amp::AbstractManoptProblem, p, j=:)
get_inequality_constraint(M::AbstractManifold, co::ConstrainedManifoldObjective, p, j=:, range=NestedPowerRepresentation())

Evaluate inequality constraints of a ConstrainedManifoldObjective objective at point p and indices j (by default : which corresponds to all indices).

get_inequality_constraint(M::AbstractManifold, emo::EmbeddedManifoldObjective, p, i)

Evaluate the ith inequality constraint $g_i(p)$ defined in the embedding, that is embed p before calling the constraint functions stored in the EmbeddedManifoldObjective.

get_preconditioner(M::AbstractManifold, mho::ManifoldHessianObjective, p, X)

Evaluate the preconditioner of the ManifoldHessianObjective mho at the point p, applied to the tangent vector X.

It usually is a symmetric, positive definite approximation of the inverse of the Hessian of the cost function f.

get_projected_point(amp::AbstractManoptProblem, p)
get_projected_point!(amp::AbstractManoptProblem, q, p)
get_projected_point(M::AbstractManifold, cso::ManifoldConstrainedSetObjective, p)
get_projected_point!(M::AbstractManifold, q, cso::ManifoldConstrainedSetObjective, p)

Project p with the projection that is stored within the ManifoldConstrainedSetObjective. This can be done in-place of q.

get_projected_point(amp::AbstractManoptProblem, p)
get_projected_point!(amp::AbstractManoptProblem, q, p)
get_projected_point(M::AbstractManifold, cso::ManifoldConstrainedSetObjective, p)
get_projected_point!(M::AbstractManifold, q, cso::ManifoldConstrainedSetObjective, p)

Project p with the projection that is stored within the ManifoldConstrainedSetObjective. This can be done in-place of q.

get_projected_point(amp::AbstractManoptProblem, p)
get_projected_point!(amp::AbstractManoptProblem, q, p)
get_projected_point(M::AbstractManifold, cso::ManifoldConstrainedSetObjective, p)
get_projected_point!(M::AbstractManifold, q, cso::ManifoldConstrainedSetObjective, p)

Project p with the projection that is stored within the ManifoldConstrainedSetObjective. This can be done in-place of q.

get_projected_point(amp::AbstractManoptProblem, p)
get_projected_point!(amp::AbstractManoptProblem, q, p)
get_projected_point(M::AbstractManifold, cso::ManifoldConstrainedSetObjective, p)
get_projected_point!(M::AbstractManifold, q, cso::ManifoldConstrainedSetObjective, p)

Project p with the projection that is stored within the ManifoldConstrainedSetObjective. This can be done in-place of q.

q = get_proximal_map(M::AbstractManifold, mpo::ManifoldProximalGradientObjective, λ, p)
get_proximal_map!(M::AbstractManifold, q, mpo::ManifoldProximalGradientObjective, λ, p)

Evaluate the proximal map of the nonsmooth component $h$ of the ManifoldProximalGradientObjective mpo at the point p on M with parameter $λ>0$.

q = get_proximal_map(M::AbstractManifold, mpo::ManifoldProximalMapObjective, λ, p)
get_proximal_map!(M::AbstractManifold, q, mpo::ManifoldProximalMapObjective, λ, p)
q = get_proximal_map(M::AbstractManifold, mpo::ManifoldProximalMapObjective, λ, p, i)
get_proximal_map!(M::AbstractManifold, q, mpo::ManifoldProximalMapObjective, λ, p, i)

Evaluate the (ith) proximal map of the ManifoldProximalMapObjective mpo at the point p of M with parameter $λ>0$.

get_residuals(M::AbstractManifold, nlso::ManifoldNonlinearLeastSquaresObjective, p)
get_residuals!(M::AbstractManifold, v, nlso::ManifoldNonlinearLeastSquaresObjective, p)

Compute the vector of residuals $F(p) ∈ ℝ^n$, $n = \sum_{i=1}^{m} n_i$. In other words this is the concatenation of the residual vectors $F_i(p)$, $i=1,…,m$ of the components of the ManifoldNonlinearLeastSquaresObjective nlso at the current point $p$ on M.

This can be computed in-place of v.

Note that even in the presence of RobustifierFunctions, these are not applied here, this function computes the “pure” residuals.

get_residuals(M::AbstractManifold, nlso::ManifoldNonlinearLeastSquaresObjective, p)
get_residuals!(M::AbstractManifold, v, nlso::ManifoldNonlinearLeastSquaresObjective, p)

Compute the vector of residuals $F(p) ∈ ℝ^n$, $n = \sum_{i=1}^{m} n_i$. In other words this is the concatenation of the residual vectors $F_i(p)$, $i=1,…,m$ of the components of the ManifoldNonlinearLeastSquaresObjective nlso at the current point $p$ on M.

This can be computed in-place of v.

Note that even in the presence of RobustifierFunctions, these are not applied here, this function computes the “pure” residuals.

X = get_subgradient(M::AbstractManifold, sgo::ManifoldSubgradientObjective, p)
get_subgradient!(M::AbstractManifold, X, sgo::ManifoldSubgradientObjective, p)

Evaluate the (sub)gradient of a ManifoldSubgradientObjective sgo at the point p.

The evaluation is done in place of X for the !-variant. The result might not be deterministic, one element of the subdifferential is returned.

get_unconstrained_objective(co::ConstrainedManifoldObjective)

Returns the internally stored unconstrained AbstractManifoldObjective within the ConstrainedManifoldObjective.

inequality_constraints_length(cmo::ConstrainedManifoldObjective)

Return the number of inequality constraints of an ConstrainedManifoldObjective cmo. This acts transparently through AbstractDecoratedManifoldObjectives

is_feasible(M::AbstractManifold, cmo::ConstrainedManifoldObjective, p; kwargs...)
is_feasible(M::AbstractManifold, o::AbstractDecoratedManifoldObjective, p; kwargs...)

Evaluate whether a point p on M is feasible with respect to the ConstrainedManifoldObjective cmo. That is for the provided inequality constraints $g: \mathcal{M} → ℝ^m$ and equality constraints $h: \mathcal{M} → ℝ^m$ from within cmo, the point $p ∈ \mathcal{M}$ is feasible if

\[g_i(p) ≤ 0, \text{ for all } i=1,…,m\quad ext{ and }\quad h_j(p) = 0, ext{ for all } j=1,…,n.\]

Keyword arguments

• check_point::Bool=true: whether to also verify that $p∈\mathcal{M}$ holds, using is_point
• error::Symbol=:none: if the point is not feasible, this symbol determines how to report the error.
  • :error: throws an error
  • :info: displays the error message as an @info
  • :none: (default) the function just returns true/false
  • :warn: displays the error message as a @warning.

The keyword error= and all other kwargs... are passed on to is_point if the point is verified (see check_point).

objective_cache_factory(M::AbstractManifold, o::AbstractManifoldObjective, cache::Symbol)

Generate a cached variant of the AbstractManifoldObjective o on the AbstractManifold M based on the symbol cache.

The following caches are available

• :Simple generates a SimpleManifoldCachedObjective
• :LRU generates a ManifoldCachedObjective where you should use the form (:LRU, [:Cost, :Gradient]) to specify what should be cached or (:LRU, [:Cost, :Gradient], 100) to specify the cache size. Here this variant defaults to (:LRU, [:Cost, :Gradient], 100), caching up to 100 cost and gradient values.^[1]

objective_cache_factory(M::AbstractManifold, o::AbstractManifoldObjective, cache::Tuple{Symbol, Array, Array})
objective_cache_factory(M::AbstractManifold, o::AbstractManifoldObjective, cache::Tuple{Symbol, Array})

Generate a cached variant of the AbstractManifoldObjective o on the AbstractManifold M, selected by cache[1].

The second element cache[2] provides further arguments to the cache and the optional third is passed down as keyword arguments.

For all available caches see the simpler variant with symbols.

reset_counters!(co::ManifoldCountObjective, value::Integer=0)

Reset all values in the count objective to value.

get_cost_and_differential(
    M, mfo::ManifoldFirstOrderObjective, p, X;
    gradient = missing, evaluated::Bool = false,
)

Evaluate cost function and differential simultaneously.

This combined evaluation is especially beneficial when you provided a combined costdifferential function within the ManifoldFirstOrderObjective mfo. When there is no separate differential, the evaluation falls back to evaluating the gradient and an inner product.

Input

• M::AbstractManifold: a Riemannian manifold $\mathcal{M}$
• mfo an ManifoldFirstOrderObjective
• p::P: a point on the manifold $\mathcal{M}$
• X::T: a tangent vector at the point $p$ on the manifold $\mathcal{M}$

Keyword Arguments

• gradient = missing provide a cache/memory to evaluate the gradient in. if not provided, this function will allocate one tangent vector
• evaluated = false if the cache is provided, this boolean specifies, whether the cache already contains the evaluated gradient and does not need to evaluate the gradient again, if possible. For example for a combined costgrad function, the evaluation can not be avoided, since we need the cost value.

get_feasibility_status(
    M::AbstractManifold,
    cmo::ConstrainedManifoldObjective,
    g = get_inequality_constraints(M, cmo, p),
    h = get_equality_constraints(M, cmo, p),
)

Generate a message about the feasibiliy of p with respect to the ConstrainedManifoldObjective. You can also provide the evaluated vectors for the values of g and h as keyword arguments, in case you had them evaluated before.

get_subgradient_function(objective::ManifoldSubgradientObjective, recursive=false; evaluation = AllocatingEvaluation())

Return the function to evaluate (just) the subgradient $∂ f(p)$. It is of the form (M, X, p) -> X to work in-place of X, where either the subgradient function using the decorator or without the decorator is used.

By default recursive is set to false, since usually to just pass the gradient function somewhere, one still wants for example the cached one or the one that still counts calls.

init_caches(M::AbstractManifold, caches, T; kwargs...)

Given a vector of symbols caches, this function sets up the NamedTuple of caches for points/vectors on M, where T is the type of cache to use.

residuals_count(nlso::ManifoldNonlinearLeastSquaresObjective)

Return the total number of residuals in ManifoldNonlinearLeastSquaresObjective nlso, which is the sum of the single block components lengths.

SymmetricLinearSystemObjective{TA,T} <: AbstractSymmetricLinearSystemObjective

Model the objective

\[f(X) = \frac{1}{2} \lVert \mathcal{A}[X] + b \rVert_{p}^2,\qquad X ∈ T_{p}\mathcal{M},\]

defined on the tangent space $T_{p}\mathcal{M}$ at $p$ on the manifold $\mathcal{M}$.

In other words this is an objective to solve $\mathcal{A}[X] = -b(p)$ for some linear symmetric operator $\mathcal{A}$ and a vector function $b$. Note the minus on the right hand side, which makes this objective especially tailored for (iteratively) solving Newton-like equations.

Fields

• A!: a symmetric, linear operator on the tangent space, see get_linear_operator
• b!: a tangent vector function, see get_vector_field

where A! is implemented as in-place operator (M, Y, p, X) -> Y, and similarly b! is a function (M, X, p) -> X implemented to work in-place of X.

Constructor

SymmetricLinearSystemObjective(A, b; evaluation=AllocatingEvaluation(), p = missing)

Generate the objective specifying whether the two parts work allocating or in-place.

Keyword Arguments

• evaluation::AbstractEvaluationType=AllocatingEvaluation(): specify whether the functions that return an array, for example a point or a tangent vector, work by allocating its result (AllocatingEvaluation) or whether they modify their input argument to return the result therein (InplaceEvaluation). Since usually the first argument is the manifold, the modified argument is the second.
• p = missing provide a point to automatically ensure the functions of the objective “act” on mutating variables.

Y = get_linear_operator(M::AbstractManifold, slso::SymmetricLinearSystemObjective, p, X)
get_linear_operator!(M::AbstractManifold, Y, slso::SymmetricLinearSystemObjective, p, X)

Evaluate the linear operator $W = \mathcal{A}[X]$ from the SymmetricLinearSystemObjective defined on the tangent space at p at the tangent vector X.

This can be evaluated in-place of Y.

get_vector_field(M::AbstractManifold, slso::SymmetricLinearSystemObjective, p)
get_vector_field!(M::AbstractManifold, Y, slso::SymmetricLinearSystemObjective, p)
get_vector_field(TpM::TangentSpace, slso::SymmetricLinearSystemObjective)
get_vector_field!(TpM::TangentSpace, Y, slso::SymmetricLinearSystemObjective)

Evaluate the stored value for computing the right hand side $b$ in $\mathcal{A}[X] = -b$, either providing a tangent space or a manifold and a point.

This can be evaluated in-place of Y.

get_vector_field(M::AbstractManifold, slso::SymmetricLinearSystemObjective, p)
get_vector_field!(M::AbstractManifold, Y, slso::SymmetricLinearSystemObjective, p)
get_vector_field(TpM::TangentSpace, slso::SymmetricLinearSystemObjective)
get_vector_field!(TpM::TangentSpace, Y, slso::SymmetricLinearSystemObjective)

Evaluate the stored value for computing the right hand side $b$ in $\mathcal{A}[X] = -b$, either providing a tangent space or a manifold and a point.

This can be evaluated in-place of Y.

get_vector_field(M::AbstractManifold, slso::SymmetricLinearSystemObjective, p)
get_vector_field!(M::AbstractManifold, Y, slso::SymmetricLinearSystemObjective, p)
get_vector_field(TpM::TangentSpace, slso::SymmetricLinearSystemObjective)
get_vector_field!(TpM::TangentSpace, Y, slso::SymmetricLinearSystemObjective)

Evaluate the stored value for computing the right hand side $b$ in $\mathcal{A}[X] = -b$, either providing a tangent space or a manifold and a point.

This can be evaluated in-place of Y.

AugmentedLagrangianCost{CO,R,T} <: AbstractConstrainedFunction{CO}

Stores the parameters $ρ ∈ ℝ$, $μ ∈ ℝ^m$, $λ ∈ ℝ^n$ of the augmented Lagrangian associated to the ConstrainedManifoldObjective co.

This struct is also a functor (M,p) -> v that can be used as a cost function within a solver, based on the internal ConstrainedManifoldObjective it computes

\[\mathcal{L}_ρ(p, μ, λ) = f(p) + \frac{ρ}{2}\biggl( \sum_{j=1}^{n}\Bigl( h_j(p) + \frac{λ_j}{ρ} \Bigr)^2 + \sum_{i=1}^{m}\max\{0, \frac{μ_i}{ρ} + g_i(p)\}^2 \biggr)\]

Fields

• co::CO, ρ::R, μ::T, λ::T as mentioned in the formula, where R should be the number type used and T the vector type.

Constructor

AugmentedLagrangianCost(co, ρ, μ, λ)

AugmentedLagrangianGrad{CO,R,T} <: AbstractConstrainedFunction{T}

Stores the parameters $ρ ∈ ℝ$, $μ ∈ ℝ^m$, $λ ∈ ℝ^n$ of the augmented Lagrangian associated to the ConstrainedManifoldObjective co.

This struct is also a functor in both formats

• (M, p) -> X to compute the gradient in allocating fashion.
• (M, X, p) to compute the gradient in an in-place fashion.

Additionally this gradient accepts a positional last argument to specify the range for the internal gradient call of the constrained objective.

Based on the internal ConstrainedManifoldObjective, it computes the gradient $\operatorname{grad}\mathcal{L}_{ρ}(p, μ, λ)$, see also AugmentedLagrangianCost.

Fields

• co::CO, ρ::R, μ::T, λ::T as mentioned in the formula, where R should be the number type used and T the vector type.

Constructor

AugmentedLagrangianGrad(co, ρ, μ, λ)

LevenbergMarquardtLinearSurrogateObjective{R<:Real,TO,TVC} <: AbstractLevenbergMarquardtLinearSurrogateObjective

The linear surrogate objective for a ManifoldNonlinearLeastSquaresObjective.

Given an ManifoldNonlinearLeastSquaresObjective objective and a penalty $λ$, this objective represents the penalized objective for the sub-problem to solve within every step of the Levenberg-Marquardt algorithm following the ideas of [TMHF00] given by

\[μ_p(X) = \frac{1}{2}\lVert \mathcal{L}(X) + y \rVert_{2}^2 + \frac{λ}{2}\lVert X \rVert_{p}^2, \qquad\text{ for }X ∈ T_{p}\mathcal{M}, λ ≥ 0,\]

where $X ∈ T_{p}\mathcal{M}$, $λ ≥ 0$ is the damping or penalty term, $\mathcal{L}: T_{p}\mathcal{M} → ℝ^n$ is a linear operator, and $y = y(p) ∈ ℝ^n$ is a vector field. For the derivation of the Riemannian case, see [BB26].

In order to build a surrogate also for the robustified Levenberg-Marquardt, introduce $α = 1 - \sqrt{1 + 2 \frac{ρ''(p)}{ρ'(p)}\lVert F(p) \rVert_{2}^2}$ and set $y = \frac{\sqrt{ρ'(p)}}{1-α}F(p)$ and $\mathcal{L}(X) = CJ_F(p)[X]$ with

\[C = \sqrt{ρ'(p)}(I-αP), \qquad P = \frac{F(p)F(p)^\mathrm{T}}{\lVert F(p) \rVert_{2}^2},\]

where $F(p) ∈ ℝ^n$ is the vector of residuals at point $p ∈ M$ and $J_F(p): T_{p}\mathcal{M} → ℝ^n$ is the Jacobian. These two can be accessed with get_vector_field for $y$ and get_linear_operator for $\mathcal{L}$, respectively. For technical details on the scaling using $α$, especially how the threshold and mode act as safeguards, see get_LevenbergMarquardt_scaling.

Fields

• objective: the ManifoldNonlinearLeastSquaresObjective to penalize
• penalty::Real: the damping term $λ$
• threshold::Real: threshold $ε$ for stabilization of $α$ as $α ≤ 1-ε$, see get_LevenbergMarquardt_scaling
• mode::Symbol: which mode to use to stabilize $α$, see the internal helper get_LevenbergMarquardt_scaling
• value_cache: a vector to store the residuals $F(p)$ at the current point p internally to avoid re-computations

Constructor

LevenbergMarquardtLinearSurrogateObjective(objective;    penalty::Real = 1e-6,    threshold::Real = 1e-4,    mode::Symbol = :Strict,    residuals = zeros(residuals_count(get_objective(objective))),)

Note that the keyword residuals initializes the value_cache field.

TrustRegionModelObjective{O<:AbstractManifoldHessianObjective} <: AbstractManifoldSubObjective{O}

A trust region model of the form

\[ m(X) = f(p) + ⟨\operatorname{grad} f(p), X⟩_p + \frac{1}{2} ⟨\operatorname{Hess} f(p)[X], X⟩_p\]

Fields

• objective: an AbstractManifoldHessianObjective proving $f$, its gradient and Hessian

Constructors

TrustRegionModelObjective(objective)

with either an AbstractManifoldHessianObjective objective or an decorator containing such an objective

get_cost(
    M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, X
)

Compute the surrogate cost. Let $F$ denote the vector of residuals (of a block), $ρ, ρ'$, $ρ''$ the value, first, and second derivative of the AbstractRobustifierFunction of the inner ManifoldNonlinearLeastSquaresObjective

\[σ_k(X) = \frac{1}{2}\lVert y + \mathcal{L}(X) \rVert_{2}^2, \qquad X ∈ T_{p}\mathcal{M}\]

where

• $\mathcal{L}(X) = CJ[X]$ see get_linear_operator with a penalty of zero.
• $y$ the rescaled vector field, see get_vector_field

get_cost(TpM::TangentSpace, neo::NormalEquationsObjective{<:LevenbergMarquardtLinearSurrogateObjective}, X)

Compute the surrogate cost when solving its normal equations.

See also get_cost(::AbstractManifold, ::LevenbergMarquardtLinearSurrogateObjective, p, X), get_linear_operator, and get_vector_field for more details.

get_cost(TpM, trmo::TrustRegionModelObjective, X)

Evaluate the tangent space TrustRegionModelObjective

\[m(X) = f(p) + ⟨\operatorname{grad} f(p), X ⟩_p + \frac{1}{2} ⟨\operatorname{Hess} f(p)[X], X⟩_p.\]

get_gradient(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, X)
get_gradient!(M::AbstractManifold, Y, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, X)

Compute the gradient of the LevenbergMarquardtLinearSurrogateObjective, which is given by

\[\begin{aligned} \operatorname{grad} μ_p(X) &= \sum_{i=1}^{m} \mathcal{L}_i^*\bigl(\mathcal{L}_i(X) + y_i \bigr) + λX\\\\ &= \sum_{i=1}^{m} J_{F_i}^*(p)\Bigl[ ρ_i' \bigl(I- b F_i(p)F_i(p)^{\mathrm{T}}\bigr)^2 J_{F_i}(p)[X] + a\sqrt{ρ_i'}\bigl(I- b F_i(p)F_i(p)^{\mathrm{T}}\bigr) F_i(p) + λX \Bigr] \end{aligned} \]

where $ρ_i' = ρ_i'(\lVert F_i(p) \rVert_2^2)$, $ρ_i'' = ρ_i''(\lVert F_i(p) \rVert_2^2)$ are the values from the AbstractRobustifierFunction ρ its first and second derivative, respectively, and $a,b$ are the get_LevenbergMarquardt_scaling values of scaling the residual and operator, respectively. See also get_jacobian and get_adjoint_jacobian.

This can be computed inplace of Y.

get_gradient(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, X)
get_gradient!(M::AbstractManifold, Y, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, X)

Compute the gradient of the LevenbergMarquardtLinearSurrogateObjective, which is given by

\[\begin{aligned} \operatorname{grad} μ_p(X) &= \sum_{i=1}^{m} \mathcal{L}_i^*\bigl(\mathcal{L}_i(X) + y_i \bigr) + λX\\\\ &= \sum_{i=1}^{m} J_{F_i}^*(p)\Bigl[ ρ_i' \bigl(I- b F_i(p)F_i(p)^{\mathrm{T}}\bigr)^2 J_{F_i}(p)[X] + a\sqrt{ρ_i'}\bigl(I- b F_i(p)F_i(p)^{\mathrm{T}}\bigr) F_i(p) + λX \Bigr] \end{aligned} \]

where $ρ_i' = ρ_i'(\lVert F_i(p) \rVert_2^2)$, $ρ_i'' = ρ_i''(\lVert F_i(p) \rVert_2^2)$ are the values from the AbstractRobustifierFunction ρ its first and second derivative, respectively, and $a,b$ are the get_LevenbergMarquardt_scaling values of scaling the residual and operator, respectively. See also get_jacobian and get_adjoint_jacobian.

This can be computed inplace of Y.

get_gradient(TpM, trmo::TrustRegionModelObjective, X)

Evaluate the gradient of the TrustRegionModelObjective

\[\operatorname{grad} m(X) = \operatorname{grad} f(p) + \operatorname{Hess} f(p)[X].\]

get_hessian(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, X, Y)
get_hessian!(M::AbstractManifold, Z, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, X, Y)

Compute the Hessian of the LevenbergMarquardtLinearSurrogateObjective, which is given by

\[\begin{aligned} \operatorname{Hess} μ_p(X)[Y] &= \sum_{i=1}^{m} \mathcal{L}_i^*\bigl(\mathcal{L}_i(Y)\bigr) + λY\\\\ &= \sum_{i=1}^{m} J_{F_i}^*(p)\Bigl[ ρ_i' \bigl(I- b F_i(p)F_i(p)^{\mathrm{T}}\bigr)^2 J_{F_i}(p)[Y] + λY \Bigr] \end{aligned} \]

where $ρ_i' = ρ_i'(\lVert F_i(p) \rVert_2^2)$, $ρ_i'' = ρ_i''(\lVert F_i(p) \rVert_2^2)$ are the values from the AbstractRobustifierFunction ρ its first and second derivative, respectively, and $b$ is the get_LevenbergMarquardt_scaling values of scaling the operator. See also get_jacobian and get_adjoint_jacobian.

This can be computed inplace of Z.

get_hessian(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, X, Y)
get_hessian!(M::AbstractManifold, Z, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, X, Y)

Compute the Hessian of the LevenbergMarquardtLinearSurrogateObjective, which is given by

\[\begin{aligned} \operatorname{Hess} μ_p(X)[Y] &= \sum_{i=1}^{m} \mathcal{L}_i^*\bigl(\mathcal{L}_i(Y)\bigr) + λY\\\\ &= \sum_{i=1}^{m} J_{F_i}^*(p)\Bigl[ ρ_i' \bigl(I- b F_i(p)F_i(p)^{\mathrm{T}}\bigr)^2 J_{F_i}(p)[Y] + λY \Bigr] \end{aligned} \]

where $ρ_i' = ρ_i'(\lVert F_i(p) \rVert_2^2)$, $ρ_i'' = ρ_i''(\lVert F_i(p) \rVert_2^2)$ are the values from the AbstractRobustifierFunction ρ its first and second derivative, respectively, and $b$ is the get_LevenbergMarquardt_scaling values of scaling the operator. See also get_jacobian and get_adjoint_jacobian.

This can be computed inplace of Z.

get_hessian(TpM, trmo::TrustRegionModelObjective, X)

Evaluate the Hessian of the TrustRegionModelObjective

\[\operatorname{Hess} m(X)[Y] = \operatorname{Hess} f(p)[Y].\]

AbstractLevenbergMarquardtLinearSurrogateObjective

Abstract supertype for Levenberg-Marquardt surrogates like LevenbergMarquardtLinearSurrogateObjective and LevenbergMarquardtLinearSurrogateCoordinatesObjective.

LevenbergMarquardtLinearSurrogateCoordinatesObjective{R<:Real,TO,TVC,TJC,TB} <: AbstractLevenbergMarquardtLinearSurrogateObjective

A subobjective similar to LevenbergMarquardtLinearSurrogateObjective but which uses coordinate-based Jacobians in a single, selected basis instead of being centered around linear operators.

Fields

• objective: the ManifoldNonlinearLeastSquaresObjective to penalize
• penalty::Real: the damping term $λ$
• threshold::Real: stabilization $ε$ for $α ≤ 1-ε$ in the rescaling of the residual and Jacobian, see get_LevenbergMarquardt_scaling
• mode::Symbol: which mode to use to stabilize α, see the internal helper get_LevenbergMarquardt_scaling
• value_cache: a vector to store the residuals $F(p)$ at the current point p internally to avoid recomputations
• jacobian_cache: a vector to store the coordinate-based Jacobian of the residuals at the current point p internally to avoid recomputations. If the Jacobian is used as a linear operator, this is just a vector of nothings.
• basis: the AbstractBasis the coordinates refer to

Constructor

LevenbergMarquardtLinearSurrogateCoordinatesObjective(objective;    penalty::Real = 1e-6,    threshold::Real = 1e-4,    mode::Symbol = :Strict,    residuals = zeros(residuals_count(get_objective(objective))),    jacobian_cache = fill(nothing, length(get_objective(objective).objective)),    basis = DefaultOrthonormalBasis(),)

Note that the keyword residuals initializes the value_cache field.

NormalEquationsObjective{O <: AbstractLinearSurrogateObjective} <: AbstractSymmetricLinearSystemObjective

A AbstractLinearSurrogateObjective might be overdetermined, and it usually is overdetermined, e.g. for the case of the LevenbergMarquardt algorithm. For this case, one considers the normal equations.

This wrapper provides the same three functions as the wrapped surrogate

• get_linear_operator to compute/evaluate the linear operator $\mathcal{L}$
• get_vector_field to compute/evaluate the vector $y$
• get_objective to provide access to the underlying surrogate

so that we obtain a symmetric linear system of equations, that can be

• solved with an iterative method like conjugate_gradient_descent or conjugate_residual
• solved as a linear system in a basis of the corresponding tangent space.

add_linear_operator_coord!(
    M::AbstractManifold, y::AbstractVector, lmsco::LevenbergMarquardtLinearSurrogateCoordinatesObjective, p, cX::AbstractVector
)

Add the (Triggs correction, residual-like) linear operator corresponding to the lmsco surrogate to vector y. It is assumed that lmsco.value_cache has been filled in step_solver! of LevenbergMarquardt, so we can just use it here.

add_normal_linear_operator!(
    M::AbstractManifold, A::AbstractMatrix, o::AbstractFirstOrderVectorFunction,
    r::AbstractRobustifierFunction, p, basis::AbstractBasis;
    value_cache = get_value(M, o, p), ε::Real, mode::Symbol
)

Add the contribution of a single block (vectorial function with its robustifier) to the linear normal operator, i.e. compute $A += J_F^*(p)[C^T C J_F(p)[X]]$ in-place of A for the given block. See get_normal_linear_operator for details

add_normal_vector_field!(M::AbstractManifold, X, o::AbstractFirstOrderVectorFunction, r::AbstractRobustifierFunction, p)
add_normal_vector_field!(M::AbstractManifold, c, o::AbstractFirstOrderVectorFunction, r::AbstractRobustifierFunction, p, B::AbstractBasis)

Add the contribution of o / r to the normal linear operator tangent vector in X or c. See get_normal_vector_field for the mathematical details. Note that this is done per every block (vectorial function with its robustifier) of the underlying ManifoldNonlinearLeastSquaresObjective and summed up.

See also get_normal_linear_operator for evaluating the corresponding linear operator of the (normal) linear system, and get_LevenbergMarquardt_scaling for details on the scaling and computation of $C$.

residual_scaling, operator_scaling = get_LevenbergMarquardt_scaling(ρ_prime::Real, ρ_double_prime::Real, FkSq::Real, threshold::Real=1.0e-5, mode::Symbol=:Strict)

Compute the scalings for the residual $y$ and within the operator $C$ that are required for the robust rescaling within LevenbergMarquardts get_vector_field and get_linear_operator, respectively. Here FkSq denotes $s = \lVert F(p) \rVert_{2}^2$ of the residual vector function $F$ evaluated at some point $p$, and ρ_prime $= ρ'(s)$ and ρ_double_prime $= ρ''(s)$ denote the current AbstractRobustifierFunction's first and second derivative evaluated at $s$.

The value for $α$ is given by

\[ α = 1 - \sqrt{1 + 2\frac{ρ''(s)}{ρ'(s)}s}\]

and hence the scaling of the residual and the one within the projection of the operator are

\[\frac{\sqrt{ρ'(s)}}{1-α} \qquad\text{ and }\qquad \begin{cases} \frac{α}{s} & \text{ if } s ≠ 0\\\\ 0 & \text{ else,}\end{cases}\]

respectively.

Numerical stability

For a unique solution that is a minimizer in a Levenberg-Marquardt step, we require α < 1 and [TMHF00] recommends to bound this even by $1-ε$ for some threshold $ε > 0$.

Furthermore if $ρ'(s) + 2ρ''(s)⋅s ≤ 0$ the Hessian is also indefinite. This can be caught by making sure the argument of the $√$ is ensured to be non-negative.

The Ceres solver even omits the second term in the square root already if $ρ(s)'' < 0$ for stability reason, which means setting $α = 0$. In the case $s = 0$ we also set the operator scaling $α / s = 0$.

This function offers two modes

• :Normal keeps negative $ρ''(s) < 0$ but makes sure the square root is well-defined.
• :Strict (default) sets $α = 0$ when $ρ''(s) < 0$ or when $s = 0$.

get_linear_operator(M::AbstractManifold, neo::NormalEquationsObjective, p, X)
get_linear_operator(M::AbstractManifold, neo::NormalEquationsObjective, p, c, B)
get_linear_operator(M::AbstractManifold, neo::NormalEquationsObjective, p, B)
get_linear_operator!(M::AbstractManifold, Y, neo::NormalEquationsObjective, p, X)
get_linear_operator!(M::AbstractManifold, b, neo::NormalEquationsObjective, p, c, B)
get_linear_operator!(M::AbstractManifold, A, neo::NormalEquationsObjective, p, B)

Evaluate the linear operator related to the normal equations of the LevenbergMarquardtLinearSurrogateObjective. See get_normal_linear_operator for details.

get_linear_operator(M::AbstractManifold, neo::NormalEquationsObjective, p, X)
get_linear_operator(M::AbstractManifold, neo::NormalEquationsObjective, p, c, B)
get_linear_operator(M::AbstractManifold, neo::NormalEquationsObjective, p, B)
get_linear_operator!(M::AbstractManifold, Y, neo::NormalEquationsObjective, p, X)
get_linear_operator!(M::AbstractManifold, b, neo::NormalEquationsObjective, p, c, B)
get_linear_operator!(M::AbstractManifold, A, neo::NormalEquationsObjective, p, B)

Evaluate the linear operator related to the normal equations of the LevenbergMarquardtLinearSurrogateObjective. See get_normal_linear_operator for details.

get_linear_operator(M::AbstractManifold, neo::NormalEquationsObjective, p, X)
get_linear_operator(M::AbstractManifold, neo::NormalEquationsObjective, p, c, B)
get_linear_operator(M::AbstractManifold, neo::NormalEquationsObjective, p, B)
get_linear_operator!(M::AbstractManifold, Y, neo::NormalEquationsObjective, p, X)
get_linear_operator!(M::AbstractManifold, b, neo::NormalEquationsObjective, p, c, B)
get_linear_operator!(M::AbstractManifold, A, neo::NormalEquationsObjective, p, B)

Evaluate the linear operator related to the normal equations of the LevenbergMarquardtLinearSurrogateObjective. See get_normal_linear_operator for details.

get_linear_operator(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, X)
get_linear_operator!(M::AbstractManifold, y, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, X)

Evaluate the linear operator $\mathcal{L}$ corresponding to the Levenberg-Marquardt surrogate objective, i.e.,

\[\mathcal{L}(X) = C J_F(p)[X] \bigr],\]

with

\[C = \sqrt{ρ'(p)}(I-αP), \qquad P = \frac{F(p)F(p)^\mathrm{T}}{\lVert F(p) \rVert_{2}^2},\]

where $α = 1 - \sqrt{1 + 2 \frac{ρ''(p)}{ρ'(p)}\lVert F(p) \rVert_{2}^2}$.

Note that this is done per every block (vectorial function with its robustifier) of the underlying ManifoldNonlinearLeastSquaresObjective and summed up.

This can be computed in-place of y.

See also get_vector_field for evaluating the corresponding vector field

get_linear_operator(M::AbstractManifold, neo::NormalEquationsObjective, p, X)
get_linear_operator(M::AbstractManifold, neo::NormalEquationsObjective, p, c, B)
get_linear_operator(M::AbstractManifold, neo::NormalEquationsObjective, p, B)
get_linear_operator!(M::AbstractManifold, Y, neo::NormalEquationsObjective, p, X)
get_linear_operator!(M::AbstractManifold, b, neo::NormalEquationsObjective, p, c, B)
get_linear_operator!(M::AbstractManifold, A, neo::NormalEquationsObjective, p, B)

Evaluate the linear operator related to the normal equations of the LevenbergMarquardtLinearSurrogateObjective. See get_normal_linear_operator for details.

get_linear_operator(M::AbstractManifold, neo::NormalEquationsObjective, p, X)
get_linear_operator(M::AbstractManifold, neo::NormalEquationsObjective, p, c, B)
get_linear_operator(M::AbstractManifold, neo::NormalEquationsObjective, p, B)
get_linear_operator!(M::AbstractManifold, Y, neo::NormalEquationsObjective, p, X)
get_linear_operator!(M::AbstractManifold, b, neo::NormalEquationsObjective, p, c, B)
get_linear_operator!(M::AbstractManifold, A, neo::NormalEquationsObjective, p, B)

Evaluate the linear operator related to the normal equations of the LevenbergMarquardtLinearSurrogateObjective. See get_normal_linear_operator for details.

get_normal_linear_operator(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, X; threshold = lmsco.threshold, mode = lmsco.mode, penalty = lmsco.penalty)
get_normal_linear_operator!(M::AbstractManifold, Y, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, X; threshold = lmsco.threshold, mode = lmsco.mode, penalty = lmsco.penalty)
get_normal_linear_operator(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p[, c], B::AbstractBasis; threshold = lmsco.threshold, mode = lmsco.mode, penalty = lmsco.penalty)
get_normal_linear_operator!(M::AbstractManifold, [A | b], lmsco::LevenbergMarquardtLinearSurrogateObjective, p[, c], B::AbstractBasis; threshold = lmsco.threshold, mode = lmsco.mode, penalty = lmsco.penalty)

Compute the linear operator $\mathcal{A}$ corresponding to the optimality conditions of the modified Levenberg-Marquardt surrogate objective, i.e. the normal conditions

\[\mathcal{A}(X) = \mathcal{L}^* \mathcal{L}(X) + λX = J_F^*(p)\bigl[ C^T C J_F(p)[X] \bigr] + λX,\]

where $λ =$penalty is a damping parameter and with $α = 1 - \sqrt{1 + 2 \frac{ρ''(p)}{ρ'(p)}\lVert F(p) \rVert_{2}^2}$ we have

\[C = \sqrt{ρ'(p)}(I-αP), \qquad P = \frac{F(p)F(p)^\mathrm{T}}{\lVert F(p) \rVert_{2}^2},\]

See get_LevenbergMarquardt_scaling for details on the scaling and α. and get_jacobian and get_adjoint_jacobian concerning $J_F$ and $J_F^*$, respectively.

There are three variants to use this function to use the corresponding linear operator

• if you provide a tangent vector X, then the linear operator is evaluated at X, the corresponding gradient Y is returned
• if you provide X in coordinates c with respect to a basis B the linear operator is evaluated and the coordinates b of the result are returned
• if you provide (just) a basis B of the tangent space, then the matrix A of the linear operator represented in this basis is returned. The relation to the second case is that $b = Ac$.

See also get_normal_vector_field for evaluating the corresponding vector field.

get_normal_vector_field(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p)
get_normal_vector_field!(M::AbstractManifold, X, lmsco::LevenbergMarquardtLinearSurrogateObjective, p)
get_normal_vector_field(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, B::AbstractBasis)
get_normal_vector_field!(M::AbstractManifold, c, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, B::AbstractBasis)

Compute the normal linear operator tangent vector $X$ corresponding to the normal equations (optimality conditions) of the Levenberg-Marquardt surrogate objective, i.e.,

\[X = J_F^*(p)[ C^T y], \quad y = \frac{\sqrt{ρ'(p)}}{1-α}F(p).\]

If you provide an AbstractBasis B $=\{Z_1,…,Z_d\}$ additionally, the result will be given in coordinates c, i.e. such that $X = \sum_{i=1}^{d} c_iZ_i$.

Note that this is done per every block (vectorial function with its robustifier) of the underlying ManifoldNonlinearLeastSquaresObjective and summed up. See also get_normal_linear_operator for evaluating the corresponding linear operator of the (normal) linear system, and get_LevenbergMarquardt_scaling for details on the scaling and computation of $C$.

get_normal_vector_field(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p)
get_normal_vector_field!(M::AbstractManifold, X, lmsco::LevenbergMarquardtLinearSurrogateObjective, p)
get_normal_vector_field(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, B::AbstractBasis)
get_normal_vector_field!(M::AbstractManifold, c, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, B::AbstractBasis)

Compute the normal linear operator tangent vector $X$ corresponding to the normal equations (optimality conditions) of the Levenberg-Marquardt surrogate objective, i.e.,

\[X = J_F^*(p)[ C^T y], \quad y = \frac{\sqrt{ρ'(p)}}{1-α}F(p).\]

If you provide an AbstractBasis B $=\{Z_1,…,Z_d\}$ additionally, the result will be given in coordinates c, i.e. such that $X = \sum_{i=1}^{d} c_iZ_i$.

Note that this is done per every block (vectorial function with its robustifier) of the underlying ManifoldNonlinearLeastSquaresObjective and summed up. See also get_normal_linear_operator for evaluating the corresponding linear operator of the (normal) linear system, and get_LevenbergMarquardt_scaling for details on the scaling and computation of $C$.

get_normal_vector_field(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p)
get_normal_vector_field!(M::AbstractManifold, X, lmsco::LevenbergMarquardtLinearSurrogateObjective, p)
get_normal_vector_field(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, B::AbstractBasis)
get_normal_vector_field!(M::AbstractManifold, c, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, B::AbstractBasis)

Compute the normal linear operator tangent vector $X$ corresponding to the normal equations (optimality conditions) of the Levenberg-Marquardt surrogate objective, i.e.,

\[X = J_F^*(p)[ C^T y], \quad y = \frac{\sqrt{ρ'(p)}}{1-α}F(p).\]

If you provide an AbstractBasis B $=\{Z_1,…,Z_d\}$ additionally, the result will be given in coordinates c, i.e. such that $X = \sum_{i=1}^{d} c_iZ_i$.

Note that this is done per every block (vectorial function with its robustifier) of the underlying ManifoldNonlinearLeastSquaresObjective and summed up. See also get_normal_linear_operator for evaluating the corresponding linear operator of the (normal) linear system, and get_LevenbergMarquardt_scaling for details on the scaling and computation of $C$.

get_normal_vector_field(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p)
get_normal_vector_field!(M::AbstractManifold, X, lmsco::LevenbergMarquardtLinearSurrogateObjective, p)
get_normal_vector_field(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, B::AbstractBasis)
get_normal_vector_field!(M::AbstractManifold, c, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, B::AbstractBasis)

Compute the normal linear operator tangent vector $X$ corresponding to the normal equations (optimality conditions) of the Levenberg-Marquardt surrogate objective, i.e.,

\[X = J_F^*(p)[ C^T y], \quad y = \frac{\sqrt{ρ'(p)}}{1-α}F(p).\]

If you provide an AbstractBasis B $=\{Z_1,…,Z_d\}$ additionally, the result will be given in coordinates c, i.e. such that $X = \sum_{i=1}^{d} c_iZ_i$.

Note that this is done per every block (vectorial function with its robustifier) of the underlying ManifoldNonlinearLeastSquaresObjective and summed up. See also get_normal_linear_operator for evaluating the corresponding linear operator of the (normal) linear system, and get_LevenbergMarquardt_scaling for details on the scaling and computation of $C$.

get_vector_field(M::AbstractManifold, neo::NormalEquationsObjective, p)
get_vector_field(M::AbstractManifold, neo::NormalEquationsObjective, p, B)
get_vector_field!(M::AbstractManifold, Y, neo::NormalEquationsObjective, p)
get_vector_field!(M::AbstractManifold, c, neo::NormalEquationsObjective, p, B)

Evaluate the vector field related to the normal equations of the LevenbergMarquardtLinearSurrogateObjective. See get_normal_vector_field for details, but note that for the NormalEquationsObjective the format is slightly different: For the variant with _normal the result is similar to the surrogate, namely we have $\mathcal{L}(X) + y$ for the surrogate and hence also the same form $\mathcal{N}(X) + z$, which has to be set to zero to find $X$.

For the objective here we consider ``$\mathcal{N}(X) = z'$, i.e. the get_vector_field z' = -z` differs by a sign.

get_vector_field(M::AbstractManifold, neo::NormalEquationsObjective, p)
get_vector_field(M::AbstractManifold, neo::NormalEquationsObjective, p, B)
get_vector_field!(M::AbstractManifold, Y, neo::NormalEquationsObjective, p)
get_vector_field!(M::AbstractManifold, c, neo::NormalEquationsObjective, p, B)

Evaluate the vector field related to the normal equations of the LevenbergMarquardtLinearSurrogateObjective. See get_normal_vector_field for details, but note that for the NormalEquationsObjective the format is slightly different: For the variant with _normal the result is similar to the surrogate, namely we have $\mathcal{L}(X) + y$ for the surrogate and hence also the same form $\mathcal{N}(X) + z$, which has to be set to zero to find $X$.

For the objective here we consider ``$\mathcal{N}(X) = z'$, i.e. the get_vector_field z' = -z` differs by a sign.

get_vector_field(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p)
get_vector_field!(M::AbstractManifold, X, lmsco::LevenbergMarquardtLinearSurrogateObjective, p)

Compute the vector field $y$ corresponding to the Levenberg-Marquardt surrogate objective, i.e.,

\[y = \frac{\sqrt{ρ'(p)}}{1-α}F(p)\]

where the scaling uses $α = 1 - \sqrt{1 + 2 \frac{ρ''(p)}{ρ'(p)}\lVert F(p) \rVert_{2}^2}$

Note that this is done per every block (vectorial function with its robustifier) of the underlying ManifoldNonlinearLeastSquaresObjective and summed up.

See also

• get_LevenbergMarquardt_scaling for details on the scaling factor
• get_linear_operator for evaluating the corresponding linear operator of the linear system

get_vector_field(M::AbstractManifold, neo::NormalEquationsObjective, p)
get_vector_field(M::AbstractManifold, neo::NormalEquationsObjective, p, B)
get_vector_field!(M::AbstractManifold, Y, neo::NormalEquationsObjective, p)
get_vector_field!(M::AbstractManifold, c, neo::NormalEquationsObjective, p, B)

Evaluate the vector field related to the normal equations of the LevenbergMarquardtLinearSurrogateObjective. See get_normal_vector_field for details, but note that for the NormalEquationsObjective the format is slightly different: For the variant with _normal the result is similar to the surrogate, namely we have $\mathcal{L}(X) + y$ for the surrogate and hence also the same form $\mathcal{N}(X) + z$, which has to be set to zero to find $X$.

For the objective here we consider ``$\mathcal{N}(X) = z'$, i.e. the get_vector_field z' = -z` differs by a sign.

get_vector_field(M::AbstractManifold, neo::NormalEquationsObjective, p)
get_vector_field(M::AbstractManifold, neo::NormalEquationsObjective, p, B)
get_vector_field!(M::AbstractManifold, Y, neo::NormalEquationsObjective, p)
get_vector_field!(M::AbstractManifold, c, neo::NormalEquationsObjective, p, B)

Evaluate the vector field related to the normal equations of the LevenbergMarquardtLinearSurrogateObjective. See get_normal_vector_field for details, but note that for the NormalEquationsObjective the format is slightly different: For the variant with _normal the result is similar to the surrogate, namely we have $\mathcal{L}(X) + y$ for the surrogate and hence also the same form $\mathcal{N}(X) + z$, which has to be set to zero to find $X$.

For the objective here we consider ``$\mathcal{N}(X) = z'$, i.e. the get_vector_field z' = -z` differs by a sign.