Linear system objectives

AbstractSymmetricLinearSystemObjective{E<:AbstractEvaluationType} <: AbstractManifoldObjective{E}

Model the objective

\[f(X) = \frac{1}{2} \lVert \mathcal{A}[X] + b \rVert_{p}^2,\qquad X ∈ T_{p}\mathcal{M},\]

defined on the tangent space $T_{p}\mathcal{M}$ at $p$ on the manifold $\mathcal{M}$.

In other words this is an objective to solve $\mathcal{A} = -b(p)$ for some linear symmetric operator $\mathcal{A}$ and a vector function $b$

Concrete subtypes of this type should/could implement

• get_cost(TpM, aslso, X) to compute/evaluate the objective
• get_gradient(TpM, aslso, X) to compute/evaluate the objectives gradient at X
• get_linear_operator(TpM, aslso, X) to compute/evaluate the linear operator $\mathcal{A}$ at X

SymmetricLinearSystemObjective{E<:AbstractEvaluationType,TA,T} <: AbstractSymmetricLinearSystemObjective{E}

Model the objective

\[f(X) = \frac{1}{2} \lVert \mathcal{A}[X] + b \rVert_{p}^2,\qquad X ∈ T_{p}\mathcal{M},\]

defined on the tangent space $T_{p}\mathcal{M}$ at $p$ on the manifold $\mathcal{M}$.

In other words this is an objective to solve $\mathcal{A} = -b(p)$ for some linear symmetric operator and a vector function. Note the minus on the right hand side, which makes this objective especially tailored for (iteratively) solving Newton-like equations.

Fields

• A!!: a symmetric, linear operator on the tangent space, see get_linear_operator
• b!!: a tangent vector function, see get_vector_field

where A!! can work as an allocating operator (M, p, X) -> Y or an in-place one (M, Y, p, X) -> Y, and similarly b!! can either be a function (M, p) -> X or (M, X, p) -> X. The first variants allocate for the result, the second variants work in-place.

Constructor

SymmetricLinearSystemObjective(A, b; evaluation=AllocatingEvaluation())

Generate the objective specifying whether the two parts work allocating or in-place.

get_linear_operator(M::AbstractManifold, lsmo::AbstractLinearSurrogateObjective, p)
get_linear_operator(M::AbstractManifold, lsmo::AbstractLinearSurrogateObjective, p, B::AbstractBasis)
get_linear_operator(M::AbstractManifold, lsmo::AbstractLinearSurrogateObjective, p, X)
get_linear_operator!(M::AbstractManifold, L, lsmo::AbstractLinearSurrogateObjective, p, B::AbstractBasis)
get_linear_operator!(M::AbstractManifold, Y, lsmo::AbstractLinearSurrogateObjective, p, X)

Return/Evaluate the linear operator $\mathcal{L}$ of the linear surrogate model lsmo at the point $p ∈ M$.

If a tangent vector X is provided, evaluate $\mathcal{L}(X)$. If a basis B is provided, return the matrix representation of $\mathcal{L}$ with respect to that basis. Otherwise return the operator as a function (TpM, X) -> Y.

get_linear_operator(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, X)
get_linear_operator!(M::AbstractManifold, y, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, X)

Evaluate the linear operator $\mathcal{L}$ corresponding to the Levenberg-Marquardt surrogate objective, i.e.,

\[\mathcal{L}(X) = C J_F(p)[X] \bigr],\]

with

\[C = \sqrt{ρ'(p)}(I-αP), \qquad P = \frac{F(p)F(p)^\mathrm{T}}{\lVert F(p) \rVert_{2}^2},\]

where $α = 1 - \sqrt{1 + 2 \frac{ρ''(p)}{ρ'(p)}\lVert F(p) \rVert_{2}^2}$.

Note that this is done per every block (vectorial function with its robustifier) of the underlying ManifoldNonlinearLeastSquaresObjective and summed up.

This can be computed in-place of y.

See also get_vector_field for evaluating the corresponding vector field

get_linear_operator(M::AbstractManifold, neo::NormalEquationsObjective, p, X)
get_linear_operator(M::AbstractManifold, neo::NormalEquationsObjective, p, c, B)
get_linear_operator(M::AbstractManifold, neo::NormalEquationsObjective, p, B)
get_linear_operator!(M::AbstractManifold, Y, neo::NormalEquationsObjective, p, X)
get_linear_operator!(M::AbstractManifold, b, neo::NormalEquationsObjective, p, c, B)
get_linear_operator!(M::AbstractManifold, A, neo::NormalEquationsObjective, p, B)

Evaluate the linear operator related to the normal equations of the [`LevenbergMarquardtLinearSurrogateObjective`](@ref),
see [`get_normal_linear_operator`](@ref) for details.

get_linear_operator(M::AbstractManifold, neo::NormalEquationsObjective, p, X)
get_linear_operator(M::AbstractManifold, neo::NormalEquationsObjective, p, c, B)
get_linear_operator(M::AbstractManifold, neo::NormalEquationsObjective, p, B)
get_linear_operator!(M::AbstractManifold, Y, neo::NormalEquationsObjective, p, X)
get_linear_operator!(M::AbstractManifold, b, neo::NormalEquationsObjective, p, c, B)
get_linear_operator!(M::AbstractManifold, A, neo::NormalEquationsObjective, p, B)

Evaluate the linear operator related to the normal equations of the [`LevenbergMarquardtLinearSurrogateObjective`](@ref),
see [`get_normal_linear_operator`](@ref) for details.

get_vector_field(M::AbstractManifold, slso::SymmetricLinearSystemObjective, p)
get_vector_field!(M::AbstractManifold, Y, slso::SymmetricLinearSystemObjective, p)
get_vector_field(TpM::TangentSpace, slso::SymmetricLinearSystemObjective)
get_vector_field!(TpM::TangentSpace, Y, slso::SymmetricLinearSystemObjective)

evaluate the stored value for computing the right hand side $b$ in $\mathcal{A}=-b$, either providing a tangent space or a manifold and a point.

get_vector_field(M::AbstractManifold, lsmo::AbstractLinearSurrogateObjective, p)
get_vector_field!(M::AbstractManifold, y, lsmo::AbstractLinearSurrogateObjective, p)

Return the vector y of the linear surrogate model lsmo at the point $p ∈ M$.

get_vector_field(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p)
get_vector_field!(M::AbstractManifold, X, lmsco::LevenbergMarquardtLinearSurrogateObjective, p)

Compute the vector field $y$ corresponding to the Levenberg-Marquardt surrogate objective, i.e.,

\[y = \frac{\sqrt{ρ'(p)}}{1-α}F(p)\]

where the scaling uses $α = 1 - \sqrt{1 + 2 \frac{ρ''(p)}{ρ'(p)}\lVert F(p) \rVert_{2}^2}$

Note that this is done per every block (vectorial function with its robustifier) of the underlying ManifoldNonlinearLeastSquaresObjective and summed up.

See also

• get_LevenbergMarquardt_scaling for details on the scaling factor
• get_linear_operator for evaluating the corresponding linear operator of the linear system

get_vector_field(M::AbstractManifold, neo::NormalEquationsObjective, p)
get_vector_field(M::AbstractManifold, neo::NormalEquationsObjective, p, B)
get_vector_field!(M::AbstractManifold, Y, neo::NormalEquationsObjective, p)
get_vector_field!(M::AbstractManifold, c, neo::NormalEquationsObjective, p, B)

Evaluate the vector field related to the normal equations of the [`LevenbergMarquardtLinearSurrogateObjective`](@ref),
see [`get_normal_vector_field`](@ref) for details,
but note that for the [`NormalEquationsObjective`](@ref) the format is slightly different:
For the variant with `_normal` the result is similar to the surrogate, namely we have
``\mathcal{L}(X) + y`` for the surrogate and hence also the same form ``\mathcal{N}(X) + z``,
which has to be set to zero to find ``X``.

For the objective here we consider ````\mathcal{N}(X) = z'``, i.e. the `get_vector_field` `z' = -z``
differs by a sign.

get_vector_field(M::AbstractManifold, neo::NormalEquationsObjective, p)
get_vector_field(M::AbstractManifold, neo::NormalEquationsObjective, p, B)
get_vector_field!(M::AbstractManifold, Y, neo::NormalEquationsObjective, p)
get_vector_field!(M::AbstractManifold, c, neo::NormalEquationsObjective, p, B)

Evaluate the vector field related to the normal equations of the [`LevenbergMarquardtLinearSurrogateObjective`](@ref),
see [`get_normal_vector_field`](@ref) for details,
but note that for the [`NormalEquationsObjective`](@ref) the format is slightly different:
For the variant with `_normal` the result is similar to the surrogate, namely we have
``\mathcal{L}(X) + y`` for the surrogate and hence also the same form ``\mathcal{N}(X) + z``,
which has to be set to zero to find ``X``.

For the objective here we consider ````\mathcal{N}(X) = z'``, i.e. the `get_vector_field` `z' = -z``
differs by a sign.

get_normal_linear_operator(M::AbstractManifold, lsmo::AbstractLinearSurrogateObjective, p)
get_normal_linear_operator(M::AbstractManifold, lsmo::AbstractLinearSurrogateObjective, p, B::AbstractBasis)
get_normal_linear_operator(M::AbstractManifold, lsmo::AbstractLinearSurrogateObjective, p, X)
get_normal_linear_operator!(M::AbstractManifold, N, lsmo::AbstractLinearSurrogateObjective, p, B::AbstractBasis)
get_normal_linear_operator!(M::AbstractManifold, Y, lsmo::AbstractLinearSurrogateObjective, p, X)

Return/Evaluate the normal operator $\mathcal{L}^* \mathcal{L}$ of the linear surrogate model lsmo at the point $p ∈ M$.

If a tangent vector X is provided, evaluate $\mathcal{L}^* \mathcal{L}(X)$. If a basis B is provided, return the matrix representation of $\mathcal{L}^* \mathcal{L}$ with respect to that basis. Otherwise return the operator as a function (TpM, X) -> Y.

get_normal_linear_operator(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, X; threshold = lmsco.threshold, mode = lmsco.mode, penalty = lmsco.penalty)
get_normal_linear_operator!(M::AbstractManifold, Y, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, X; threshold = lmsco.threshold, mode = lmsco.mode, penalty = lmsco.penalty)
get_normal_linear_operator(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p[, c], B::AbstractBasis; threshold = lmsco.threshold, mode = lmsco.mode, penalty = lmsco.penalty)
get_normal_linear_operator!(M::AbstractManifold, [A | b], lmsco::LevenbergMarquardtLinearSurrogateObjective, p[, c], B::AbstractBasis; threshold = lmsco.threshold, mode = lmsco.mode, penalty = lmsco.penalty)

Compute the linear operator $\mathcal{A}$ corresponding to the optimality conditions of the modified Levenberg-Marquardt surrogate objective, i.e. the normal conditions

\[\mathcal{A}(X) = \mathcal{L}^* \mathcal{L}(X) + λX = J_F^*(p)\bigl[ C^T C J_F(p)[X] \bigr] + λX,\]

where $λ =$penalty is a damping parameter and with $α = 1 - \sqrt{1 + 2 \frac{ρ''(p)}{ρ'(p)}\lVert F(p) \rVert_{2}^2}$ we have

\[C = \sqrt{ρ'(p)}(I-αP), \qquad P = \frac{F(p)F(p)^\mathrm{T}}{\lVert F(p) \rVert_{2}^2},\]

See get_LevenbergMarquardt_scaling for details on the scaling and α. and get_jacobian and get_adjoint_jacobian concerning $J_F$ and $J_F^*$, respectively.

There are three variants to use this function to use the corresponding linear operator

• if you provide a tangent vector X, then the linear operator is evaluated at X, the corresponding gradient Y is returned
• if you provide X in coordinates c with respect to a basis B the linear operator is evaluated and the coordinates b of the result are returned
• if you provide (just) a basis B of the tangent space, then the matrix A of the linear operator represented in this basis is returned. The relation to the second case is that $b = Ac$.

See also get_normal_vector_field for evaluating the corresponding vector field.

get_normal_vector_field(M::AbstractManifold, lsmo::AbstractLinearSurrogateObjective, p)
get_normal_vector_field!(M::AbstractManifold, y, lsmo::AbstractLinearSurrogateObjective, p)

Return the normal vector $\mathcal{L}^*(y)$ of the linear surrogate model lsmo at the point $p ∈ M$.

get_normal_vector_field(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p)
get_normal_vector_field!(M::AbstractManifold, X, lmsco::LevenbergMarquardtLinearSurrogateObjective, p)
get_normal_vector_field(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, B::AbstractBasis)
get_normal_vector_field!(M::AbstractManifold, c, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, B::AbstractBasis)

Compute the normal linear operator tangent vector $X$ corresponding to the normal equations (optimality conditions) of the Levenberg-Marquardt surrogate objective, i.e.,

\[X = J_F^*(p)[ C^T y], \quad y = \frac{\sqrt{ρ'(p)}}{1-α}F(p).\]

If you provide an AbstractBasis B $=\{Z_1,…,Z_d\}$ additionally, the result will be given in coordinates c, i.e. such that $X = \sum_{i=1}^{d} c_iZ_i$.

Note that this is done per every block (vectorial function with its robustifier) of the underlying ManifoldNonlinearLeastSquaresObjective and summed up. See also get_normal_linear_operator for evaluating the corresponding linear operator of the (normal) linear system, and get_LevenbergMarquardt_scaling for details on the scaling and computation of $C$.

get_normal_vector_field(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p)
get_normal_vector_field!(M::AbstractManifold, X, lmsco::LevenbergMarquardtLinearSurrogateObjective, p)
get_normal_vector_field(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, B::AbstractBasis)
get_normal_vector_field!(M::AbstractManifold, c, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, B::AbstractBasis)

Compute the normal linear operator tangent vector $X$ corresponding to the normal equations (optimality conditions) of the Levenberg-Marquardt surrogate objective, i.e.,

\[X = J_F^*(p)[ C^T y], \quad y = \frac{\sqrt{ρ'(p)}}{1-α}F(p).\]

If you provide an AbstractBasis B $=\{Z_1,…,Z_d\}$ additionally, the result will be given in coordinates c, i.e. such that $X = \sum_{i=1}^{d} c_iZ_i$.

Note that this is done per every block (vectorial function with its robustifier) of the underlying ManifoldNonlinearLeastSquaresObjective and summed up. See also get_normal_linear_operator for evaluating the corresponding linear operator of the (normal) linear system, and get_LevenbergMarquardt_scaling for details on the scaling and computation of $C$.

get_normal_vector_field(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p)
get_normal_vector_field!(M::AbstractManifold, X, lmsco::LevenbergMarquardtLinearSurrogateObjective, p)
get_normal_vector_field(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, B::AbstractBasis)
get_normal_vector_field!(M::AbstractManifold, c, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, B::AbstractBasis)

Compute the normal linear operator tangent vector $X$ corresponding to the normal equations (optimality conditions) of the Levenberg-Marquardt surrogate objective, i.e.,

\[X = J_F^*(p)[ C^T y], \quad y = \frac{\sqrt{ρ'(p)}}{1-α}F(p).\]

If you provide an AbstractBasis B $=\{Z_1,…,Z_d\}$ additionally, the result will be given in coordinates c, i.e. such that $X = \sum_{i=1}^{d} c_iZ_i$.

Note that this is done per every block (vectorial function with its robustifier) of the underlying ManifoldNonlinearLeastSquaresObjective and summed up. See also get_normal_linear_operator for evaluating the corresponding linear operator of the (normal) linear system, and get_LevenbergMarquardt_scaling for details on the scaling and computation of $C$.

get_normal_vector_field(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p)
get_normal_vector_field!(M::AbstractManifold, X, lmsco::LevenbergMarquardtLinearSurrogateObjective, p)
get_normal_vector_field(M::AbstractManifold, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, B::AbstractBasis)
get_normal_vector_field!(M::AbstractManifold, c, lmsco::LevenbergMarquardtLinearSurrogateObjective, p, B::AbstractBasis)

Compute the normal linear operator tangent vector $X$ corresponding to the normal equations (optimality conditions) of the Levenberg-Marquardt surrogate objective, i.e.,

\[X = J_F^*(p)[ C^T y], \quad y = \frac{\sqrt{ρ'(p)}}{1-α}F(p).\]

If you provide an AbstractBasis B $=\{Z_1,…,Z_d\}$ additionally, the result will be given in coordinates c, i.e. such that $X = \sum_{i=1}^{d} c_iZ_i$.

Note that this is done per every block (vectorial function with its robustifier) of the underlying ManifoldNonlinearLeastSquaresObjective and summed up. See also get_normal_linear_operator for evaluating the corresponding linear operator of the (normal) linear system, and get_LevenbergMarquardt_scaling for details on the scaling and computation of $C$.