Common functions

ApproxHessianBFGS{P, G, T, B<:AbstractBasis{ℝ}, VTR} <: AbstractApproximateHessianFunction

A functor to approximate the Hessian by the BFGS update.

Fields

• gradient!: the gradient function (either allocating or mutating, see evaluation parameter).
• scale::Bool: whether to scale the initial approximating operator.
• vector_transport_method::AbstractVectorTransportMethod: a vector transport $\mathcal T_{⋅←⋅}$ to use, see the section on vector transports

Internal temporary fields

• p_tmp: a temporary storage for the current point p.
• grad_tmp: a temporary storage for the gradient at the current p.
• matrix: a temporary storage for the matrix representation of the approximating operator.
• basis: a temporary storage for an orthonormal basis at the current p.

Constructor

ApproxHessianBFGS(M, p, gradF; kwargs...)

Keyword arguments

• evaluation::AbstractEvaluationType=AllocatingEvaluation(): specify whether the functions that return an array, for example a point or a tangent vector, work by allocating its result (AllocatingEvaluation) or whether they modify their input argument to return the result therein (InplaceEvaluation). Since usually the first argument is the manifold, the modified argument is the second.
• initial_operator=Matrix{Float64}(I, manifold_dimension(M), manifold_dimension(M)): the matrix representation of the initial approximating operator.
• basis=default_basis(M, typeof(p)): an orthonormal basis in the tangent space of the initial iterate p.
• scale=true: the value of the scale field above.

ApproxHessianFiniteDifference{P, T, G, RTR, VTR, R <: Real} <: AbstractApproximateHessianFunction

A functor to approximate the Hessian by a finite difference of gradient evaluation.

Given a point p and a direction X and the gradient $\operatorname{grad} f(p)$ of a function $f$ the Hessian is approximated as follows: let $c$ be a stepsize, $X ∈ T_{p}\mathcal{M}$ a tangent vector and $q = \operatorname{retr}_p(\frac{c}{\lVert X \rVert_p}X)$ be a step in direction $X$ of length $c$ following a retraction. Then the Hessian is approximated by the finite difference of the gradients, where $\mathcal T_{⋅←⋅}$ is a vector transport.

\[\operatorname{Hess}f(p)[X] ≈ \frac{\lVert X \rVert}{c}\Bigl( \mathcal T_{p←q}\bigl(\operatorname{grad}f(q)\bigr) - \operatorname{grad}f(p) \Bigr)\]

Fields

• gradient!: the gradient function (either allocating or mutating, see evaluation parameter)
• steplength: a step length for the finite difference
• retraction_method::AbstractRetractionMethod: a retraction $\operatorname{retr}$ to use, see the section on retractions
• vector_transport_method::AbstractVectorTransportMethod: a vector transport $\mathcal T_{⋅←⋅}$ to use, see the section on vector transports

Internal temporary fields

• grad_tmp: a temporary storage for the gradient at the current p
• grad_tmp_dir: a temporary storage for the gradient at the current p_dir
• p_dir::P: a temporary storage for the forward direction (or the $q$ in the formula)

Constructor

ApproxHessianFiniteDifference(M, p, grad_f; kwargs...)

Keyword arguments

• steplength=2^-14: step length $c$ to approximate the gradient evaluations
• evaluation::AbstractEvaluationType=AllocatingEvaluation(): specify whether the functions that return an array, for example a point or a tangent vector, work by allocating its result (AllocatingEvaluation) or whether they modify their input argument to return the result therein (InplaceEvaluation). Since usually the first argument is the manifold, the modified argument is the second.
• retraction_method::AbstractRetractionMethod=default_retraction_method(M, typeof(p)): a retraction $\operatorname{retr}$ to use, see the section on retractions
• vector_transport_method::AbstractVectorTransportMethod=default_vector_transport_method(M, typeof(p)): a vector transport $\mathcal T_{⋅←⋅}$ to use, see the section on vector transports

ApproxHessianSymmetricRankOne{P, G, T, B<:AbstractBasis{ℝ}, VTR, R<:Real} <: AbstractApproximateHessianFunction

A functor to approximate the Hessian by the symmetric rank one update.

Fields

• gradient!: the gradient function (either allocating or mutating, see evaluation parameter).
• ν: a small real number to ensure that the denominator in the update does not become too small and thus the method does not break down.
• vector_transport_method::AbstractVectorTransportMethod: a vector transport $\mathcal T_{⋅←⋅}$ to use, see the section on vector transports

Internal temporary fields

• p_tmp: a temporary storage for the current point p.
• grad_tmp: a temporary storage for the gradient at the current p.
• matrix: a temporary storage for the matrix representation of the approximating operator.
• basis: a temporary storage for an orthonormal basis at the current p.

Constructor

ApproxHessianSymmetricRankOne(M, p, gradF; kwargs...)

Keyword arguments

• evaluation::AbstractEvaluationType=AllocatingEvaluation(): specify whether the functions that return an array, for example a point or a tangent vector, work by allocating its result (AllocatingEvaluation) or whether they modify their input argument to return the result therein (InplaceEvaluation). Since usually the first argument is the manifold, the modified argument is the second.
• initial_operator=Matrix{Float64}(I, manifold_dimension(M), manifold_dimension(M)): the matrix representation of the initial approximating operator.
• basis=default_basis(M, typeof(p)): an orthonormal basis in the tangent space of the initial iterate p.
• nu=-1.0: the value $ν$ above; a negative value disables the safeguard on the denominator.

reflect(M, p, x; kwargs...)
reflect!(M, q, p, x; kwargs...)

Reflect the point x from the manifold M at point p.

The formula is given by

\[\operatorname{refl}_p(x) = \operatorname{retr}_p(-\operatorname{retr}^{-1}_p x),\]

where $\operatorname{retr}$ and $\operatorname{retr}^{-1}$ denote a retraction and an inverse retraction, respectively. This can also be done in place of q.

Keyword Arguments

• retraction_method::AbstractRetractionMethod=default_retraction_method(M, typeof(p)): a retraction $\operatorname{retr}$ to use, see the section on retractions
• inverse_retraction_method::AbstractInverseRetractionMethod=default_inverse_retraction_method(M, typeof(p)): an inverse retraction $\operatorname{retr}^{-1}$ to use, see the section on retractions and their inverses
• X::T =zero_vector(M, p): a tangent vector at the point $p$ on the manifold $\mathcal{M}$ as temporary memory to compute the inverse retraction in place; otherwise this is the memory that would be allocated anyways.

reflect(M, pr::Function, x; kwargs...)
reflect!(M, q, pr::Function, x; kwargs...)

Reflect the point x from the manifold M at the point $p = \operatorname{prox}(x)$, where the proximal map is given by pr.

The formula is given by

\[\operatorname{refl}_p(q) = \operatorname{retr}_p(-\operatorname{retr}^{-1}_p q),\]

where $\operatorname{retr}$ and $\operatorname{retr}^{-1}$ denote a retraction and an inverse retraction, respectively.

This can also be done in place of q.

Keyword Arguments

• retraction_method::AbstractRetractionMethod=default_retraction_method(M, typeof(p)): a retraction $\operatorname{retr}$ to use, see the section on retractions
• inverse_retraction_method::AbstractInverseRetractionMethod=default_inverse_retraction_method(M, typeof(p)): an inverse retraction $\operatorname{retr}^{-1}$ to use, see the section on retractions and their inverses
• X=zero_vector(M,p): temporary memory to compute the inverse retraction in place; otherwise this is the memory that would be allocated anyways.

InplaceManifoldFunction{result, F} <: AbstractDecoratedManifoldFunction{F}

Wrapper for a function to ensure it works in-place. Since the action to perform to the provided return value differs per type, the following cases for results are available:

• :Point use copyto! for a point on a manifold
• :Points use an element-wise copyto! for points
• :TangentVector use copyto! for a tangent vector
• :TangentVectors use an elementwise copyto! for tangent vectors
• :Number assume the result to be a 0-dimensional array.
• :NumberAndTangentVector for the combination (c, X) of a number and a tangent vector – return c and handle X with the copyto! for a tangent vector
• :MaybeResizeVector for a vector to return, make sure the size is adapted if needed. This is useful e.g. for return values of sub solvers that might vary in length
• :Default (also all other symbols) just use a plain copyto!

For those that require an additional point like the tangent vectors, the point is taken as the point_index entry of the args...

Fields

• f::F : the function to be wrapped of the form (M, args...) -> v
• point_index: which of the arguments args... is the point to be used in the copy
  • the default that works for most functions is 1
  • for the proximal map (M, λ, p) this is index 2

The type parameter result specifies which of the cases above to use.

Constructor

InplaceManifoldFunction(f, result = :Point)

MutableManifoldFunction{result, P, F} <: AbstractDecoratedManifoldFunction{F}

A wrapper for a function defined on a manifold to ensure it works on mutable variables, internally “unwrapping” them to numbers before calling the function that is wrapped.

Since the function works on immutable input types, it is assumed to work allocating. To use it within objectives of Manopt.jl, consider wrapping e.g. gradient or Hessian functions furthermore in an InplaceManifoldFunction.

Fields

• f::F : the function to be wrapped of the form (M, args...) -> v

The type parameter result specifies the type of result. If the result is expected to be a :Number, it is kept as is; for anything else, like a :Point or :TangentVector, the result is returned (again) as a mutable variable.

Constructor

MutableManifoldFunction(f, p::P, result = :Number)MutableManifoldFunction(f, P, result = :Number)

Initialize the wrapper for a function f defined on a manifold, where p is a point on the manifold, to store the original point type P for the arguments.

maybe_unwrap_variable(p::P, q::P)
maybe_unwrap_variable(p::P, q::Vector{P})

Undo the wrapping performed by maybe_wrap_variable, i.e. given the original input variable p and the possibly wrapped variable q, return the unwrapped variable, i.e. if q is a 1-element vector of same element-type P as the type of p, return this one element.

maybe_wrap_function(f, p, evaluation = InplaceEvaluation(); result = :Number)
maybe_wrap_function(f, evaluation = InplaceEvaluation(); result = :Number)

Wrap a function f defined on a manifold to work in-place on mutable variables, i.e. first if the input variable p is a number, the function f is wrapped in a MutableManifoldFunction. If the function then has an AllocatingEvaluation as its evaluation type, it is wrapped in an InplaceManifoldFunction to work in-place of the result.

The first step is skipped if the input variable p is not a number, missing or not provided.

maybe_wrap_variable(v)

For a number variable v wrap it in a 1-element vector to make it mutable. Otherwise return the variable as is.

CondensedKKTVectorField{O<:ConstrainedManifoldObjective,T,R} <: AbstractConstrainedSlackFunction{T,R}

Given the constrained optimization problem

\[\begin{aligned} \min_{p ∈ \mathcal{M}} & f(p)\\\\ \text{ subject to } &g_i(p) ≤ 0 \quad \text{ for } i= 1, …, m,\\\\ \quad & h_j(p) = 0 \quad\text{ for } j=1,…,n, \end{aligned} \]

we reformulate the KKT conditions of the Lagrangian from the optimality conditions of the Lagrangian

\[\mathcal{L}(p, μ, λ) = f(p) + \sum_{j=1}^{n} λ_jh_j(p) + \sum_{i=1}^{m} μ_ig_i(p)\]

in a perturbed / barrier method in a condensed form using a slack variable $s ∈ ℝ^m$ and a barrier parameter $β$ and the Riemannian gradient of the Lagrangian with respect to the first parameter $\operatorname{grad}_p \mathcal{L}(p, μ, λ)$.

Let $\mathcal{N} = \mathcal{M} × ℝ^n$. We obtain the linear system

\[\mathcal{A}(p,λ)[X,Y] = -b(p,λ),\qquad \text{where } (X,Y) ∈ T_{(p, λ)}\mathcal{N}\]

where $\mathcal{A}: T_{(p, λ)}\mathcal{N} → T_{(p, λ)}\mathcal{N}$ is a linear operator and this struct models the right hand side $b(p,λ) ∈ T_{(p, λ)}\mathcal{N}$ given by

\[b(p,λ) = \begin{pmatrix} \operatorname{grad} f(p) + \displaystyle\sum_{j=1}^{n} λ_j \operatorname{grad} h_j(p) + \displaystyle\sum_{i=1}^{m} μ_i \operatorname{grad} g_i(p) + \displaystyle\sum_{i=1}^{m} \frac{μ_i}{s_i}\bigl( μ_i(g_i(p)+s_i) + β - μ_is_i \bigr)\operatorname{grad} g_i(p)\\ h(p)\end{pmatrix}\]

Fields

• cmo the ConstrainedManifoldObjective
• μ::T the vector in $ℝ^m$ of coefficients for the inequality constraints
• s::T the vector in $ℝ^m$ of slack variables
• β::R the barrier parameter $β∈ℝ$

Constructor

CondensedKKTVectorField(cmo, μ, s, β)

CondensedKKTVectorFieldJacobian{O<:ConstrainedManifoldObjective,T,R}  <: AbstractConstrainedSlackFunction{T,R}

Given the constrained optimization problem

\[\begin{aligned} \min_{p ∈ \mathcal{M}} & f(p)\\\\ \text{subject to} & g_i(p) ≤ 0 \quad\text{ for } i= 1, …, m,\\\\ \quad & h_j(p)=0 \quad \text{ for } j=1,…,n, \end{aligned} \]

we reformulate the KKT conditions of the Lagrangian from the optimality conditions of the Lagrangian

\[\mathcal{L}(p, μ, λ) = f(p) + \sum_{j=1}^{n} λ_jh_j(p) +\sum_{i=1}^{m} μ_ig_i(p)\]

in a perturbed / barrier method in an enhanced as well as condensed form, using $\operatorname{grad}_p \mathcal{L}(p, μ, λ)$, the Riemannian gradient of the Lagrangian with respect to the first parameter.

Let $\mathcal{N} = \mathcal{M} × ℝ^n$. We obtain the linear system

\[\mathcal{A}(p,λ)[X,Y] = -b(p,λ),\qquad \text{where } X ∈ T_p\mathcal{M}, Y ∈ ℝ^n\]

where $\mathcal{A}: T_{(p,λ)}\mathcal{N} → T_{(p,λ)}\mathcal{N}$ is a linear operator on $T_{(p,λ)}\mathcal{N} = T_p\mathcal{M} × ℝ^n$ given by

\[\mathcal{A}(p,λ)[X,Y] = \begin{pmatrix} \operatorname{Hess}_p\mathcal{L}(p, μ, λ)[X] + \displaystyle\sum_{i=1}^{m} \frac{μ_i}{s_i} ⟨\operatorname{grad} g_i(p), X⟩\operatorname{grad} g_i(p) + \displaystyle\sum_{j=1}^{n} Y_j \operatorname{grad} h_j(p)\\ \Bigl( ⟨\operatorname{grad} h_j(p), X⟩ \Bigr)_{j=1}^n\end{pmatrix}\]

Fields

• cmo the ConstrainedManifoldObjective
• μ::T the vector in $ℝ^m$ of coefficients for the inequality constraints
• s::T the vector in $ℝ^m$ of slack variables
• β::R the barrier parameter $β∈ℝ$

Constructor

CondensedKKTVectorFieldJacobian(cmo, μ, s, β)

ExactPenaltyCost{S, CO, R} <: AbstractManifoldFunction

Represent the cost of the exact penalty method based on a ConstrainedManifoldObjective co and a parameter $ρ$ given by

\[f(p) + ρ\Bigl( \sum_{i=1}^{m} \max\{0,g_i(p)\} + \sum_{j=1}^{n} \lvert h_j(p) \rvert \Bigr),\]

where an additional parameter $u$ is used as well as a smoothing technique, for example LogarithmicSumOfExponentials or LinearQuadraticHuber to obtain a smooth cost function. This struct is also a functor (M,p) -> v of the cost $v$.

Fields

• ρ, u: as described in the mathematical formula.
• co: the original cost

Constructor

ExactPenaltyCost(co::ConstrainedManifoldObjective, ρ, u; smoothing=LinearQuadraticHuber())

ExactPenaltyGrad{S, CO, R} <: AbstractConstrainedFunction{R}

Represent the gradient of the ExactPenaltyCost based on a ConstrainedManifoldObjective co and a parameter $ρ$ and a smoothing technique, which uses an additional parameter $u$.

This struct is also a functor in both formats

• (M, p) -> X to compute the gradient in allocating fashion.
• (M, X, p) to compute the gradient in an in-place fashion.

Fields

• ρ, u as stated before
• co the nonsmooth objective

Constructor

ExactPenaltyGrad(co::ConstrainedManifoldObjective, ρ, u; smoothing=LinearQuadraticHuber())

KKTVectorField{O<:ConstrainedManifoldObjective}

Implement the vector field $F$ of the KKT-conditions, including a slack variable for the inequality constraints.

Given the LagrangianCost

\[\mathcal{L}(p; μ, λ) = f(p) + \sum_{i=1}^{m} μ_ig_i(p) + \sum_{j=1}^{n} λ_jh_j(p)\]

the LagrangianGradient

\[\operatorname{grad}\mathcal{L}(p, μ, λ) = \operatorname{grad}f(p) + \sum_{j=1}^{n} λ_j \operatorname{grad} h_j(p) + \sum_{i=1}^{m} μ_i \operatorname{grad} g_i(p),\]

and introducing the slack variables $s=-g(p) ∈ ℝ^m$ the vector field is given by

\[F(p, μ, λ, s) = \begin{pmatrix} \operatorname{grad}_p \mathcal{L}(p, μ, λ)\\ g(p) + s\\ h(p)\\ μ ⊙ s\end{pmatrix},\]

where $p ∈ \mathcal{M}$, $μ, s ∈ ℝ^m$ and $λ ∈ ℝ^n$, and $⊙$ denotes the Hadamard (or elementwise) product.

Fields

• cmo the ConstrainedManifoldObjective

Constructor

KKTVectorField(cmo::ConstrainedManifoldObjective)

Example

Define F = KKTVectorField(cmo) for some ConstrainedManifoldObjective cmo and let N be the product manifold of $\mathcal{M}×ℝ^m×ℝ^n×ℝ^m$. Then, you can call this cost as F(N, q) or as the in-place variant F(N, Y, q), where q is a point on N and Y is a tangent vector at q for the result.

KKTVectorFieldAdjointJacobian{O<:ConstrainedManifoldObjective}

Implement the Adjoint of the Jacobian of the vector field $F$ of the KKT-conditions, including a slack variable for the inequality constraints, see KKTVectorField and KKTVectorFieldJacobian.

\[\operatorname{J}^* F(p, μ, λ, s)[X, Y, Z, W] = \begin{pmatrix} \operatorname{Hess}_p \mathcal{L}(p, μ, λ)[X] + \displaystyle\sum_{i=1}^{m} Y_i \operatorname{grad} g_i(p) + \displaystyle\sum_{j=1}^{n} Z_j \operatorname{grad} h_j(p)\\ \Bigl( ⟨\operatorname{grad} g_i(p), X⟩ + s_iW_i\Bigr)_{i=1}^m\\ \Bigl( ⟨\operatorname{grad} h_j(p), X⟩ \Bigr)_{j=1}^n\\ μ ⊙ W + Y\end{pmatrix},\]

where $⊙$ denotes the Hadamard (or elementwise) product.

See also the LagrangianHessian $\operatorname{Hess}_p \mathcal{L}(p, μ, λ)[X]$.

Fields

• cmo the ConstrainedManifoldObjective

Constructor

KKTVectorFieldAdjointJacobian(cmo::ConstrainedManifoldObjective)

Generate the Adjoint Jacobian of the KKT vector field related to some ConstrainedManifoldObjective cmo.

Example

Define AdJF = KKTVectorFieldAdjointJacobian(cmo) for some ConstrainedManifoldObjective cmo and let N be the product manifold of $\mathcal{M}×ℝ^m×ℝ^n×ℝ^m$. Then, you can call this cost as AdJF(N, q, Y) or as the in-place variant AdJF(N, Z, q, Y), where q is a point on N and Y and Z are tangent vectors at q.

KKTVectorFieldJacobian{O<:ConstrainedManifoldObjective}

Implement the Jacobian of the vector field $F$ of the KKT-conditions, including a slack variable for the inequality constraints, see KKTVectorField and KKTVectorFieldAdjointJacobian.

\[\operatorname{J} F(p, μ, λ, s)[X, Y, Z, W] = \begin{pmatrix} \operatorname{Hess}_p \mathcal{L}(p, μ, λ)[X] + \displaystyle\sum_{i=1}^{m} Y_i \operatorname{grad} g_i(p) + \displaystyle\sum_{j=1}^{n} Z_j \operatorname{grad} h_j(p)\\ \Bigl( ⟨\operatorname{grad} g_i(p), X⟩ + W_i\Bigr)_{i=1}^m\\ \Bigl( ⟨\operatorname{grad} h_j(p), X⟩ \Bigr)_{j=1}^n\\ μ ⊙ W + s ⊙ Y\end{pmatrix}\]

where $⊙$ denotes the Hadamard (or elementwise) product.

See also the LagrangianHessian $\operatorname{Hess}_p \mathcal{L}(p, μ, λ)[X]$.

Fields

• cmo the ConstrainedManifoldObjective

Constructor

KKTVectorFieldJacobian(cmo::ConstrainedManifoldObjective)

Generate the Jacobian of the KKT vector field related to some ConstrainedManifoldObjective cmo.

Example

Define JF = KKTVectorFieldJacobian(cmo) for some ConstrainedManifoldObjective cmo and let N be the product manifold of $\mathcal{M}×ℝ^m×ℝ^n×ℝ^m$. Then, you can call this cost as JF(N, q, Y) or as the in-place variant JF(N, Z, q, Y), where q is a point on N and Y and Z are tangent vectors at q.

KKTVectorFieldNormSq{O<:ConstrainedManifoldObjective}

Implement the square of the norm of the vector field $F$ of the KKT-conditions, including a slack variable for the inequality constraints, see KKTVectorField, to which this functor applies the norm. In [LY24] this is called the merit function.

Fields

• cmo the ConstrainedManifoldObjective

Constructor

KKTVectorFieldNormSq(cmo::ConstrainedManifoldObjective)

Example

Define f = KKTVectorFieldNormSq(cmo) for some ConstrainedManifoldObjective cmo and let N be the product manifold of $\mathcal{M}×ℝ^m×ℝ^n×ℝ^m$. Then, you can call this cost as f(N, q), where q is a point on N.

KKTVectorFieldNormSqGradient{O<:ConstrainedManifoldObjective}

Compute the gradient of the KKTVectorFieldNormSq $φ(p,μ,λ,s) = \lVert F(p,μ,λ,s) \rVert^2$, that is of the norm squared of the KKTVectorField $F$.

This is given in [LY24] as the gradient of their merit function, which we can write with the adjoint $J^*$ of the Jacobian

\[\operatorname{grad} φ = 2\operatorname{J}^* F(p, μ, λ, s)[F(p, μ, λ, s)],\]

and hence is computed with KKTVectorFieldAdjointJacobian and KKTVectorField.

For completeness, the gradient reads, using the LagrangianGradient $L = \operatorname{grad}_p \mathcal{L}(p,μ,λ) ∈ T_p\mathcal{M}$, for a shorthand of the first component of $F$, as

\[\operatorname{grad} φ = 2 \begin{pmatrix} \operatorname{Hess}_p \mathcal{L}(p,μ,λ)[L] + (g_i(p) + s_i)\operatorname{grad} g_i(p) + h_j(p)\operatorname{grad} h_j(p)\\ \Bigl( ⟨\operatorname{grad} g_i(p), L⟩ + s_i\Bigr)_{i=1}^m + μ ⊙ s ⊙ s\\ \Bigl( ⟨\operatorname{grad} h_j(p), L⟩ \Bigr)_{j=1}^n\\ g + s + μ ⊙ μ ⊙ s\end{pmatrix},\]

where $⊙$ denotes the Hadamard (or element wise) product.

Fields

• cmo the ConstrainedManifoldObjective

Constructor

KKTVectorFieldNormSqGradient(cmo::ConstrainedManifoldObjective)

Example

Define grad_f = KKTVectorFieldNormSqGradient(cmo) for some ConstrainedManifoldObjective cmo and let N be the product manifold of $\mathcal{M}×ℝ^m×ℝ^n×ℝ^m$. Then, you can call this cost as grad_f(N, q) or as the in-place variant grad_f(N, Y, q), where q is a point on N and Y is a tangent vector at q for the resulting gradient.

LagrangianCost{CO,T} <: AbstractConstrainedFunction{T}

Implement the Lagrangian of a ConstrainedManifoldObjective co.

\[\mathcal{L}(p; μ, λ) = f(p) + \sum_{i=1}^{m} μ_ig_i(p) + \sum_{j=1}^{n} λ_jh_j(p)\]

Fields

• co::CO, μ::T, λ::T as mentioned, where T represents a vector type.

Constructor

LagrangianCost(co, μ, λ)

Create a functor for the Lagrangian with fixed dual variables.

Example

When you directly want to evaluate the Lagrangian $\mathcal{L}$ you can also call

LagrangianCost(co, μ, λ)(M,p)

LagrangianGradient{CO,T} <: AbstractConstrainedFunction{T}

The gradient of the Lagrangian of a ConstrainedManifoldObjective co with respect to the variable $p$. The formula reads

\[\operatorname{grad}_p \mathcal{L}(p; μ, λ) = \operatorname{grad} f(p) + \sum_{i=1}^{m} μ_i \operatorname{grad} g_i(p) + \sum_{j=1}^{n} λ_j \operatorname{grad} h_j(p)\]

Fields

• co::CO, μ::T, λ::T as mentioned, where T represents a vector type.

Constructor

LagrangianGradient(co, μ, λ)

Create a functor for the Lagrangian with fixed dual variables.

Example

When you directly want to evaluate the gradient of the Lagrangian $\operatorname{grad}_p \mathcal{L}$ you can also call LagrangianGradient(co, μ, λ)(M,p) or LagrangianGradient(co, μ, λ)(M,X,p) for the in-place variant.

LagrangianHessian{CO, T} <: AbstractConstrainedFunction{T}

The Hessian of the Lagrangian of a ConstrainedManifoldObjective co with respect to the variable $p$. The formula reads

\[\operatorname{Hess}_p \mathcal{L}(p; μ, λ)[X] = \operatorname{Hess} f(p) + \sum_{i=1}^{m} μ_i \operatorname{Hess} g_i(p)[X] + \sum_{j=1}^{n} λ_j \operatorname{Hess} h_j(p)[X]\]

Fields

• co::CO, μ::T, λ::T as mentioned, where T represents a vector type.

Constructor

LagrangianHessian(co, μ, λ)

Create a functor for the Lagrangian with fixed dual variables.

Example

When you directly want to evaluate the Hessian of the Lagrangian $\operatorname{Hess}_p \mathcal{L}$ you can also call LagrangianHessian(co, μ, λ)(M, p, X) or LagrangianHessian(co, μ, λ)(M, Y, p, X) for the in-place variant.

LinearQuadraticHuber <: SmoothingTechnique

Specify a smoothing based on $\max\{0,x\} ≈ \mathcal{P}(x,u)$ for some $u$, where

\[\mathcal{P}(x,u) = \begin{cases} 0 & \text{ if } x ≤ 0,\\\\ \frac{x^2}{2u} & \text{ if } 0 < x ≤ u\\\\ x-\frac{u}{2} & \text{ if } x ≥ u\end{cases}\]

LinearizedDCCost

A functor (M, p) → ℝ to represent the inner problem of a ManifoldDifferenceOfConvexObjective. This is a cost function of the form

\[ F_{p_k,X_k}(p) = g(p) - ⟨X_k, \log_{p_k}p⟩\]

for a point p_k and a tangent vector X_k at p_k (for example outer iterates) that are stored within this functor as well.

Fields

• g: a function
• pk: a point on a manifold
• Xk: a tangent vector at pk

Both interim values can be set using set_parameter!(::LinearizedDCCost, ::Val{:p}, p) and set_parameter!(::LinearizedDCCost, ::Val{:X}, X), respectively.

Constructor

LinearizedDCCost(g, p, X)

LinearizedDCGrad

A functor (M, p) → X, or in-place (M, X, p) → X, to represent the gradient of the inner problem of a ManifoldDifferenceOfConvexObjective. This is a gradient function of the form

\[ F_{p_k,X_k}(p) = g(p) - ⟨X_k, \log_{p_k}p⟩\]

its gradient is given by using $F=F_1(F_2(p))$, where $F_1(X) = ⟨X_k,X⟩$ and $F_2(p) = \log_{p_k}p$ and the chain rule as well as the adjoint differential of the logarithmic map with respect to its argument for $D^*F_2(p)$

\[ \operatorname{grad} F(q) = \operatorname{grad}g(q) - DF_2^*(q)[X]\]

for a point pk and a tangent vector Xk at pk (the outer iterates) that are stored within this functor as well.

Fields

• grad_g!: the gradient of $g$ (see also LinearizedDCCost)
• pk: a point on a manifold
• Xk: a tangent vector at pk

Both interim values can be set using set_parameter!(::LinearizedDCGrad, ::Val{:p}, p) and set_parameter!(::LinearizedDCGrad, ::Val{:X}, X), respectively.

Constructor

LinearizedDCGrad(grad_g, p, X; evaluation=AllocatingEvaluation())

Where you specify whether grad_g is AllocatingEvaluation or InplaceEvaluation, while this function still provides both signatures.

LogarithmicSumOfExponentials <: SmoothingTechnique

Specify a smoothing based on $\max\{a,b\} ≈ u \log(\mathrm{e}^{\frac{a}{u}}+\mathrm{e}^{\frac{b}{u}})$ for some $u$.

ProximalDCCost

A functor (M, p) → ℝ to represent the inner cost function of a ManifoldDifferenceOfConvexProximalObjective. This is the cost function of the proximal map of g.

\[F_{p_k}(p) = \frac{1}{2λ}d_{\mathcal{M}}(p_k,p)^2 + g(p)\]

for a point pk and a proximal parameter $λ$.

Fields

• g: a function
• pk: a point on a manifold
• λ: the prox parameter

Both interim values can be set using set_parameter!(::ProximalDCCost, ::Val{:p}, p) and set_parameter!(::ProximalDCCost, ::Val{:λ}, λ), respectively.

Constructor

ProximalDCCost(g, p, λ)

ProximalDCGrad

A functor (M, p) → X, or in-place (M, X, p) → X, to represent the gradient of the inner cost function of a ManifoldDifferenceOfConvexProximalObjective. This is the gradient function of the proximal map cost function of g. Based on

\[F_{p_k}(p) = \frac{1}{2λ}d_{\mathcal{M}}(p_k,p)^2 + g(p)\]

it reads

\[\operatorname{grad} F_{p_k}(p) = \operatorname{grad} g(p) - \frac{1}{λ}\log_p p_k\]

for a point pk and a proximal parameter λ.

Fields

• grad_g!: a gradient function
• pk: a point on a manifold
• λ: the prox parameter

Both interim values can be set using set_parameter!(::ProximalDCGrad, ::Val{:p}, p) and set_parameter!(::ProximalDCGrad, ::Val{:λ}, λ), respectively.

Constructor

ProximalDCGrad(grad_g, pk, λ; evaluation=AllocatingEvaluation())

Where you specify whether grad_g is AllocatingEvaluation or InplaceEvaluation, while this function still always provides both signatures.

abstract type SmoothingTechnique

Specify a smoothing technique, see for example ExactPenaltyCost and ExactPenaltyGrad.