Constrained objectives

ConstrainedManifoldObjective{T<:AbstractEvaluationType, C<:ConstraintType} <: AbstractManifoldObjective{T}

Describes a constrained objective

\[\begin{aligned} \operatorname*{arg\,min}_{p ∈ \mathcal{M}} & f(p)\\ \text{subject to}\quad&g_i(p) ≤ 0 \quad \text{ for } i= 1, …, m,\\ \quad & h_j(p)=0 \quad \text{ for } j=1,…,n, \end{aligned}\]

Fields

• objective: an AbstractManifoldObjective representing the unconstrained objective, that is containing cost $f$, the gradient of the cost $f$ and maybe the Hessian.
• equality_constraints: an AbstractManifoldObjective representing the equality constraints

$h: \mathcal{M} → ℝ^n$ also possibly containing its gradient and/or Hessian

• inequality_constraints: an AbstractManifoldObjective representing the inequality constraints

$g: \mathcal{M} → ℝ^m$ also possibly containing its gradient and/or Hessian

Constructors

ConstrainedManifoldObjective(f, grad_f;
    g=nothing,
    grad_g=nothing,
    h=nothing,
    grad_h=nothing;
    hess_f=nothing,
    hess_g=nothing,
    hess_h=nothing,
    equality_constraints=nothing,
    inequality_constraints=nothing,
    evaluation=AllocatingEvaluation(),
    M = nothing,
    p = isnothing(M) ? nothing : rand(M),
    atol = 0,
)

Generate the constrained objective based on all involved single functions f, grad_f, g, grad_g, h, grad_h, and optionally a Hessian for each of these. With equality_constraints and inequality_constraints you have to provide the dimension of the ranges of h and g, respectively. You can also provide a manifold M and a point p to use one evaluation of the constraints to automatically try to determine these sizes.

ConstrainedManifoldObjective(mho::AbstractManifoldObjective;
    equality_constraints = nothing,
    inequality_constraints = nothing
)

Generate the constrained objective either with explicit constraints $g$ and $h$, and their gradients, or in the form where these are already encapsulated in VectorGradientFunctions.

Both variants require that at least one of the constraints (and its gradient) is provided. If any of the three parts provides a Hessian, the corresponding object, that is a ManifoldHessianObjective for f or a VectorHessianFunction for g or h, respectively, is created.

Feasibility of points with respect to the constraints is determined up to the tolerance atol.

ManifoldConstrainedSetObjective{E, MO, PF, IF} <: AbstractManifoldObjective{E}

Model a constrained objective restricted to a set

\[\operatorname*{arg\,min}_{p ∈ \mathcal{C}} f(p)\]

where $\mathcal{C} ⊂ \mathcal{M}$ is a convex closed subset.

Fields

• objective::AbstractManifoldObjective the (unconstrained) objective, which contains $f$ and for example its gradient $\operatorname{grad} f$.
• project!!::PF a projection function $\operatorname{proj}_{\mathcal{C}}: \mathcal{M} → \mathcal{C}$ that projects onto the set $\mathcal{C}$.
• indicator::IF the indicator function $ι_{\mathcal{C}}(p) = \begin{cases} 0 &\text{ for }p∈\mathcal{C}\\\\ ∞ &\text{ else.}\end{cases}$

Constructor

ManifoldConstrainedSetObjective(f, grad_f, project!!; kwargs...)

Generate the constrained objective for a given function f its gradient grad_f and a projection project!! $\operatorname{proj}_{\mathcal{C}}$.

Keyword arguments

• evaluation::AbstractEvaluationType=AllocatingEvaluation(): specify whether the functions that return an array, for example a point or a tangent vector, work by allocating its result (AllocatingEvaluation) or whether they modify their input argument to return the result therein (InplaceEvaluation). Since usually the first argument is the manifold, the modified argument is the second.
• indicator=nothing: the indicator function $ι_{\mathcal{C}}(p)$. If not provided a test, whether the projection yields the same point is performed. For the InplaceEvaluation this required one allocation.

ConstrainedManoptProblem{
    TM <: AbstractManifold,
    O <: AbstractManifoldObjective
    HR<:Union{AbstractPowerRepresentation,Nothing},
    GR<:Union{AbstractPowerRepresentation,Nothing},
    HHR<:Union{AbstractPowerRepresentation,Nothing},
    GHR<:Union{AbstractPowerRepresentation,Nothing},
} <: AbstractManoptProblem{TM}

A constrained problem might feature different ranges for the (vectors of) gradients of the equality and inequality constraints.

The ranges are required in a few places to allocate memory and access elements correctly, they work as follows:

Assume the objective is

\[\begin{aligned} \operatorname*{arg\,min}_{p ∈ \mathcal{M}} & f(p)\\ \text{subject to } & g_i(p) ≤ 0 \quad \text{ for all } i=1,…,m,\ \quad & h_j(p)=0 \quad \text{ for all } j=1,…,n. \end{aligned}\]

then the gradients can (classically) be considered as vectors of the components gradients, for example $\bigl(\operatorname{grad} g_1(p), \operatorname{grad} g_2(p), …, \operatorname{grad} g_m(p) \bigr)$.

In another interpretation, this can be considered a point on the tangent space at $P = (p,…,p) ∈ \mathcal{M}^m$, so in the tangent space to the PowerManifold $\mathcal{M}^m$. The case where this is a NestedPowerRepresentation this agrees with the interpretation from before, but on power manifolds, more efficient representations exist.

To then access the elements, the range has to be specified. That is what this problem is for.

Constructor

ConstrainedManoptProblem(
    M::AbstractManifold,
    co::ConstrainedManifoldObjective;
    range=NestedPowerRepresentation(),
    gradient_equality_range=range,
    gradient_inequality_range=range
    hessian_equality_range=range,
    hessian_inequality_range=range
)

Creates a constrained Manopt problem specifying an AbstractPowerRepresentation for both the gradient_equality_range and the gradient_inequality_range, respectively.

AbstractConstrainedFunctor{T}

A common supertype for functors that model constraint functions.

This supertype provides access for the fields $λ$ and $μ$, the dual variables of constraints of type T.

AbstractConstrainedSlackFunctor{T,R}

A common supertype for functors that model constraint functions with slack.

This supertype additionally provides access for the fields

• μ::T the dual for the inequality constraints
• s::T the slack parameter, and
• β::R the the barrier parameter

which is also of type T.

LagrangianCost{CO,T} <: AbstractConstrainedFunctor{T}

Implement the Lagrangian of a ConstrainedManifoldObjective co.

\[\mathcal{L}(p; μ, λ) = f(p) + \sum_{i=1}^{m} μ_ig_i(p) + \sum_{j=1}^{n} λ_jh_j(p)\]

Fields

• co::CO, μ::T, λ::T as mentioned, where T represents a vector type.

Constructor

LagrangianCost(co, μ, λ)

Create a functor for the Lagrangian with fixed dual variables.

Example

When you directly want to evaluate the Lagrangian $\mathcal{L}$ you can also call

LagrangianCost(co, μ, λ)(M,p)

LagrangianGradient{CO,T}

The gradient of the Lagrangian of a ConstrainedManifoldObjective co with respect to the variable $p$. The formula reads

\[\operatorname{grad}_p \mathcal{L}(p; μ, λ) = \operatorname{grad} f(p) + \sum_{i=1}^{m} μ_i \operatorname{grad} g_i(p) + \sum_{j=1}^{n} λ_j \operatorname{grad} h_j(p)\]

Fields

• co::CO, μ::T, λ::T as mentioned, where T represents a vector type.

Constructor

LagrangianGradient(co, μ, λ)

Create a functor for the Lagrangian with fixed dual variables.

Example

When you directly want to evaluate the gradient of the Lagrangian $\operatorname{grad}_p \mathcal{L}$ you can also call LagrangianGradient(co, μ, λ)(M,p) or LagrangianGradient(co, μ, λ)(M,X,p) for the in-place variant.

LagrangianHessian{CO, V, T}

The Hessian of the Lagrangian of a ConstrainedManifoldObjective co with respect to the variable $p$. The formula reads

\[\operatorname{Hess}_p \mathcal{L}(p; μ, λ)[X] = \operatorname{Hess} f(p) + \sum_{i=1}^{m} μ_i \operatorname{Hess} g_i(p)[X] + \sum_{j=1}^{n} λ_j \operatorname{Hess} h_j(p)[X]\]

Fields

• co::CO, μ::T, λ::T as mentioned, where T represents a vector type.

Constructor

LagrangianHessian(co, μ, λ)

Create a functor for the Lagrangian with fixed dual variables.

Example

When you directly want to evaluate the Hessian of the Lagrangian $\operatorname{Hess}_p \mathcal{L}$ you can also call LagrangianHessian(co, μ, λ)(M, p, X) or LagrangianHessian(co, μ, λ)(M, Y, p, X) for the in-place variant.

equality_constraints_length(co::ConstrainedManifoldObjective)

Return the number of equality constraints of an ConstrainedManifoldObjective. This acts transparently through AbstractDecoratedManifoldObjectives

inequality_constraints_length(cmo::ConstrainedManifoldObjective)

Return the number of inequality constraints of an ConstrainedManifoldObjective cmo. This acts transparently through AbstractDecoratedManifoldObjectives

get_equality_constraint(amp::AbstractManoptProblem, p, j=:)
get_equality_constraint(M::AbstractManifold, objective, p, j=:)

Evaluate equality constraints of a ConstrainedManifoldObjective objective at point p and indices j (by default : which corresponds to all indices).

get_grad_equality_constraint(amp::AbstractManoptProblem, p, j)
get_grad_equality_constraint(M::AbstractManifold, co::ConstrainedManifoldObjective, p, j, range=NestedPowerRepresentation())
get_grad_equality_constraint!(amp::AbstractManoptProblem, X, p, j)
get_grad_equality_constraint!(M::AbstractManifold, X, co::ConstrainedManifoldObjective, p, j, range=NestedPowerRepresentation())

Evaluate the gradient or gradients of the equality constraint $(\operatorname{grad} h(p))_j$ or $\operatorname{grad} h_j(p)$,

See also the ConstrainedManoptProblem to specify the range of the gradient.

get_grad_inequality_constraint(amp::AbstractManoptProblem, p, j=:)
get_grad_inequality_constraint(M::AbstractManifold, co::ConstrainedManifoldObjective, p, j=:, range=NestedPowerRepresentation())
get_grad_inequality_constraint!(amp::AbstractManoptProblem, X, p, j=:)
get_grad_inequality_constraint!(M::AbstractManifold, X, co::ConstrainedManifoldObjective, p, j=:, range=NestedPowerRepresentation())

Evaluate the gradient or gradients of the inequality constraint $(\operatorname{grad} g(p))_j$ or $\operatorname{grad} g_j(p)$,

See also the ConstrainedManoptProblem to specify the range of the gradient.

get_hess_equality_constraint(amp::AbstractManoptProblem, p, j=:)
get_hess_equality_constraint(M::AbstractManifold, co::ConstrainedManifoldObjective, p, j, range=NestedPowerRepresentation())
get_hess_equality_constraint!(amp::AbstractManoptProblem, X, p, j=:)
get_hess_equality_constraint!(M::AbstractManifold, X, co::ConstrainedManifoldObjective, p, j, range=NestedPowerRepresentation())

Evaluate the Hessian or Hessians of the equality constraint $(\operatorname{Hess} h(p))_j$ or $\operatorname{Hess} h_j(p)$,

See also the ConstrainedManoptProblem to specify the range of the Hessian.

get_hess_inequality_constraint(amp::AbstractManoptProblem, p, X, j=:)
get_hess_inequality_constraint(M::AbstractManifold, co::ConstrainedManifoldObjective, p, j=:, range=NestedPowerRepresentation())
get_hess_inequality_constraint!(amp::AbstractManoptProblem, Y, p, j=:)
get_hess_inequality_constraint!(M::AbstractManifold, Y, co::ConstrainedManifoldObjective, p, X, j=:, range=NestedPowerRepresentation())

Evaluate the Hessian or Hessians of the inequality constraint $(\operatorname{Hess} g(p)[X])_j$ or $\operatorname{Hess} g_j(p)[X]$,

See also the ConstrainedManoptProblem to specify the range of the Hessian.

get_inequality_constraint(amp::AbstractManoptProblem, p, j=:)
get_inequality_constraint(M::AbstractManifold, co::ConstrainedManifoldObjective, p, j=:, range=NestedPowerRepresentation())

Evaluate inequality constraints of a ConstrainedManifoldObjective objective at point p and indices j (by default : which corresponds to all indices).

get_projected_point(amp::AbstractManoptProblem, p)
get_projected_point!(amp::AbstractManoptProblem, q, p)
get_projected_point(M::AbstractManifold, cso::ManifoldConstrainedSetObjective, p)
get_projected_point!(M::AbstractManifold, q, cso::ManifoldConstrainedSetObjective, p)

Project p with the projection that is stored within the ManifoldConstrainedSetObjective. This can be done in-place of q.

get_projected_point(amp::AbstractManoptProblem, p)
get_projected_point!(amp::AbstractManoptProblem, q, p)
get_projected_point(M::AbstractManifold, cso::ManifoldConstrainedSetObjective, p)
get_projected_point!(M::AbstractManifold, q, cso::ManifoldConstrainedSetObjective, p)

Project p with the projection that is stored within the ManifoldConstrainedSetObjective. This can be done in-place of q.

get_projected_point(amp::AbstractManoptProblem, p)
get_projected_point!(amp::AbstractManoptProblem, q, p)
get_projected_point(M::AbstractManifold, cso::ManifoldConstrainedSetObjective, p)
get_projected_point!(M::AbstractManifold, q, cso::ManifoldConstrainedSetObjective, p)

Project p with the projection that is stored within the ManifoldConstrainedSetObjective. This can be done in-place of q.

get_projected_point(amp::AbstractManoptProblem, p)
get_projected_point!(amp::AbstractManoptProblem, q, p)
get_projected_point(M::AbstractManifold, cso::ManifoldConstrainedSetObjective, p)
get_projected_point!(M::AbstractManifold, q, cso::ManifoldConstrainedSetObjective, p)

Project p with the projection that is stored within the ManifoldConstrainedSetObjective. This can be done in-place of q.

get_unconstrained_objective(co::ConstrainedManifoldObjective)

Returns the internally stored unconstrained AbstractManifoldObjective within the ConstrainedManifoldObjective.

is_feasible(M::AbstractManifold, cmo::ConstrainedManifoldObjective, p, kwargs...)
is_feasible(M::AbstractManifold, o::AbstractDecoratedManifoldObjective, p, kwargs...)

Evaluate whether a point p on M is feasible with respect to the ConstrainedManifoldObjective cmo. That is for the provided inequality constraints $g: \mathcal{M} → ℝ^m$ and equality constraints $h: \mathcal{M} → ℝ^m$ from within cmo, the point $p ∈ \mathcal{M}$ is feasible if

\[g_i(p) ≤ 0, \text{ for all } i=1,…,m\quad ext{ and }\quad h_j(p) = 0, ext{ for all } j=1,…,n.\]

Keyword arguments

• check_point::Bool=true: whether to also verify that `p∈\mathcal{M} holds, using is_point
• error::Symbol=:none: if the point is not feasible, this symbol determines how to report the error.
  • :error: throws an error
  • :info: displays the error message as an @info
  • :none: (default) the function just returns true/false
  • :warn: displays the error message as a @warning.

The keyword error= and all other kwargs... are passed on to is_point if the point is verified (see check_point).

get_feasibility_status(
    M::AbstractManifold,
    cmo::ConstrainedManifoldObjective,
    g = get_inequality_constraints(M, cmo, p),
    h = get_equality_constraints(M, cmo, p),
)

Generate a message about the feasibiliy of p with respect to the ConstrainedManifoldObjective. You can also provide the evaluated vectors for the values of g and h as keyword arguments, in case you had them evaluated before.