Plans for solvers

For any optimisation performed in Manopt.jl information is required about both the optimisation task or “problem” at hand as well as the solver and all its parameters. This together is called a plan in Manopt.jl and it consists of two data structures:

The Manopt Problem describes all static data of a task, most prominently the manifold and the objective.
The Solver State describes all varying data and parameters for the solver that is used. This also means that each solver has its own data structure for the state.

By splitting these two parts, one problem can be define an then be solved using different solvers.

Still there might be the need to set certain parameters within any of these structures. For that there is

Manopt.set_parameter! — Function

set_parameter!(f, element::Symbol , args...)

For any f and a Symbol e, dispatch on its value so by default, to set some args... in f or one of uts sub elements.

set_parameter!(element::Symbol, value::Union{String,Bool,<:Number})

Set global Manopt parameters addressed by a symbol element. W This first dispatches on the value of element.

The parameters are stored to the global settings using Preferences.jl.

Passing a value of "" deletes the corresponding entry from the preferences. Whenever the LocalPreferences.toml is modified, this is also issued as an @info.

set_parameter!(amo::AbstractManifoldObjective, element::Symbol, args...)

Set a certain args... from the AbstractManifoldObjective amo to value. This function should dispatch onVal(element)`.

Currently supported

:Cost passes to the get_cost_function
:Gradient passes to the get_gradient_function
:SubGradient passes to the get_subgradient_function

set_parameter!(ams::AbstractManoptProblem, element::Symbol, field::Symbol , value)

Set a certain field/element from the AbstractManoptProblem ams to value. This function usually dispatches on Val(element). Instead of a single field, also a chain of elements can be provided, allowing to access encapsulated parts of the problem.

Main values for element are :Manifold and :Objective.

set_parameter!(ams::DebugSolverState, ::Val{:Debug}, args...)

Set certain values specified by args... into the elements of the debugDictionary

set_parameter!(ams::RecordSolverState, ::Val{:Record}, args...)

Set certain values specified by args... into the elements of the recordDictionary

set_parameter!(c::StopAfter, :MaxTime, v::Period)

Update the time period after which an algorithm shall stop.

set_parameter!(c::StopAfterIteration, :;MaxIteration, v::Int)

Update the number of iterations after which the algorithm should stop.

set_parameter!(c::StopWhenChangeLess, :MinIterateChange, v::Int)

Update the minimal change below which an algorithm shall stop.

set_parameter!(c::StopWhenCostLess, :MinCost, v)

Update the minimal cost below which the algorithm shall stop

set_parameter!(c::StopWhenEntryChangeLess, :Threshold, v)

Update the minimal cost below which the algorithm shall stop

set_parameter!(c::StopWhenGradientChangeLess, :MinGradientChange, v)

Update the minimal change below which an algorithm shall stop.

set_parameter!(c::StopWhenGradientNormLess, :MinGradNorm, v::Float64)

Update the minimal gradient norm when an algorithm shall stop

set_parameter!(c::StopWhenStepsizeLess, :MinStepsize, v)

Update the minimal step size below which the algorithm shall stop

set_parameter!(c::StopWhenSubgradientNormLess, :MinSubgradNorm, v::Float64)

Update the minimal subgradient norm when an algorithm shall stop

set_parameter!(ams::AbstractManoptSolverState, element::Symbol, args...)

Set a certain field or semantic element from the AbstractManoptSolverState ams to value. This function passes to Val(element) and specific setters should dispatch on Val{element}.

By default, this function just does nothing.

set_parameter!(ams::DebugSolverState, ::Val{:SubProblem}, args...)

Set certain values specified by args... to the sub problem.

set_parameter!(ams::DebugSolverState, ::Val{:SubState}, args...)

Set certain values specified by args... to the sub state.

set_parameter!(c::StopWhenResidualIsReducedByFactorOrPower, :ResidualPower, v)

Update the residual Power θ to v.

set_parameter!(c::StopWhenResidualIsReducedByFactorOrPower, :ResidualFactor, v)

Update the residual Factor κ to v.

Manopt.get_parameter — Function

get_parameter(f, element::Symbol, args...)

Access arbitrary parameters from f addressed by a symbol element.

For any f and a Symbol e dispatch on its value by default, to get some element from f potentially further qualified by args....

This functions returns nothing if f does not have the property element

get_parameter(element::Symbol; default=nothing)

Access global Manopt parameters addressed by a symbol element. This first dispatches on the value of element.

If the value is not set, default is returned.

The parameters are queried from the global settings using Preferences.jl, so they are persistent within your activated Environment, see also set_parameter!.

Currently used settings

:Mode the mode can be set to "Tutorial" to get several hints especially in scenarios, where the optimisation on manifolds is different from the usual “experience” in (classical, Euclidean) optimization. Any other value has the same effect as not setting it.

:KeywordsErrorMode specify how to handle the case when unknown keywords are passed to a solver. Since solvers often pass their keywords on to internal structures, to e.g. decorate the objective or the state, checking keywords has its own method in Manopt.jl. This parameter specifies how to handle the case where unknown keywords are handled.

"none" does not report and the keyword gets just ignored
"warn" issues a warning (default)
"error" throw a ManoptKeywordError

all other symbol values are treated the same as :none.

Manopt.status_summary — Function

status_summary(e)

Return a string reporting about the current status of e, where e is a type from Manopt.

This method is similar to show but just returns a string. It might also be more verbose in explaining, or hide internal information.

The following symbols are used.

Symbol	Used in	Description
`:Activity`	`DebugWhenActive`	activity of the debug action stored within
`:Basepoint`	`TangentSpace`	the point the tangent space is at
`:Cost`	generic	the cost function (within an objective, as pass down)
`:Debug`	`DebugSolverState`	the stored `debugDictionary`
`:Gradient`	generic	the gradient function (within an objective, as pass down)
`:Iterate`	generic	the (current) iterate, similar to `set_iterate!`, within a state
`:Manifold`	generic	the manifold (within a problem, as pass down)
`:Objective`	generic	the objective (within a problem, as pass down)
`:SubProblem`	generic	the sub problem (within a state, as pass down)
`:SubState`	generic	the sub state (within a state, as pass down)
`:λ`	`ProximalDCCost`, `ProximalDCGrad`	set the proximal parameter within the proximal sub objective elements
`:Population`	`ParticleSwarmState`	a certain population of points, for example `particle_swarm`s swarm
`:Record`	`RecordSolverState`
`:TrustRegionRadius`	`TrustRegionsState`	the trust region radius, equivalent to `:σ`
`:ρ`, `:u`	`ExactPenaltyCost`, `ExactPenaltyGrad`	Parameters within the exact penalty objective
`:ρ`, `:μ`, `:λ`	`AugmentedLagrangianCost`, `AugmentedLagrangianGrad`	Parameters of the Lagrangian function
`:p`, `:X`	`LinearizedDCCost`, `LinearizedDCGrad`	Parameters withing the linearized functional used for the sub problem of the difference of convex algorithm

Any other lower case name or letter as well as single upper case letters access fields of the corresponding first argument. for example :p could be used to access the field s.p of a state. This is often, where the iterate is stored, so the recommended way is to use :Iterate from before.

Since the iterate is often stored in the states fields s.p one could access the iterate often also with :p and similarly the gradient with :X. This is discouraged for both readability as well as to stay more generic, and it is recommended to use :Iterate and :Gradient instead in generic settings.

You can further activate a “Tutorial” mode by set_parameter!(:Mode, "Tutorial"). Internally, the following convenience function is available.

Manopt.is_tutorial_mode — Function

is_tutorial_mode()

A small internal helper to indicate whether tutorial mode is active.

You can set the mode by calling set_parameter!(:Mode, "Tutorial") or deactivate it by set_parameter!(:Mode, "").

A factory for providing manifold defaults

In several cases a manifold might not yet be known at the time a (keyword) argument should be provided. Therefore, any type with a manifold default can be wrapped into a factory.

Manopt.ManifoldDefaultsFactory — Type

ManifoldDefaultsFactory{M,T,A,K}

A generic factory to postpone the instantiation of certain types from within Manopt.jl, in order to be able to adapt it to defaults from different manifolds and/or postpone the decission on which manifold to use to a later point

For now this is established for

This factory stores necessary and optional parameters as well as keyword arguments provided by the user to later produce the type this factory is for.

Besides a manifold as a fallback, the factory can also be used for the (maybe simpler) types from the list of types that do not require the manifold.

Fields

M::Union{Nothing,AbstractManifold}: provide a manifold for defaults
args::A: arguments (args...) that are passed to the type constructor
kwargs::K: keyword arguments (kwargs...) that are passed to the type constructor
constructor_requires_manifold::Bool: indicate whether the type construtor requires the manifold or not

Constructor

ManifoldDefaultsFactory(T, args...; kwargs...)
ManifoldDefaultsFactory(T, M, args...; kwargs...)

Input

T a subtype of types listed above that this factory is to produce
M (optional) a manifold used for the defaults in case no manifold is provided.
args... arguments to pass to the constructor of T
kwargs... keyword arguments to pass (overwrite) when constructing T.

Keyword arguments

requires_manifold=true: indicate whether the type constructor this factory wraps requires the manifold as first argument or not.

All other keyword arguments are internally stored to be used in the type constructor

as well as arguments and keyword arguments for the update rule.

see also

_produce_type

Manopt._produce_type — Function

_produce_type(t::T, M::AbstractManifold)
_produce_type(t::ManifoldDefaultsFactory{T}, M::AbstractManifold)

Use the ManifoldDefaultsFactory{T} to produce an instance of type T. This acts transparent in the way that if you provide an instance t::T already, this will just be returned.

Keyword arguments and their verification

Internally Manopt.jl passes keywords for the (high-level) solver functions to several inner functions, e.g. to add debug or caching. Besides the documentation, one can check with the internal function Manopt.accepted_keywords which keywords a solver accepts.

A solver also warns, if a keyword is passed, that is not handled by the solver or any of the inner functions it calls.

Manopt.Keywords — Type

Keywords

A small internal struct to represent a set of keywords,

Fields

accepted=Set{Symbol}() a Set of symbols of keywords a certain function accepts
deprecated=Set{Symbol}() a Set of symbols of keywords a certain function has deprecated
from=nothing the function the keywords are (directly or indirectly) come from or accepted in. to indicate that these are not associated with a certain function, use nothing. to Indicate ane empty set, use nothing.
origins a dictionary that specifies for every keyword the function it is passed to. this usually should point to the function it is directly passed to.

Constructor

Keywords(
    accepted=Set{Symbol}(), deprecated=Set{Symbol}();
    from::Type=nothing)

Generate a Keywords wrapper, where both default to being the empty set. For pretty printing you can provide a type they belong to.

Manopt.accepted_keywords — Function

accepted_keywords(problem)
accepted_keywords(objective)
accepted_keywords(solver)
accepted_keywords(stepsize)

Return a set of keywords, see Keywords, a certain element of Manopt.jl accepts when constructed.

This function uses direct_keywords to find keywords a function directly accepts, and calls_with_kwargs to find functions it passes keyword to, where they also might be accepted. In order for nonmutating functions f to work the same as their mutating variants f!, the allocating one, one should set calls_with_kwargs(f) = (f,).

this also includes keywords that are passed on to internal structures, also specified using calls_with_kwargs.

Manopt.ManoptKeywordError — Type

ManoptKeywordError <: Exception

An error to indicate that a certain function received keywords it does not accept.

Fields

f the function that received the keywords
kw::Keywords the keywords that were not accepted

Constructor

ManoptKeywordError(f, kw::Keywords)

Manopt.add! — Function

add!(kw::Keywords, kw2::Keywords)

Append the Keywords kw2 to kw, i.e. union the accepted and deprecated keywords, as well as their origins, but keep first parameter of kw. Also their origin takes precedence.

Manopt.calls_with_kwargs — Function

calls_with_kwargs(f)

Return a tuple of functions f calls and passes its kwargs... to.

Manopt.direct_keywords — Function

direct_keywords(problem)
direct_keywords(objective)
direct_keywords(solver)
direct_keywords(stepsize)

Return a set of keywords a function directly would work with.

Manopt.keywords_accepted — Function

keywords_accepted(f, mode=:warn, kw::Keywords=accepted_keywords(f); kwargs...)

Given a function f, Keywords kw it accepts, check if kwargs... are accepted by those keywords and warn if deprecated keywords are passed

For keywords that are not accepted/processed here, the mode argument provides how to report the result, either :warn or :error on keywords that are not accepted.