Common Debug Output

DebugCallback <: DebugAction

Debug for a simple callback function, mainly for compatibility to other solvers and if a user already has a callback function or functor available

The expected format of the callback is that of a function with signature (problem, state, k) -> nothing. A simple callback of the signature () -> nothing can be specified by simple=true. In this case the callback is wrapped in a function of the generic form above.

Constructor

DebugCallback(callback; simple=false)

DebugChange(M=DefaultManifold(); kwargs...)

debug for the amount of change of the iterate (stored in get_iterate of the AbstractManoptSolverState) during the last iteration. See DebugEntryChange for the general case

Keyword parameters

• storage=StoreStateAction( [:Iterate] ): storage of the previous iterate
• prefix="Last Change: ": prefix of the debug output (ignored if you set format)
• io=stdout: default stream to print the debug to.
• format="$(prefix)%f": format to print the output
• inverse_retraction_method::AbstractInverseRetractionMethod=default_inverse_retraction_method(M): an inverse retraction $\operatorname{retr}^{-1}$ to use, see the section on retractions and their inverses

DebugCost <: DebugAction

print the current cost function value, see get_cost.

Constructors

DebugCost()

Parameters

• format="f(x): %f": format to print the output ("current cost: %f" when long=true)
• io=stdout: default stream to print the debug to.
• long=false: short form to set the format to f(x): (default) or current cost: and the cost
• at_init=true: whether to print also at initialization

DebugDivider <: DebugAction

print a small divider (default " | ").

Constructor

DebugDivider(divider=" | "; io=stdout, at_init=true)

DebugDualChange(; kwargs...)
DebugDualChange((X, n); kwargs...)

Print the change of the dual variable.

This is similar to DebugChange (see their constructors for details), but uses a different calculation of the change, since the dual variable lives in (possibly different) tangent spaces.

DebugDualResidual <: DebugAction

A Debug action to print the dual residual. The constructor accepts some (shared) storage or a tuple (p, X, n) of initial values, which should at least record :Iterate, :X and :n.

Constructor

DebugDualResidual(; kwargs...)

Keyword arguments

• io=stdout: stream to perform the debug to
• format="$prefix%s": format to print the dual residual, using the prefix by default
• prefix="Dual Residual: ": short form to just set the prefix
• storage (a new StoreStateAction) to store values for the debug.

DebugEntry <: DebugAction

print a certain fields entry during the iterates, where a format can be specified how to print the entry.

Additional fields

• field: symbol the entry can be accessed with within AbstractManoptSolverState
• at_init: whether to print also at initialization

Constructor

DebugEntry(f; prefix="$f:", format = "$prefix %s", io=stdout, at_init=true)

DebugEntryChange{T} <: DebugAction

print a certain entries change during iterates

Additional fields

• io: an IO stream for the debug
• format: format to print (uses the prefix by default)
• field: Symbol the field can be accessed with within AbstractManoptSolverState
• distance: function (problem, state, x1, x2) to compute the change/distance between two values of the entry
• storage: a StoreStateAction to store the previous value of :f

Constructors

DebugEntryChange(f,d)

Keyword arguments

• io=stdout: an IOStream used for the debug
• prefix="Change of $f:": the prefix
• storage=StoreStateAction([f]): a StoreStateAction
• initial_value=NaN: an initial value for the change of the entry f.
• format="$prefix%s": format to print the change

DebugFeasibility <: DebugAction

Display information about the feasibility of the current iterate

Fields

• format: a vector of symbols and string formatting the output
• io: default stream to print the debug to.
• at_init: whether to print also at initialization

The following symbols are filled with values

• :Feasible display true or false depending on whether the iterate is feasible
• :FeasibleEq display = or ≠ equality constraints are fulfilled or not
• :FeasibleIneq display ≤ or ≰ inequality constraints are fulfilled or not
• :NumEq display the number of equality constraints infeasible
• :NumEqNz display the number of equality constraints infeasible if exists
• :NumIneq display the number of inequality constraints infeasible
• :NumIneqNz display the number of inequality constraints infeasible if exists
• :TotalEq display the sum of how much the equality constraints are violated
• :TotalInEq display the sum of how much the inequality constraints are violated

format to print the output.

Constructor

DebugFeasibility(    format=["feasible: ", :Feasible];    io::IO=stdout,    at_init::Bool=true,)

DebugGradient <: DebugAction

debug for the gradient evaluated at the current iterate

Constructors

DebugGradient(; long=false, prefix=long ? "Gradient: " : "grad f(p):", format="$prefix%s", io=stdout, at_init=false)

display the short (false) or long (true) default text for the gradient, or set the prefix manually. Alternatively the complete format can be set.

DebugGradientChange(M::AbstractManifold=DefaultManifold(); kwargs...)

debug for the amount of change of the gradient (stored in get_gradient of the AbstractManoptSolverState) during the last iteration. See DebugEntryChange for the general case

Keyword parameters

• storage=StoreStateAction( [:Iterate, :Gradient] ): storage of the action for previous data
• prefix="Last Change: ": prefix of the debug output (ignored if you set format)
• io=stdout: default stream to print the debug to.
• format="$(prefix)%f": format to print the output
• vector_transport_method::AbstractVectorTransportMethod=default_vector_transport_method(M, typeof(p)): a vector transport $\mathcal T_{⋅←⋅}$ to use, see the section on vector transports

DebugGradientNorm <: DebugAction

debug for the norm of the gradient evaluated at the current iterate.

Constructors

DebugGradientNorm(; long=false, prefix=long ? "Norm of the Gradient: " : "|grad f(p)|:", format="$prefix%s", io=stdout, at_init=true)

display the short (false) or long (true) default text for the gradient norm.

DebugIfEntry <: DebugAction

Issue a warning, info, or error if a certain field does not pass a the check.

The message is printed in this case. If it contains a @printf argument identifier, that one is filled with the value of the field. That way you can print the value in this case as well.

Fields

• io: an IO stream
• check: a function that takes the value of the field as input and returns a boolean
• field: symbol the entry can be accessed with within AbstractManoptSolverState
• msg: if the check fails, this message is displayed
• type: symbol specifying the type of display, possible values :print, :warn, :info, :error, where :print prints to io.
• at_init: whether to print also at initialization

Constructor

DebugIfEntry(field, check=(>(0)); type=:warn, message=":$f nonpositive.", io=stdout, at_init=true)

DebugIterate <: DebugAction

debug for the current iterate (stored in get_iterate of the AbstractManoptSolverState).

Constructor

DebugIterate(; kwargs...)

Keyword arguments

• io=stdout: default stream to print the debug to.
• format="$prefix %s": format how to print the current iterate
• long=false: whether to have a long ("current iterate:") or a short ("p:") prefix default
• prefix: (see long for default) set a prefix to be printed before the iterate
• at_init=false: whether to print also at initialization

DebugIteration <: DebugAction

Constructor

DebugIteration()

Keyword parameters

• format="# %-6d": format to print the output
• io=stdout: default stream to print the debug to.

debug for the current iteration, prefixed with # by the default format, and printed as Initial at initialization.

DebugMessages <: DebugAction

An AbstractManoptSolverState or one of its sub steps like a Stepsize might generate warnings throughout their computations. This debug can be used to :Print them or to display them as :Info, :Warning or even :Error, depending on the message type.

Constructor

DebugMessages(mode=:Info, warn=:Once; io::IO=stdout)

Initialize the messages debug to a certain mode. Available modes are

• :Error: issue the messages as an error and hence stop at any issue occurring
• :Info: issue the messages as an @info
• :Print: print messages to the stream io.
• :Warning: issue the messages as a warning

The warn level can be set to :Once to only display the first message, to :Always to report every message, one can set it to :No, to deactivate this, then this DebugAction is inactive. All other symbols are handled as if they were :Always

DebugPrimalDualResidual <: DebugAction

A Debug action to print the primal dual residual. The constructor accepts some (shared) storage or a tuple (p, X, n) of initial values, which should at least record :Iterate, :X and :n.

Constructor

DebugPrimalDualResidual()

with the keywords

Keyword arguments

• io=stdout: stream to perform the debug to
• format="$prefix%s": format to print the primal dual residual, using the prefix by default
• prefix="PD Residual: ": short form to just set the prefix
• storage (a new StoreStateAction) to store values for the debug.

DebugPrimalResidual <: DebugAction

A Debug action to print the primal residual. The constructor accepts some (shared) storage or a tuple (p, X, n) of initial values, which should at least record :Iterate, :X and :n.

Constructor

DebugPrimalResidual(; kwargs...)

Keyword arguments

• io=stdout: stream to perform the debug to
• format="$prefix%s": format to print the primal residual, using the prefix by default
• prefix="Primal Residual: ": short form to just set the prefix
• storage (a new StoreStateAction) to store values for the debug.

DebugProximalParameter <: DebugAction

print the current proximal point algorithm parameter stored in the field λ of the AbstractManoptSolverState.

DebugStepsize <: DebugAction

debug for the current step size.

Constructors

DebugStepsize(; long=false, prefix=long ? "step size:" : "s:", format="$prefix%s", io=stdout, at_init=true)

display the a prefix in front of the step size.

DebugStoppingCriterion <: DebugAction

print the Reason provided by the stopping criterion. Usually this should be empty, unless the algorithm stops.

Fields

• prefix="": a prefix printed before the reason the solver stopped
• io=stdout: default stream to print the debug to.

Constructor

DebugStoppingCriterion(prefix = ""; io::IO=stdout)

DebugTime()

Measure time and print the intervals. Using start=true you can start the timer on construction, for example to measure the runtime of an algorithm overall (adding)

The measured time is rounded using the given time_accuracy and printed after canonicalization.

Keyword parameters

• io=stdout: default stream to print the debug to.
• format="$prefix %s": format to print the output, where %s is the canonicalized time.
• mode=:Cumulative: whether to display the total time or reset on every call using :Iterative.
• prefix="time spent:": prefix of the debug output (ignored if you set format)
• start=false: indicate whether to start the timer on creation or not. Otherwise it might only be started on first call.
• time_accuracy=Millisecond(1): round the time to this period before printing the canonicalized time

DebugWarnIfCostIncreases <: DebugAction

print a warning if the cost increases.

Note that this provides an additional warning for gradient descent with its default constant step size.

Constructor

DebugWarnIfCostIncreases(warn=:Once; tol=1e-13)

Initialize the warning to warning level (:Once) and introduce a tolerance for the test of 1e-13.

The warn level can be set to :Once to only warn the first time the cost increases, to :Always to report an increase every time it happens, and it can be set to :No to deactivate the warning, then this DebugAction is inactive. All other symbols are handled as if they were :Always

DebugWarnIfCostNotFinite <: DebugAction

A debug to see when the cost is not finite, for example Inf or NaN.

Constructor

DebugWarnIfCostNotFinite(warn=:Once)

Initialize the warning to warn :Once.

This can be set to :Once to only warn the first time the cost is NaN. It can also be set to :No to deactivate the warning, but this makes this Action also useless. All other symbols are handled as if they were :Always

DebugWarnIfFieldNotFinite <: DebugAction

A debug to see when a field of the solver state is not finite, for example Inf or NaN

Constructor

DebugWarnIfFieldNotFinite(field::Symbol, warn=:Once)

Initialize the warning to warn :Once.

This can be set to :Once to only warn the first time the field is not finite. It can also be set to :No to deactivate the warning, but this makes this Action also useless. All other symbols are handled as if they were :Always

Example

DebugWarnIfFieldNotFinite(:Gradient)

Creates a DebugAction to track whether the gradient does not get NaN or Inf.

DebugWarnIfGradientNormTooLarge{T} <: DebugAction

A debug to warn when an evaluated gradient at the current iterate is larger than (a factor times) the maximal (recommended) stepsize at the current iterate.

Constructor

DebugWarnIfGradientNormTooLarge(factor::T=1.0, warn=:Once)

Initialize the warning to warn :Once.

This can be set to :Once to only warn the first time the gradient norm is too large. It can also be set to :No to deactivate the warning, but this makes this Action also useless. All other symbols are handled as if they were :Always

Example

DebugWarnIfGradientNormTooLarge(2.0)

Creates a DebugAction warning when the gradient norm exceeds twice the maximal stepsize at the current iterate.

DebugWarnIfLagrangeMultiplierIncreases <: DebugAction

print a warning if the Lagrange parameter based value $-ξ$ of the bundle method increases.

Constructor

DebugWarnIfLagrangeMultiplierIncreases(warn=:Once; tol=1e2)

Initialize the warning to warning level (:Once) and introduce a tolerance for the test of 1e2.

The warn level can be set to :Once to only warn the first time the multiplier increases, to :Always to report an increase every time it happens, and it can be set to :No to deactivate the warning, then this DebugAction is inactive. All other symbols are handled as if they were :Always.

DebugWarnIfStepsizeCollapsed <: DebugAction

print a warning if the backtracking stopped because the stepsize fell below a given threshold. This threshold is specified by the stop_when_stepsize_less field.

Constructor

DebugWarnIfStepsizeCollapsed(tol::T=1e-8, warn=:Once)

Initialize the warning to warning level (:Once) with a tolerance for stop_when_stepsize_less set to tol (1e-8).

The warn level can be set to :Once to only warn the first time the step size collapses, to :Always to report a collapse every time it happens, and it can be set to :No to deactivate the warning, then this DebugAction is inactive. All other symbols are handled as if they were :Always

DebugWhenActive <: DebugAction

evaluate and print debug only if the active boolean is set. This can be set from outside and is for example triggered by DebugEvery on debugs on the subsolver.

This method does not perform any print itself but relies on its children's prints.

For now, the main interaction is with DebugEvery which might activate or deactivate this debug

Fields

• active: a boolean that can (de-)activated from outside to turn on/off debug
• always_update: whether or not to call the other debugs with negative iteration numbers (k < 0) while inactive

Constructor

DebugWhenActive(d::DebugAction, active=true, always_update=true)

DebugDualBaseChange(; storage=StoreStateAction([:n]), kwargs...)

Print the change of the dual base variable by using DebugEntryChange, see their constructors for detail, on the field n of the state.

DebugDualBaseIterate(; kwargs...)

Print the dual base variable by using DebugEntry, see their constructors for detail. This method is further set to display the field n of the state.

DebugDualIterate(; kwargs...)

Print the dual variable by using DebugEntry, see their constructors for detail. This method is further set to display the field X of the state.

DebugPrimalBaseChange(opts...; prefix="Primal Base Change:", kwargs...)

Print the change of the primal base variable by using DebugEntryChange, see their constructors for detail, on the field m of the state.

DebugPrimalBaseIterate()

Print the primal base variable by using DebugEntry, see their constructors for detail. This method is further set to display the field m of the state.

DebugPrimalChange(; storage=StoreStateAction([:Iterate]), prefix="Primal Change: ", kwargs...)

Print the change of the primal variable by using DebugChange, see their constructors for detail.

DebugPrimalIterate(opts...;kwargs...)

Print the primal variable by using DebugIterate, see their constructors for detail.

DebugActionFactory(s)

create a DebugAction where

• a String yields the corresponding divider
• a DebugAction is passed through
• a Symbol creates a DebugEntry of that symbol, with the exceptions listed in DebugActionFactory.
• a Tuple{Symbol,String} creates a DebugEntry of that symbol where the String specifies the format.
• a <:Function creates a DebugCallback with the function as callback.

DebugActionFactory(s::Symbol)

Convert certain Symbols in the debug=[ ... ] vector to DebugActions Currently the following ones are done. Note that the Shortcut symbols should all start with a capital letter.

• :Cost creates a DebugCost
• :Change creates a DebugChange
• :Feasibility creates a DebugFeasibility
• :Gradient creates a DebugGradient
• :GradientChange creates a DebugGradientChange
• :GradientNorm creates a DebugGradientNorm
• :Iterate creates a DebugIterate
• :Iteration creates a DebugIteration
• :IterativeTime creates a DebugTime(; mode=:Iterative)
• :ProxParameter creates a DebugProximalParameter()
• :Stepsize creates a DebugStepsize
• :Stop creates a DebugStoppingCriterion()
• :Time creates a DebugTime
• :WarnStepsize creates a DebugWarnIfStepsizeCollapsed
• :WarnBundle creates a DebugWarnIfLagrangeMultiplierIncreases
• :WarnCost creates a DebugWarnIfCostNotFinite
• :WarnGradient creates a DebugWarnIfFieldNotFinite for the :Gradient.
• :WarningMessages creates a DebugMessages(:Warning)
• :InfoMessages creates a DebugMessages(:Info)
• :ErrorMessages creates a DebugMessages(:Error)
• :Messages creates a DebugMessages() (the same as :InfoMessages)

any other symbol s creates a DebugEntry(s) to print that field of the solver state.

DebugActionFactory(t::Tuple{Symbol,String})

Convert certain Symbols in the debug=[ ... ] vector to DebugActions Currently the following ones are done, where the string in t[2] is passed as the format of the corresponding debug. Note that the Shortcut symbols t[1] should all start with a capital letter.

• :Change creates a DebugChange
• :Cost creates a DebugCost
• :Feasibility creates a DebugFeasibility
• :Gradient creates a DebugGradient
• :GradientChange creates a DebugGradientChange
• :GradientNorm creates a DebugGradientNorm
• :Iterate creates a DebugIterate
• :Iteration creates a DebugIteration
• :ProxParameter creates a DebugProximalParameter
• :Stepsize creates a DebugStepsize
• :Stop creates a DebugStoppingCriterion, where t[2] is used as its prefix and not as a format
• :Time creates a DebugTime
• :IterativeTime creates a DebugTime(; mode=:Iterative)
• :Messages, :InfoMessages, :WarningMessages, and :ErrorMessages create the corresponding DebugMessages, where t[2] is used for the warn level (a Symbol such as :Always)

any other symbol s creates a DebugEntry(s) with the given format, to print that field of the solver state.

DebugFactory(a::Vector)

Generate a dictionary of DebugActions.

First all Symbols, Strings and DebugActions are collected, excluding :Stop, :WhenActive and any Int. This collected vector is added to the :Iteration => [...] pair. :Stop is added as a DebugStoppingCriterion to the :Stop => [...] pair. If necessary, these pairs are created

For each Pair of a Symbol and a Vector, the DebugGroupFactory is called for the Vector and the result is added to the debug dictionary's entry with said symbol. This is wrapped into the DebugWhenActive, when the :WhenActive symbol is present

Return value

A dictionary for the different entry points where debug can happen, each containing a DebugAction to call.

Note that upon initialization the :Start, :BeforeIteration, and :Iteration entries are called with iteration number 0 to reset them (and maybe already print), while the :Stop entry is called with -1, so that it is only reset.

If an Int k is present, all entries but :Start and :Stop are wrapped into a DebugEvery(k).

Examples

1. Providing a simple vector of symbols, numbers and strings like

   [:Iterate, " | ", :Cost, :Stop, 10]

   Adds a group to :Iteration of three actions (DebugIterate, DebugDivider(" | "), and DebugCost) as a DebugGroup inside an DebugEvery to only be executed every 10th iteration. It also adds the DebugStoppingCriterion to the :Stop entry of the dictionary.

2. The same can also be written a bit more precise as

   DebugFactory([:Iteration => [:Iterate, " | ", :Cost, 10], :Stop])

3. We can even make the stopping criterion concrete and pass Actions directly, for example to make the stop more concrete, we get

   DebugFactory([:Iteration => [:Iterate, " | ", DebugCost(), 10], :Stop => [:Stop]])

DebugGroupFactory(a::Vector)

Generate a DebugGroup of DebugActions. The following rules are used

1. Any Symbol is passed to DebugActionFactory
2. Any (Symbol, String) generates similar actions as in 1., but the string is used for format=, see DebugActionFactory
3. Any String is passed to DebugActionFactory
4. Any Function generates a DebugCallback.
5. Any DebugAction is included as is.

If this results in more than one DebugAction a DebugGroup of these is build.

If any integers are present, the last of these is used to wrap the group in a DebugEvery(k).

If :WhenActive is present, the resulting Action is wrapped in DebugWhenActive, making it deactivatable by its parent solver.

reset!(d::DebugTime)

reset the internal time of a DebugTime, that is start from now again.

stop!(d::DebugTime)

stop the internal time of a DebugTime, that is set the time to 0 (undefined)