Debug Output
Debug output can easily be added to any solver run. On the high level interfaces, like gradient_descent
, you can just use the debug=
keyword.
Manopt.DebugAction
— TypeDebugAction
A DebugAction
is a small functor to print/issue debug output. The usual call is given by (amp::AbstractManoptProblem, ams::AbstractManoptSolverState, i) -> s
, where i
is the current iterate.
By convention i=0
is interpreted as "For Initialization only", i.e. only debug info that prints initialization reacts, i<0
triggers updates of variables internally but does not trigger any output. Finally typemin(Int)
is used to indicate a call from stop_solver!
that returns true afterwards.
Fields (assumed by subtypes to exist)
print
method to perform the actual print. Can for example be set to a file export,
or to @info. The default is the print
function on the default Base.stdout
.
Manopt.DebugChange
— TypeDebugChange(M=DefaultManifold())
debug for the amount of change of the iterate (stored in get_iterate(o)
of the AbstractManoptSolverState
) during the last iteration. See DebugEntryChange
for the general case
Keyword Parameters
storage
– (StoreStateAction( [:Gradient] )
) – (eventually shared) the storage of the previous actionprefix
– ("Last Change:"
) prefix of the debug output (ignored if you setformat
)io
– (stdout
) default steream to print the debug to.format
- ("$prefix %f"
) format to print the output using an sprintf format.inverse_retraction_method
- (default_inverse_retraction_method(M)
) the inverse retraction to be used for approximating distance.
Manopt.DebugCost
— TypeDebugCost <: DebugAction
print the current cost function value, see get_cost
.
Constructors
DebugCost()
Parameters
format
- ("$prefix %f"
) format to print the output using sprintf and a prefix (seelong
).io
– (stdout
) default steream to print the debug to.long
- (false
) short form to set the format tof(x):
(default) orcurrent cost:
and the cost
Manopt.DebugDivider
— TypeDebugDivider <: DebugAction
print a small div
ider (default " | "
).
Constructor
DebugDivider(div,print)
Manopt.DebugEntry
— TypeDebugEntry <: DebugAction
print a certain fields entry of type {T} during the iterates, where a format
can be specified how to print the entry.
Addidtional Fields
field
– Symbol the entry can be accessed with withinAbstractManoptSolverState
Constructor
DebugEntry(f; prefix="$f:", format = "$prefix %s", io=stdout)
Manopt.DebugEntryChange
— TypeDebugEntryChange{T} <: DebugAction
print a certain entries change during iterates
Additional Fields
print
– (print
) function to print the resultprefix
– ("Change of :Iterate"
) prefix to the print outformat
– ("$prefix %e"
) format to print (uses the `prefix by default and scientific notation)field
– Symbol the field can be accessed with withinAbstractManoptSolverState
distance
– function (p,o,x1,x2) to compute the change/distance between two values of the entrystorage
– aStoreStateAction
to store the previous value of:f
Constructors
DebugEntryChange(f,d)
Keyword arguments
io
(stdout
) anIOStream
prefix
("Change of $f"
)storage
(StoreStateAction((f,))
) aStoreStateAction
initial_value
an initial value for the change ofo.field
.format
– ("$prefix %e"
) format to print the change
Manopt.DebugEvery
— TypeDebugEvery <: DebugAction
evaluate and print debug only every $i$th iteration. Otherwise no print is performed. Whether internal variables are updates is determined by always_update
.
This method does not perform any print itself but relies on it's childrens print.
Constructor
DebugEvery(d::DebugAction, every=1, always_update=true)
Initialise the DebugEvery.
Manopt.DebugGradientChange
— TypeDebugGradientChange()
debug for the amount of change of the gradient (stored in get_gradient(o)
of the AbstractManoptSolverState
o
) during the last iteration. See DebugEntryChange
for the general case
Keyword Parameters
storage
– (StoreStateAction( (:Gradient,) )
) – (eventually shared) the storage of the previous actionprefix
– ("Last Change:"
) prefix of the debug output (ignored if you setformat
)io
– (stdout
) default steream to print the debug to.format
- ("$prefix %f"
) format to print the output using an sprintf format.
Manopt.DebugGroup
— TypeDebugGroup <: DebugAction
group a set of DebugAction
s into one action, where the internal prints are removed by default and the resulting strings are concatenated
Constructor
DebugGroup(g)
construct a group consisting of an Array of DebugAction
s g
, that are evaluated en bloque
; the method does not perform any print itself, but relies on the internal prints. It still concatenates the result and returns the complete string
Manopt.DebugIfEntry
— TypeDebugIfEntry <: DebugAction
Issue a warning, info or error if a certain field does not pass a check
Fields
io
– an io streamcheck
– a function that takes the value of thefield
as input and returns a booleanfield
– Symbol the entry can be accessed with withinAbstractManoptSolverState
msg
- is the check fails, this message is displayedtype
– Symbol specifying the type of display, possible values:print
,: warn
,:info
,:error
, where:print
prints toio
.
Constructor
DebugEntry(field, check=(>(0)); type=:warn, message=":$f is nonnegative", io=stdout)
Manopt.DebugIterate
— TypeDebugIterate <: DebugAction
debug for the current iterate (stored in get_iterate(o)
).
Constructor
DebugIterate()
Parameters
io
– (stdout
) default steream to print the debug to.long::Bool
whether to printx:
orcurrent iterate
Manopt.DebugIteration
— TypeDebugIteration <: DebugAction
Constructor
DebugIteration()
Keyword parameters
format
- ("# %-6d"
) format to print the output using an sprintf format.io
– (stdout
) default steream to print the debug to.
debug for the current iteration (prefixed with #
by )
Manopt.DebugMessages
— TypeDebugMessages <: DebugAction
An AbstractManoptSolverState
or one of its substeps like a Stepsize
might generate warnings throughout their compuations. This debug can be used to :print
them display them as :info
or :warnings
or even :error
, depending on the message type.
Constructor
DebugMessages(mode=:Info; io::IO=stdout)
Initialize the messages debug to a certain mode
. Available modes are
:Error
– issue the messages as an error and hence stop at any issue occuring:Info
– issue the messages as an@info
:Print
– print messages to the steamio
.:Warning
– issue the messages as a warning
Manopt.DebugSolverState
— TypeDebugSolverState <: AbstractManoptSolverState
The debug options append to any options a debug functionality, i.e. they act as a decorator pattern. Internally a Dict
ionary is kept that stores a DebugAction
for several occasions using a Symbol
as reference. The default occasion is :All
and for example solvers join this field with :Start
, :Step
and :Stop
at the beginning, every iteration or the end of the algorithm, respectively
The original options can still be accessed using the get_state
function.
Fields (defaults in brackets)
options
– the options that are extended by debug informationdebugDictionary
– aDict{Symbol,DebugAction}
to keep track of Debug for different actions
Constructors
DebugSolverState(o,dA)
construct debug decorated options, where dD
can be
- a
DebugAction
, then it is stored within the dictionary at:All
- an
Array
ofDebugAction
s, then it is stored as adebugDictionary
within:All
. - a
Dict{Symbol,DebugAction}
. - an Array of Symbols, String and an Int for the
DebugFactory
Manopt.DebugStoppingCriterion
— TypeDebugStoppingCriterion <: DebugAction
print the Reason provided by the stopping criterion. Usually this should be empty, unless the algorithm stops.
Manopt.DebugTime
— TypeDebugTime()
Measure time and print the intervals. Using start=true
you can start the timer on construction, for example to measure the runtime of an algorithm overall (adding)
The measured time is rounded using the given time_accuracy
and printed after canonicalization.
Keyword Parameters
prefix
– ("Last Change:"
) prefix of the debug output (ignored if you setformat
)io
– (stdout
) default steream to print the debug to.format
- ("$prefix %s"
) format to print the output using an sprintf format, where%s
is the canonicalized time`.mode
– (:cumulative
) whether to display the total time or reset on every call using:iterative
.start
– (false
) indicate whether to start the timer on creation or not. Otherwise it might only be started on firsr call.time_accuracy
– (Millisecond(1)
) round the time to this period before printing the canonicalized time
Manopt.DebugWarnIfCostIncreases
— TypeDebugWarnIfCostIncreases <: DebugAction
print a warning if the cost increases.
Note that this provides an additional warning for gradient descent with its default constant step size.
Constructor
DebugWarnIfCostIncreases(warn=:Once; tol=1e-13)
Initialize the warning to warning level (:Once
) and introduce a tolerance for the test of 1e-13
.
The warn
level can be set to :Once
to only warn the first time the cost increases, to :Always
to report an increase every time it happens, and it can be set to :No
to deactivate the warning, then this DebugAction
is inactive. All other symbols are handled as if they were :Always:
Manopt.DebugWarnIfCostNotFinite
— TypeDebugWarnIfCostNotFinite <: DebugAction
A debug to see when a field (value or array within the AbstractManoptSolverState is or contains values that are not finite, for example Inf
or Nan
.
Constructor
DebugWarnIfCostNotFinite(field::Symbol, warn=:Once)
Initialize the warning to warn :Once
.
This can be set to :Once
to only warn the first time the cost is Nan. It can also be set to :No
to deactivate the warning, but this makes this Action also useless. All other symbols are handled as if they were :Always:
Manopt.DebugWarnIfFieldNotFinite
— TypeDebugWarnIfFieldNotFinite <: DebugAction
A debug to see when a field from the options is not finite, for example Inf
or Nan
Constructor
DebugWarnIfFieldNotFinite(field::Symbol, warn=:Once)
Initialize the warning to warn :Once
.
This can be set to :Once
to only warn the first time the cost is Nan. It can also be set to :No
to deactivate the warning, but this makes this Action also useless. All other symbols are handled as if they were :Always:
Example
DebugWaranIfFieldNotFinite(:Gradient)
Creates a [DebugAction
] to track whether the gradient does not get Nan
or Inf
.
Manopt.DebugWhenActive
— TypeDebugWhenActive <: DebugAction
evaluate and print debug only if the active boolean is set. This can be set from outside and is for example triggered by DebugEvery
on debugs on the subsolver.
This method does not perform any print itself but relies on it's childrens print.
For now, the main interaction is with DebugEvery
which might activate or deactivate this debug
Fields
always_update
– whether or not to call the order debugs with iteration-1
in in active stateactive
– a boolean that can (de-)activated from outside to enable/disable debug
Constructor
DebugWhenActive(d::DebugAction, active=true, always_update=true)
Initialise the DebugSubsolver.
Manopt.DebugActionFactory
— MethodDebugActionFactory(s)
create a DebugAction
where
- a
String
yields the correspoinding divider - a
DebugAction
is passed through - a [
Symbol
] createsDebugEntry
of that symbol, with the exceptions of:Change
,:Iterate
,:Iteration
, and:Cost
. - a
Tuple{Symbol,String}
creates aDebugEntry
of that symbol where the String specifies the format.
Manopt.DebugActionFactory
— MethodDebugActionFactory(s::Symbol)
Convert certain Symbols in the debug=[ ... ]
vector to DebugAction
s Currently the following ones are done. Note that the Shortcut symbols should all start with a capital letter.
:Cost
creates aDebugCost
:Change
creates aDebugChange
:GradientChange
creates aDebugGradientChange
:GradientNorm
creates aDebugGradientNorm
:Iterate
creates aDebugIterate
:Iteration
creates aDebugIteration
:IterativeTime
creates aDebugTime
(:Iterative)
:Stepsize
creates aDebugStepsize
:WarnCost
creates aDebugWarnIfCostNotFinite
:WarnGradient
creates aDebugWarnIfFieldNotFinite
for the::Gradient
.:Time
creates aDebugTime
:WarningMessages
creates aDebugMessages
(:Warning)
:InfoMessages
creates aDebugMessages
(:Info)
:ErrorMessages
creates aDebugMessages
(:Error)
:Messages
creates aDebugMessages
()
(i.e. the same as:InfoMessages
)
any other symbol creates a DebugEntry(s)
to print the entry (o.:s) from the options.
Manopt.DebugActionFactory
— MethodDebugActionFactory(t::Tuple{Symbol,String)
Convert certain Symbols in the debug=[ ... ]
vector to DebugAction
s Currently the following ones are done, where the string in t[2]
is passed as the format
the corresponding debug. Note that the Shortcut symbols t[1]
should all start with a capital letter.
:Cost
creates aDebugCost
:Change
creates aDebugChange
:GradientChange
creates aDebugGradientChange
:Iterate
creates aDebugIterate
:Iteration
creates aDebugIteration
:Stepsize
creates aDebugStepsize
:Time
creates aDebugTime
:IterativeTime
creates aDebugTime
(:Iterative)
any other symbol creates a DebugEntry(s)
to print the entry (o.:s) from the options.
Manopt.DebugFactory
— MethodDebugFactory(a)
given an array of Symbol
s, String
s DebugAction
s and Ints
- The symbol
:Stop
creates an entry of to display the stopping criterion at the end (:Stop => DebugStoppingCriterion()
), for further symbols seeDebugActionFactory
- The symbol
:Subsolver
wraps alldictionary
entries withDebugWhenActive
that can be set from outside. - Tuples of a symbol and a string can be used to also specify a format, see
DebugActionFactory
- any string creates a
DebugDivider
- any
DebugAction
is directly included - an Integer
k
introduces that debug is only printed everyk
th iteration
Return value
This function returns a dictionary with an entry :All
containing one general DebugAction
, possibly a DebugGroup
of entries. It might contain an entry :Start
, :Step
, :Stop
with an action (each) to specify what to do at the start, after a step or at the end of an Algorithm, respectively. On all three occastions the :All
action is exectued. Note that only the :Stop
entry is actually filled when specifying the :Stop
symbol.
Example
The array
[:Iterate, " | ", :Cost, :Stop, 10]
Adds a group to :All
of three actions (DebugIteration
, DebugDivider
with " | "
to display, DebugCost
) as a DebugGroup
inside an DebugEvery
to only be executed every 10th iteration. It also adds the DebugStoppingCriterion
to the :Stop
entry of the dictionary.
Manopt.reset!
— Methodreset!(d::DebugTime)
reset the internal time of a DebugTime
, that is start from now again.
Manopt.stop!
— Methodstop!(d::DebugTime)
stop the reset the internal time of a DebugTime
, that is set the time to 0 (undefined)
Technical Details: The Debug Solver
The decorator to print debug during the iterations can be activated by decorating the state of a solver and implementing your own DebugAction
s. For example printing a gradient from the GradientDescentState
is automatically available, as explained in the gradient_descent
solver.
Manopt.initialize_solver!
— Methodinitialize_solver!(amp::AbstractManoptProblem, dss::DebugSolverState)
Extend the initialization of the solver by a hook to run debug that were added to the :Start
and :All
entries of the debug lists.
Manopt.step_solver!
— Methodstep_solver!(amp::AbstractManoptProblem, dss::DebugSolverState, i)
Extend the i
th step of the solver by a hook to run debug prints, that were added to the :Step
and :All
entries of the debug lists.
Manopt.stop_solver!
— Methodstop_solver!(amp::AbstractManoptProblem, dss::DebugSolverState, i)
Extend the check, whether to stop the solver by a hook to run debug, that were added to the :Stop
and :All
entries of the debug lists.