Edit on GitHub

Solvers

Solvers can be applied to AbstractManoptProblems with solver specific AbstractManoptSolverState.

List of Algorithms

The following algorithms are currently available

SolverFunction & StateObjective
Alternating Gradient Descentalternating_gradient_descent AlternatingGradientDescentState$f=(f_1,\ldots,f_n)$, $\operatorname{grad} f_i$
Chambolle-PockChambollePock, ChambollePockState (using TwoManifoldProblem)$f=F+G(Λ\cdot)$, $\operatorname{prox}_{σ F}$, $\operatorname{prox}_{τ G^*}$, $Λ$
Conjugate Gradient Descentconjugate_gradient_descent, ConjugateGradientDescentState$f$, $\operatorname{grad} f$
Cyclic Proximal Pointcyclic_proximal_point, CyclicProximalPointState$f=\sum f_i$, $\operatorname{prox}_{\lambda f_i}$
Douglas–RachfordDouglasRachford, DouglasRachfordState$f=\sum f_i$, $\operatorname{prox}_{\lambda f_i}$
Exact Penalty Methodexact_penalty_method, ExactPenaltyMethodState$f$, $\operatorname{grad} f$, $g$, $\operatorname{grad} g_i$, $h$, $\operatorname{grad} h_j$
Frank-Wolfe algorithmFrank_Wolfe_method, FrankWolfeStatesub-problem solver
Gradient Descentgradient_descent, GradientDescentState$f$, $\operatorname{grad} f$
Levenberg-MarquardtLevenbergMarquardt, LevenbergMarquardtState$f = \sum_i f_i$ $\operatorname{grad} f_i$ (Jacobian)
Nelder-MeadNelderMead, NelderMeadState$f$
Augmented Lagrangian Methodaugmented_Lagrangian_method, AugmentedLagrangianMethodState$f$, $\operatorname{grad} f$, $g$, $\operatorname{grad} g_i$, $h$, $\operatorname{grad} h_j$
Particle Swarmparticle_swarm, ParticleSwarmState$f$
Primal-dual Riemannian semismooth Newton Algorithmprimal_dual_semismooth_Newton, PrimalDualSemismoothNewtonState (using TwoManifoldProblem)$f=F+G(Λ\cdot)$, $\operatorname{prox}_{σ F}$ & diff., $\operatorname{prox}_{τ G^*}$ & diff., $Λ$
Quasi-Newton Methodquasi_Newton, QuasiNewtonState$f$, $\operatorname{grad} f$
Steihaug-Toint Truncated Conjugate-Gradient Methodtruncated_conjugate_gradient_descent, TruncatedConjugateGradientState$f$, $\operatorname{grad} f$, $\operatorname{Hess} f$
Subgradient Methodsubgradient_method, SubGradientMethodState$f$, $∂ f$
Stochastic Gradient Descentstochastic_gradient_descent, StochasticGradientDescentState$f = \sum_i f_i$, $\operatorname{grad} f_i$
The Riemannian Trust-Regions Solvertrust_regions, TrustRegionsState$f$, $\operatorname{grad} f$, $\operatorname{Hess} f$

Note that the solvers (their AbstractManoptSolverState, to be precise) can also be decorated to enhance your algorithm by general additional properties, see debug output and recording values. This is done using the debug= and record= keywords in the function calls. Similarly, since 0.4 we provide a (simple) caching of the objective function using the cache= keyword in any of the function calls..

Technical Details

The main function a solver calls is

which is a framework that you in general should not change or redefine. It uses the following methods, which also need to be implemented on your own algorithm, if you want to provide one.

Manopt.initialize_solver! — Function
initialize_solver!(ams::AbstractManoptProblem, amp::AbstractManoptSolverState)

Initialize the solver to the optimization AbstractManoptProblem amp by initializing the necessary values in the AbstractManoptSolverState amp.

source
initialize_solver!(amp::AbstractManoptProblem, dss::DebugSolverState)

Extend the initialization of the solver by a hook to run debug that were added to the :Start and :All entries of the debug lists.

source
initialize_solver!(ams::AbstractManoptProblem, rss::RecordSolverState)

Extend the initialization of the solver by a hook to run records that were added to the :Start entry.

source
Manopt.step_solver! — Function
step_solver!(amp::AbstractManoptProblem, ams::AbstractManoptSolverState, i)

Do one iteration step (the ith) for an AbstractManoptProblemp by modifying the values in the AbstractManoptSolverState ams.

source
step_solver!(amp::AbstractManoptProblem, dss::DebugSolverState, i)

Extend the ith step of the solver by a hook to run debug prints, that were added to the :Step and :All entries of the debug lists.

source
step_solver!(amp::AbstractManoptProblem, rss::RecordSolverState, i)

Extend the ith step of the solver by a hook to run records, that were added to the :Iteration entry.

source
Manopt.get_solver_return — Function
get_solver_return(O::AbstractManoptSolverState)

determine the result value of a call to a solver. By default this returns the same as get_solver_result, i.e. the last iterate or (approximate) minimizer.

get_solver_return(O::ReturnSolverState)

return the internally stored state of the ReturnSolverState instead of the minimizer. This means that when the state are decorated like this, the user still has to call get_solver_result on the internal state separately.

source