LevenbergMarquardt(M, f, jacobian_f, p, num_components=-1)

Solve an optimization problem of the form

\[\operatorname{arg\,min}_{p ∈ \mathcal M} \frac{1}{2} \lVert f(p) \rVert^2,\]

where $f\colon\mathcal M \to ℝ^d$ is a continuously differentiable function, using the Riemannian Levenberg-Marquardt algorithm Peeters, Tech. Rep., 1993. The implementation follows Algorithm 1 Adachi, Okuno, Takeda, Preprint, 2022


  • M – a manifold $\mathcal M$
  • f – a cost function $F: \mathcal M→ℝ^d$
  • jacobian_f – the Jacobian of $f$. The Jacobian jacF is supposed to accept a keyword argument basis_domain which specifies basis of the tangent space at a given point in which the Jacobian is to be calculated. By default it should be the DefaultOrthonormalBasis.
  • p – an initial value $p ∈ \mathcal M$
  • num_components – length of the vector returned by the cost function (d). By default its value is -1 which means that it will be determined automatically by calling F one additional time. Only possible when evaluation is AllocatingEvaluation, for mutating evaluation this must be explicitly specified.

These can also be passed as a NonlinearLeastSquaresObjective, then the keyword jacobian_tangent_basis below is ignored


  • evaluation – (AllocatingEvaluation) specify whether the gradient works by allocation (default) form gradF(M, x) or InplaceEvaluation in place, i.e. is of the form gradF!(M, X, x).
  • retraction_method – (default_retraction_method(M, typeof(p))) a retraction(M,x,ξ) to use.
  • stopping_criterion – (StopWhenAny(StopAfterIteration(200),StopWhenGradientNormLess(1e-12))) a functor inheriting from StoppingCriterion indicating when to stop.
  • expect_zero_residual – (false) whether or not the algorithm might expect that the value of residual (objective) at mimimum is equal to 0.
  • initial_residual_values – the initial residual vector of the cost function f.
  • initial_jacobian_f – the initial Jacobian of the cost function f.

All other keyword arguments are passed to decorate_state! for decorators or decorate_objective!, respectively. If you provide the ManifoldGradientObjective directly, these decorations can still be specified


the obtained (approximate) minimizer $p^*$, see get_solver_return for details




LevenbergMarquardtState{P,T} <: AbstractGradientSolverState

Describes a Gradient based descent algorithm, with


A default value is given in brackets if a parameter can be left out in initialization.

  • x – a point (of type P) on a manifold as starting point
  • stop – (StopAfterIteration(200) | StopWhenGradientNormLess(1e-12) | StopWhenStepsizeLess(1e-12)) a StoppingCriterion
  • retraction_method – (default_retraction_method(M, typeof(p))) the retraction to use, defaults to the default set for your manifold.
  • residual_values – value of $F$ calculated in the solver setup or the previous iteration
  • residual_values_temp – value of $F$ for the current proposal point
  • jacF – the current Jacobian of $F$
  • gradient – the current gradient of $F$
  • step_vector – the tangent vector at x that is used to move to the next point
  • last_stepsize – length of step_vector
  • η – parameter of the algorithm, the higher it is the more likely the algorithm will be to reject new proposal points
  • damping_term – current value of the damping term
  • damping_term_min – initial (and also minimal) value of the damping term
  • β – parameter by which the damping term is multiplied when the current new point is rejected
  • expect_zero_residual – (false) if true, the algorithm expects that the value of residual (objective) at mimimum is equal to 0.


LevenbergMarquardtState(M, initialX, initial_residual_values, initial_jacF; initial_vector), kwargs...)

Generate Levenberg-Marquardt options.

See also

gradient_descent, LevenbergMarquardt



S. Adachi, T. Okuno and A. Takeda. Riemannian Levenberg-Marquardt Method with Global and Local Convergence Properties. ArXiv Preprint (2022).
R. Peeters. On a Riemannian version of the Levenberg-Marquardt algorithm. Technical Report 0011, VU University Amsterdam, Faculty of Economics, Business Administration and Econometrics (1993).